Hacker Read top | best | new | newcomments | leaders | about | bookmarklet login

Thanks for the tip on fabricate, it appears to be just the tool I was looking for. A light weight dependency based "build" system in python that is not focused on distutils or making python packages. We shall see how well it works for data processing.


sort by: page size:

After being a bit frustrated with both command line tools such as Fabric as well as build utilities like SCons, I thought I'd spend a bit of time thinking about what would make an awesome build utility for Python. I modeled a lot of it after Rake, due to its immense popularity. However, I distinctly tried to make things more explicit and more Pythonic. I created an example script that would be fed to this new tool.

http://pastebin.com/8j8FUYSd

I'd like to hear any feedback you might have on this. Would you like a tool such as this? What do you like or dislike about this format? What features do you really like from Fabric, SConstruct, or Rake?


FYI, this is an early-stage Python build system that's sort of buildout-like, but more transparent and less "enterprisey".

Bear in mind that this is the "minimum viable product". I'm looking for feedback in terms of what's a pain about it and what features would be cool to add to it.


Can you give practical examples of what you are mentioning? Like, how to achieve ""static builds"" with python? Got me interested!

The good thing is if you aren't FAANG scale, you probably don't need to write anything that involves significant build process hassle. If you do, you probably have enough people and a good process to kinda get by.

Someone has almost certainly packaged the thing you want with Python bindings.


A python dependency is unfortunately out of the question for me, but that is a really cool project.

There's also Flyte, which is natively data aware and schedules tasks around data dependencies. The syntax is essentially pure python too.

Years ago, someone showed me a python tool for build automation that was written in an imperative style although it was declarative underneath. Like a builder pattern. That looked pretty promising, but I have misremembered then name and so can’t find it again.

I’d rather have something like that for infrastructure management.


fabric is a tool that's using python as an embedded language; i don't really consider it python any more as i consider Makefile python.

Maybe make, python and pip. From what I gather this is a node wrapper it's actually python that runs the model

Jasper is another cool project that has a bit better of a foundation. You can make little Python modules for pretty much anything.

Python is pretty great for rapidly putting together prototypes. You ever give that a shot?

Python is for build tooling I think

I use pybuilder to setup Python projects. With it, you can have unit testing, linting, packaging, costumized tasks, plugin, etc.

Neat, definitely something I would want to give a try.

I see though that it only supports pure python for building packages, does that mean that it doesn't build if you are dependent on compiled libraries?

Is there also a plan to add some of the functionality of bundling tools such as web-pack into this build phase? like automated css optimization, image compression... Could be handy for some django/flask projects.


Python doesn't require creating artifacts. It requires dependency resolvement which Docker is perfect for.

I write my own automation/build tools in Python

Envoy pynotify and fnmatch pretty much do the heavy lifting.

I do it this way because

    Since I know Python no extra syntax knowledge is required

    Python is going to be readable to me in 6 months when I've 
    forgotten the syntax of whatever build tool I just used

    *All* my machines desktop and servers already have Python 
    installed
    
    *It's fast, merging 20-30 largish text files takes around 
    10-15milliseconds

    * It's very very flexible (it's Python)
I used make extensively when I was at Uni and quite frankly if I never have to touch it again I would be a happy bunny.

It's an incredibly powerful and clever tool hiding behind an interface (that second to Git) is the worst I've ever had to use.

EDIT: I've actually been toying with the idea of writing a python library to simplify things as lots of the code I end up writing is very similar across projects, it's been one of my "think about in shower" projects for quite a while.


The tool might built in Python, but it doesn't really expose the python api to the user, which (among other things) makes for super clean build files.

It's also very fast since it uses Ninja for the actual build phase.


Given your background, consider trying out hylang, which transpiles to Python and can use Python libraries. I played around with it a few times a couple years ago, but never developed a serious project due to limited tooling. The situation might have since improved. I've considered putting together an LSP server for it because it'd fun to use for personal projects and scripting.

Sounds wonderful compared to dealing with Python tooling.
next

Legal | privacy