Distributed Build Architecture

Vincent Massol proposes a distributed build architecture using JavaSpaces...

I personally prefer this architecture over one where the orchestrator directly sends build requests to build agents as I find it more scalable and more flexible.

Yeah, and robust as well. Just watch for that place where the build agent does a 'read' for a request. It should be a 'take'. Before the take though it should 'write' it's intent to take perhaps including an estimated time of completion. Then if it fails, another process can monitor the results and restore the request if it suspects or detects the agent failed to complete.

If the request included an estimate of time to build then build agents can predict their loads and which requests they should accept. These estimates can come from actual builds and improve themselves over time. Many possibilities for coordinated processes to evolve on their own through "metadata".

Would you like such a distributed build system?

Actually I'd rather use tools that did not require such massive builds, but the idea applies generally to many kinds of transformation activities. This is an ideal example of services built using simple, general coordination mechanisms.

You can build *all* kinds of coordination patterns using the simple Linda-like vocabulary.