Comments on Processes, Etc.

Le Roux Bodenstein writes in email...

Some comments on some of your recent posts: You were talking a lot about scaling to more processors, app servers becoming the new operating systems, etc.

Aren't app servers just big processes that manage large amounts of threads? Don't multiple processes scale more easily because of "share nothing"? What's wrong with operating systems? They are basically (among a few other things) very efficient process schedulers. (modern unixes, at least)

"Shared nothing" is important. If the multiplexor does not enforce shared nothing then convention should. The new multiplexor (e.g. the web/app server) is lighterweight than the older OS multiplexor. Shared nothing lightweight processes are the best of both worlds.

I think if anything we should work on making inter-process communication easier. Processes should be tied closer into languages and inter-process communication should be abstracted. Even more than some newer languages did with threads.

I agree. But we need to lose the distinction between local and remote processes. e.g. Erlang processes us the same simple communication abstraction whether they are co-located or remote. So do HTTP processes of course, and XMPP, and... Perhaps the OS is the last bastion of the local/remote distinction? (The now defunct Apollo Domain OS being one of the rare exceptions.)

I think "lightweight processes and threads" will lose popularity. Multiple cores and processors effectively make the "heavy-weight overhead of processes" a mute point. There's a lot less context switching when things run on their own processors and once a process is started (which is as far as I know quite expensive compared to starting a thread, but I am not an expert) it can just keep running until it is needed.

I agree heavyweight processes will perform better, but lightweight processes will perform that much better. The OS process approach can and I am sure will be improved but making improvements there is probably a longer road than taking the Erlang approach in a specific web/app server or language runtime implementation.

Anyway... my argument is I don't think app servers will help much on operating systems with decent kernels. In fact - they will just be in the way. Or an extra layer. Or what little use they have can be added to the os.

I think this is a fine way to approach the problem but as I said I think it is a longer road. Hopefully the OS designers will move in this direction, having taken some best practices from the web/app server and language runtime designers. (For that matter hopefully the web/app server and language runtime designers will take some best practices from their peers.)

I don't think we need new programming languages either. I think Python, Ruby, etc. can probably just be "fixed" to make multiple-process development a lot easier...

We continue to be in agreement.