"I have a mind like a steel... uh... thingy." Patrick Logan's weblog.

Search This Blog

Wednesday, May 09, 2007

Take Five: Processes

Guido van Rossum (via Joe Gregario) (via Sean McGrath) on "threads"...

The difference is, for an OS kernel, there really isn't any other way to benefit from multiple CPUs. But for Python, there is -- run multiple processes instead of threads!
Yeah, I don't like threads one bit.

How much support does Python have (in the distribution or from other contributions) for starting and stopping processes on the same or other machines? And for communicating with them?

I am all for this approach, but little attention is being paid that I know of to programming this way, regarding "all the little things" that are needed to develop, test, and deploy concurrent, distributed systems. Not to mention "post modern" systems using various languages.

The "common language runtime folks" (clr/dlr and jvm, etc.) are salivating over using each others classes. We should be more interested in using each others processes.

Just because Java was once aimed at a set-top box OS that didn't support multiple address spaces, and just because process creation in Windows used to be slow as a dog, doesn't mean that multiple processes (with judicious use of IPC) aren't a much better approach to writing apps for multi-CPU boxes than threads.
Even in the same address space it's better to pretend you're not. Java applets, etc. were aimed at the everyday developers, so why did the language designers give them such things as threads and locks? More out of habit and lack of questioning their hidden assumptions than anything else, I'd bet.

Yeah, it will be a good thing when we can do all our cross-platform, distributed process management stuff in IronicPython and/or Jython! (This might have been more interesting even five years ago. Now we know we could have been, and should be running in the *opposite* direction. Instead of running at full speed, we're still docked, with the steamliner facing the *wrong* direction.)

"Common Language Runtimes" -- phooey. Think broader. Patrick Mueller points to his investigations with Java in a comment to this post. Is Microsoft's CLR/DLR going to do a better job at cross-platform, distributed process management???

4 comments:

Ian Bicking said...

There's a few things in Python for multiple processes. It's not nearly as good as it should be, but in the last year or so it's gotten better. I guess the things I can think of are py.exec (IPC over ssh or other transports), processing (thread-like, I think uses forking), and then there's a shared queue/heap whose name I can't remember (some woman's name?) but I've heard good things about.

Patrick Mueller said...

How much support does Python have (in the distribution or from other contributions) for starting and stopping processes on the same or other machines?

Here's my latest rant on Java's process facilities:

http://tinyurl.com/27yey3

I recently had to write some process launching code, and literally gave up doing it in Java. It's so painful, and there's little useful function there anyway. I rewrote that bit in a few lines of Python. Already knew the basics of fork/exec and piping bits, but this post was useful:

http://www.myelin.co.nz/post/2003/3/13/

Patrick Logan said...

Yeah, it will be a good thing when we can do all our cross-platform, distributed process management stuff in IronicPython and/or Jython!

"Common Language Runtimes" -- phooey.

Hyakugei said...

the shared queue/heap thing is PyLinda (http://www-users.cs.york.ac.uk/~aw/pylinda/), an implementation of Tuple Spaces (http://en.wikipedia.org/wiki/Tuple_space).

Blog Archive

About Me

Portland, Oregon, United States
I'm usually writing from my favorite location on the planet, the pacific northwest of the u.s. I write for myself only and unless otherwise specified my posts here should not be taken as representing an official position of my employer. Contact me at my gee mail account, username patrickdlogan.