Postmodern I/O

Update: As it turns out, get_line is intended primarily for writing interactive tty character-by-character apps. On the erlang-questions list, I think, someone on the implementation team announced they're updating the performance guides. And now I assume the i/o and regexp implementations. End.

Tim Bray's note to Erlang...

I like you. Really, I do. But until you can read lines of text out of a file and do basic pattern-matching against them acceptably fast (which most people would say is faster than Ruby), you’re stuck in a niche; you’re a thought experiment and a consciousness-raiser and an engineering showpiece, but you’re not a general-purpose tool. Sorry

I've never had to do a lot of really fast I/O in Erlang, so Tim's excercise (and Steve Loughran's a while back) have been useful for me.

Fortunately for Erlang, making improvements to the I/O and perhaps the regexp libraries should be a fair bit easier than making concurrency and distributed system improvements in other languages.

If I had to do a lot of really fast I/O and Erlang did not pan out for that, I would probably turn to Gambit Scheme. In fact Gambit can do really nice Erlang-like concurrency as well as really fast I/O. It just doesn't have all the OTP libraries that Erlang has.

Maybe Gambit will get there some day. Maybe someone will port Ruby to Gambit, so Ruby can run really fast and have really fast I/O too. And have really nice concurrency. All without building a new virtual machine from scratch or building on top of the JVM.

(If you're interested in that, let me know. I just don't care enough for the cruft in Ruby to write an implementation of it for my own use. There're a helluva lot of good things that fall out of this approach, like secure multiple application spaces per OS process, multiple languages with message-passing integration per OS process, "engines" that can be metered, pet-named secure access to resources, etc. OK -- that's its own blog post.)

If I had to do a lot of really fast I/O in the context of a reliable, scalable distributed system, I would probably do the really fast I/O in Gambit Scheme connected to an Erlang external port. Or this could be Python or Ruby if you wanted something from those systems.

For example instead of implementing a rule engine in Erlang, I'd integrate from Erlang to PyClips (a really nice integration of Python and the Clips rules engine) like this. That seems like the way to develop postmodern systems... use good tools for appropriate situations, especially if they are built to be integrated easily. Programming today of any size probably leads to multiple languages, by its very nature.

Meanwhile Tim's I/O results have been taken to the erlang-questions list. With those numbers there seems to be some low hanging fruit that may or may not require some underlying systems coding. Already solutions are pouring in just using a different approach than the apparently slow get_line.