Re: Question about message passing paradigm

Responding to a thread on the erlang-questions list...

The problem we are discussing is processes B, C, D hold information X, Y, Z respectively; process A wants a coherent snapshot of X, Y,Z.

There are actually two slightly different cases depending on A needs "X Y Z as of *now*" (A, B, C, and D must all synchronise), or A needs "X Y Z as of *some* time in the recent past" (B, C, and D must all synchronise but can then send the information to A without waiting for A to receive it).

I like this problem because it is simple yet subtle. One way that it is subtle is that in "multithread" programming most people STILL think in terms of a single absolute time shared by all threads. (After all, there _is_ such a thing, the system clock. And yes, it's not exactly true, but it's close enough to make people _think_ it's true.) But when you start thinking about Erlang and especially *distributed* Erlang, you start realising that "now" is a pretty fuzzy concept.

Yes, the problem seems simple yet subtle. The downside is there are many unwritten constraints (or not) on any specific problem that could lead the solution alternatives one way or another. Unless you want to really dig into those, then the cost/benefit of one solution or another could be more or less off.

e.g. why not coordinate through an in-memory database? This could be reasonable, or not. We don't know enough.

Why not schedule the source processes to send a message on a periodic or scheduled basis? This could be reasonable, or not, and cut down the message traffic, which seemed to be a concern.

Why is sending fewer than N messages a concern? Why does one process have to collect the information? How much information? How tight is the deadline? Is "now" an actual timestamp or just some unknown point in time that a request has been received? How close to "now" do the other "nows" have to be with respect to each other? Can you widen that window if it would decrease the effort to build?

If synchronization across the processes is needed then is an "eventually consistent" approach reasonable if it lowers the effort to build?

Interesting stuff, but challenging to talk about in when the details are too abstract.

Thoughts on Hotels and Coffee in Belfast

As long as I'm writing down my thoughts on this trip: I've been in two hotels in Belfast, the Hilton and the Radisson. Both are nice, close to the city center, and on the water.

They both have internet frustrations: neither have wireless in the rooms; the Hilton charges for wired internet in the rooms at ₤15 per 24 hours! When you pay that fee then they also throw in wireless _in the lobby_! Well, it's BT and the voucher is good for any BT provider in that 24 hour period. I found BT is the provider around some nearby coffee shops.

The Radisson at least provides free, but wired, internet. The problem then is the MacBook's adapter for UK plugs do not fit in the limited space above the immovable desk provided in the room! The nearest outlet that provides enough clearance for the adapter is across the room on the other side of the bed. If I had a 25' extension cord for either electricity or the internet, then this might be less frustrating.

(Let me update this with a warning: if you intend to use the internets with any expectation of getting bits back to your laptop, avoid the Radisson altogether. Band-width-wise this turned out to be simply unusable for long stretches.)

And so on to coffee: Caffe Nero is the best I've found in the city center so far, and the best I've found anywhere for a large chain. http://www.caffenero.com/

Thoughts on Continental Airlines

I've flown Continental Airlines twice this year-to-date. Thrice if you count the flight that was actually run by Northwestern, but I'd rather discount that one.

Continental has to be my top choice, for the food service alone. They serve hot meals, and a fair bit of it with each meal, that actually taste like real food.

I do think they recently took another inch out of the leg room though.

Thoughts on the Newark Airport

I can think of several airports to spend over a quarter of a day in that might even be somewhat pleasant. My own PDX would be high on that list.

Not making the cut would be Newark, NJ. Has this airport been even considered for remodeling since I was last here, about 25 years ago???

The most curious difference: the bars are run by the breweries. And so there is a "Miller's" bar and a "Budweiser" bar. Thankfully there is also a "Sam Adams" bar.

I guess folks in the NYC area do not hate _all_ things about Boston.

But what happens when you are traveling with a party with varying tastes?

George Carlin - Modern Man



The eBay Reporting Provision: Apparently Just More Of The Same

Steve Dekorte relays information about a last-minute provision in the housing bill, and writes...

So now if you buy a bong on your credit card, the feds will know and may have probable cause to invade your home.

But I think most of our bong purchases are safe because there is a $10k USD minimum for transactions to be reported. :0

Another thing I learned: I called Senator Dodd's office and was told this provision is an enforcement mechanism for a law already in place. Apparently these transactions already are required to be reported, but many are not. And so this provision is intended to collect more dough.

Not that that excuses the invasion of privacy, but it's just not going to affect as many people because of the minimum reporting, and the invasion of privacy already is law. I called both my senators, Wyden and Smith, to oppose the provision.

Capitol Hill switchboard: 1-866-928-3035.

Belfast

I head out for Belfast on Wednesday. Mostly business, some time with friends, and some time to wander around. Any suggestions?

I'll probably go on some walking tours of the city, and I think there's a bus that goes to Giant's Causeway.

George Carlin, Class Clown

I remember the first couple of rock albums that I bought as a teenager in the early-mid 1970s: The Who's "Who's Next", Aerosmith's "Toys in the Attic". I was not a musician and could only muster up a bit of air guitar in my room, by myself. I wasn't about to let loose in front of friends.

But *comedy* was something else. There were two comedy albums that everyone I ran with *had* to own. There was no pressure: we just all owned them. And we all memorized every line of every track. And we all repeated them to each other over and over. We weer all comedians.

One of those albums was Cheech and Chong's "Big Bambu".

The other one was George Carlin's "Class Clown".

This was the first time I heard his "Seven Dirty Words" bit. There's at least one word I _learned_ from that list. I very well may have peed my pants once or twice listening to that list.

http://en.wikipedia.org/wiki/Class_Clown

George Carlin. Here's to you...

1. Shit - The bird shit on the statue.
2. Piss - I have to piss like a race horse.
3. Fuck - Fuck you.
4. Cunt - She has a beautiful cunt.
5. Cocksucker - Go to hell, you cocksucker.
6. Motherfucker - You are a motherfucker.
7. Tits - Hey, nice tits.

Where You Find It

The Portland Tribune has a story about a local boy dying after having
been ill (shortness of breath, stomach cramps) for a week. His family
believes in "faith healing" and does not believe in western medicine
or apparently any medicine. (What would they consider food?)

http://www.portlandtribune.com/news/story.php?story_id=121379440855614400

Religion is what it is for each individual and family. In this case I
don't think the boy was given a choice, let alone the tools to make up
his own mind. That's not unusual. Most religion is passed down I
suspect.

As a Type I diabetic who knows almost nothing about health or
medicine, I have to say what little description this article provides
of the boy's symptoms, and the duration of them before his death,
*could* indicate Type I diabetes. This is a very manageable illness,
and whatever the true cause of death, the week in which the family
could have sought treatment but did not, was tragically long.

The boy also happens to be the cousin of a baby girl who also died
after the family refused medical treatment. Does this family simply
rationalize that their god does not love them? Or does not love these
specific kids? Or that everyone just dies when they die? (Which again
makes me wonder, why does anyone in their family eat meals?)

Filed under "apparently unnecessary tragedies to children, but I don't
understand".

SVG and Flash

Steve Dekorte wonders...

"Considering that ActionScript is really just Javascript with some
rendering APIs, once SVG is ubiquitous (effectively taking care of the
rendering API), will there be much point to Flash?"

http://www.dekorte.com/blog/blog.cgi?do=item&id=3468

(Aside: ActionScript is _worse_ than JavaScript, as it is a kind of
JavaScript+Java-semi-static-typing hybrid. Anyway...)

What Steve's really getting at is SVG vs. Flash. Flash goes well
beyond rendering, into multimedia, interactivity, environment/network
integration, etc. SVG has some support for JavaScript callbacks and
not much interactivity beyond that. SVG currently only (mainly?) runs
in some browsers, with all the pro's and con's. (I think Adobe's SVG
engine used to run stand-alone. Maybe others do as well?)

Flash could be a good foundation for an SVG engine. SVG could be a
good foundation for a Flash-like replacement.

I'd like to see SVG advance into a more fully capable engine for
*interactive* structured graphics, and beyond that a more fully
capable platform for many kinds of applications. SVG could be our best
hope for moving beyond the browser per se.

Those foundations require a simpler language than JavaScript and a
better security model than the browser or even Flash. The
interactivity model should provide a mix of
declarative/constrain/dataflow as well as imperative (i.e. callbacks
and mutable data, e.g. display lists).

The Emerging Enterprise in the Cloud

Jon Rose writes...

I have been working on a case study with Claude Courbois of NASDAQ for a few months now. We finally finished it up and published it over at InfoQ.com. NASDAQ has a very cool application, Market Replay, which utilizes AIR (desktop) & S3 (data). I am personally fascinated by how much the world has changed, as the platforms allowed NASDAQ to roll out an enterprise application with very minimal hardware infrastructure...

These technologies... I believe that their innovation will really mean more opportunities... They are enabling us to bring to market many ideas that were previously too difficult or costly to implement.

Or even just ideas too time-consuming too implement. Development groups with an idea can go and grab a fair bit of infrastructure without all the hoopla needed to requisition all the pieces within an IT data center. If the ideas prove out then in some cases they could or should be brought back "in-house". In other cases, they may be fine in the cloud.

Erlang Supervisor Hierarchies in Scala

Initial attempts to duplicate Erlang in other languages tend to make a lot of simplifying assumptions, either knowingly or unknowingly. Distribution is a big feature often overlooked initially, and beyond that "supervisor hierarchies" in which various distributed processes are either workers (do the real work) or supervisors (ensure the real work continues in the face of process or node failures).

Jonas has started an implementation of supervisor hierarchies for Scala's agents, as well as a basic "generic server" framework approximating Erlang's gen_server, etc. So far I think it's all running in one JVM, but it's a good start at illustrating all that Erlang does on behalf of a distributed system.

Above the Fray

Steve Yegge relays a fracas about language interoperability...

He [Chris] asked this really innocent question. He goes, "You know, I don't really know much about this stuff..."

Which is bad, you know. When somebody says that to you at Foo Camp, it means they're settin' you up.

He says, "So how do these languages talk to each other?"

And the room just erupted! It was chaos. All these people are like, "Oh, it's easy!" And the rest of them are like "No, it's hard!" And they're arguing, and arguing, and arguing. They argued for an hour.

And then they stood up, still arguing, and they kept talking about it, heading into the dinner tent. And they sat down, going at it for like three hours.

It was chaos.

Because some people were saying, "Well, you know, if Ruby's gonna call Python, well, uh, you just call, right? You just share the same stack, right?"

And the others are like, "Well, what about different calling conventions? What if they support optional vs. non-optional arguments? What if they support default arguments? What about the threading model? What about the semantics of, you know, the this pointer? What about all this stuff?"

And they're like (waving hands) "Ooooh, we'll gloss over it, gloss over it, smooth it over." And the reply is: "You can't. This is fundamental. These languages work differently!"

And oh my god, it was really interesting. And it was also very clear that it's ten years of research and implementation practice before they get this right. Before you'll be able to have a stack of calls, where you're calling from library to function, library to function in different languages.

So today, VMs are good for interoperability, but you've gotta use a bridge.

I thought this was going to be interesting, then I found the conversation was about language interoperability *within* a VM.

Y2K+8

Douglas Purdy of MSFT...

"One of the reasons this topic comes up in Microsoft circles is the
fact that so few Microsoft desktop applications use the CLR. As you
may be aware, there is a performance cost associated with running code
in the CLR and garbage collection often gets painted as the chief
offender."

http://douglaspurdy.com/2008/06/12/objective-c-20-garbage-collection/

All of a sudden...

...the evening is perfect, the city is shiny, the Timbers are winning. May has arrived, and it's only June!

Web Too Oh

Douglas Crockford writes...

"In the long term, we need to completely replace JavaScript and the
DOM with something designed to be safe from the ground up."

...noting that Firefox, and any browser generally, is a horrible
platform for applications.

http://blog.360.yahoo.com/blog-TBPekxc1dLNy5DOloPfzVvFIVOWMB0li?p=865

I think...

...it's about to snow.

Mornin'

Back in the 70s Jimmy Carter put a solar water heater on the roof of the White House, mostly as a symbol for a new direction of conservation and energy independence. Reagan promptly tore it down as a symbol of, well, he ran on the notion of bringing us "Morning in America", implying we have no reason to feel bad: we should feel "good" again.

We're Americans. We should not have bad news, and we should not "sacrifice" anything, well, at least not any comforts. Besides that the energy industry hated the idea of conservation and energy independence.

Carter also set the objective that America not become more dependent on foreign oil than we were at that time, 30 years ago now. Reagan convinced us we didn't need a pessimist like that.

Mornin'.

gosh, try goosh

(via Stefan Tilkov)

http://goosh.org/

e.g. type "place portland oregon (enter)"

Dumbest Name Ever From Apple

"Mobile Me" --- are you f'ing kidding me???

Bah. http://www.apple.com/mobileme/

You can also get there from http://www.me.com/

Joy. So now I can say to my wife, upload those photos from your camera to "Mobile Me" then bring them down to your "My Documents".

Oh, wait. Could you please bring them down to *my* "My Documents"?

Gawd. "Live Mesh" seems perfectly sexy compared to "Mobile Me".

Toward a New Fast Squeak VM: Eliot Miranda's Blog

Eliot Miranda is one of those really smart VM implementors. He's been working on Smalltalk VMs for a long time, including one of the best: VisualWorks. After a brief sidetrack with the (fairly uninteresting IMO) Newspeak project, he's now at work on a better VM for Squeak. He's got a blog about that.

Does the history of computing matter?

(via Bill de hÓra)

If I were the author of a significant IDE product I would take a fair bit of time to understand the history of my marketplace. Apparently Dmitry Jemerov does not follow that philosophy. I guess that has not hurt his success any. So I have to wonder how much the history of our field matters to its future.

Consider the following image of a Symbolics hypertext system from the mid-late 1980s. Who noticed at the time? Who would've believed such a thing was possible then or even now using a "dynamic language"? 150 years from now what does anyone need to remember? A couple of blips maybe: Netscape and Excel?

Dmitry writes...

Groovy is interesting, but it's a dynamic language and has all the problems of dynamic languages. It's OK for small pieces of code. But building something the size of IntelliJ IDEA [in a dynamic language] would be a complete nightmare.

So let's consider history's demonstration that this is false.

• GNU Emacs - Lisp
• Zmacs - a predecessor to GNU Emacs, Lisp, running on the Lisp machine
• Various Lisp machine operating systems including Symbolic's Genera, one from Texas Instruments based on one from MIT, one (at least) from Xerox
• Symbolic's Concordia authoring tool and Document Examiner hypertext system - em, Lisp again
• Smalltalk tools, which I've written about several times already

As Patrick Mueller writes...

Since Smalltalk I've generally been using Eclipse for work stuff, but the effort in extending the environment has just been too much work for me to invest in, though I've tried a few times. I settle for writing ant scripts; that's how bad things have gotten - I'm programming in XML.

What's on the horizon? Eclipse has E4. Which seems like it's largely a cleanup/sanitization of Eclipse, vs a complete re-think. I suspect I will write as many extensions for E4 as I have for the previous versions of Eclipse. Perhaps I'll sit that out and wait for Steve Northover's E5. Frankly, I think it's time we started fresh; let's call it F1.

And Joe Gregorio writes...

Poor old disrespected Smalltalk, all those years of work, all that cutting edge research, and nary a bit of credit, which is particularly galling if you think about the fact that, to date, the language that has benefited the most from Smalltalk is Java.

Mike's Gotta Job For You Mebbe

Mike Herrick's posted an opening at CSI doing the open source incl. Rails and JRuby. Good folks to work with.

June pdxfunc: Functional Programming With Dynamic Languages

Monday's topic for the Portland Functional Programming Study Group (pdxfunc) has been announced. The meeting will take place Monday, June 9, 7pm, at CubeSpace on Grand Avenue.

Please join us at the next meeting of pdxfunc, the Portland Functional Programming Study Group. The group meets monthly for presentations, demos and discussions. We welcome programmers interested in all functional languages, and the meetings feature content for coders of all skill levels. If interested, please subscribe to our mailing list at

http://groups.google.com/group/pdxfunc

PRESENTATION:

Title: Functional programming with dynamic languages

Summary: Dynamic, multi-paradigm languages such as Ruby, Python and Perl allow programmers great flexibility in how they can write their code, although most use an imperative (AKA "procedural") style. However, using a functional style selectively can help provide clearer and shorter code, reduce bugs, and improve performance. This presentation will provide a basic overview of functional programming, sample code demonstrating how to use functional programming features of dynamic languages, compare and contrast code samples of functional and imperative styles, and discuss when each style is appropriate. No previous knowledge of functional programming is necessary.

Bio: Igal Koshevoy is a Business-Technology Consultant with over a decade of experience creating sophisticated, business-critical systems using open source technologies. He is the author of an open source server automation toolkit available at "automateit.org", and supports local community efforts through Calagator, Portland Ruby Brigade, Portland Functional Programming Study Group, Legion of Tech, Ignite Portland, etc.

See you there!

PS: Sorry for the short notice, but Kevin's talk on XML parsing with Haskell had to be delayed. Hopefully he'll be able to present it next month.

Microsoft Wants To Give You A Vulnerable Flash

From Steve Loughran...

SANS is issuing an emergency warning that XP SP3 installs a version of flash with security holes, even over a later version.

And you use Windows, again, why?

Acrobatics

News from Adobe today, heading in the direction they were talking about last year for PDF and Flash. PDF is an open standard. I don't know how that spec-wise matches what it takes for Adobe to incorporate SWFs into Reader, techno-wise, and stuff...

Two big announcements today. One, we’re announcing Reader 9 which includes the ability to render SWF content inside of your PDF. That means you can actually put Flash/Flex content directly into created PDFs. That opens up a lot of very, very interesting use cases like exposing form entry applications right inside of the document or starting to use a lot of video content inside of your PDFs. I think this is one of the more interesting things to happen with the Adobe/Macromedia merger. With PDF’s right’s management built in, this also provides you a way to protect your SWF content from being decompiled.

The second announcement, my favorite of the day, is that we’re rolling out Acrobat.com. You’ve seen a lot of the services in different parts of the web, but we’re finally bringing them together under one umbrella brand with one account and a lot of functionality. You get Buzzword, “Share”, and “Brio” all as part of Acrobat.com and the line between the online software and the desktop software is starting to blur. For instance right from the Adobe Reader menus you can create a new Buzzword document or start a collaborative meeting in “Brio” which as been named Connect Now. Your Buzzword documents can now export to PDF and you can use Share to both create PDFs from documents as well as make it easy to send them to friends or embed them on your blog. There’s even an Acrobat.com AIR widget that lets you drag and drop files and create PDFs from them.

I think it's cool. And another set of signs that Adobe "gets" some things competitors either don't get or have not figured out how to execute on successfully. This is all good news for "the office" and multiple offices working together over the internets. Not to mention education, entertainment, and everything else.

Also, via Mike Chambers today, a bunch of useful reference guide stuff from Adobe re: RIA and ActionScript.

I don't know. Maybe I am just an Adobe "fan boy" so to speak. I think they're doing a lot of things well.

Toward Better UI Programming Models

Every now and then I complain about the browser's programming model. There's just been better models in the past on desktops, and there must be a ways forward that (1) "don't hurt the web" [see below] and (2) provide at least the ease of the popular models of the past. Most of the popular models of the past are based on some variation of Model-View-Controller from Smalltalk-80... frameworks like Swing, MFC, etc.

Those models are still incredibly low level compared to other practical-yet-relatively-unknown models that evolved over the last 30 years or more. Most of these other models that I can recall are all in the ballpark of contraint-based and data-flow-based systems.

Here are some off the top of my head for interested readers, many of whom I suspect may *only* have client-side experience with HTML and AJAX if they are younger than a certain age, maybe 35? I'll fill out links and descriptions as I get time.

• ThingLab - And subsequent constraint solvers that came out of Alan Borning's groups for Smalltalk, Java, C, etc. (Many developers are familiar with contraints for layout, less familiar with constraints used within domain model and GUI integration. For a recent discussion on contraints in layout see: Michael Lucas-Smith, Vassili Bykov, Andres Valloud)
• Alternate Reality Kit (ARK) - Smalltalk
• Morphic - originally influenced by ARK and implemented in Self, then in Squeak Smalltalk, and now Lively Kernel in Javascript.
• Fabrik
• Hypercard (on this list for its unique simplicity as opposed to its programming model per se)
• Garnet - CommonLisp, read the papers and try the system if you can - great stuff.
• Amulet - Garnet's C++ successor, a bit more clunk but available to more programmers
• EZDraw - Joel Bartlet's Scheme->C system
• Figlets - Haskell
• Functional Reactive Programming (FRP) (data-flow-ish) systems like: FrTime (PLT Scheme), Frappe (Java), Flapjax (Javascript), and various Haskell systems.

Please Don't Hurt The Web

I don't think HTML and AJAX should or could go away any time soon. But I don't think all "clients" have to be entirely (or at all) made of HTML and AJAX. The more RESTful the web becomes the less clients have to consist of HTML and AJAX. That's a big reason I like AIR so much: HTML and AJAX can contain other models like Flex and PDF; and Flex can contain HTML and AJAX; RESTful services can be integrated into any of this; and so on.

Dan Ingalls Interview

A really great interview with Dan Ingalls from Randal Schwartz and Leo Laporte. This goes into all kinds of Smalltalk and other history from the 70s and 80s and then into more current stuff like Lively Kernel.

At one brief moment, Dan mentions Fabrik, which is a visual data flow programming model he built at Apple in Smalltalk in the 80s.

http://methodsandmessages.vox.com/library/post/dan-ingalls-interviewed-on-floss-weekly.html

XMPP Gears and Damn All We Need Now Is a Better UI Programming Model

Bill de hÓra predicts...

if Gears (or something like it) becomes ubiquitous (and it will), then eventually XMPP will become a protocol option alongside HTTP, allowing data to be pushed down into the client.

I'm for it. Note Adobe's AIR has all of these things, and then some, not to mention an API for either Flash/Flex or HTML/Ajax-based UIs.

I'd love to see more competion to the browser's current UI programming model, which is still what you are left with after adopting Gears. At least AIR gives you a choice, and Flex is open, so conceivably Google, Mozilla, or whoever could build off of Flex.

(I discount Lively Kernel for the time being. It's probably got more promise than I give it credit for. SVG has a lot of interesting capabilities especially for display, but it's still too clunky for the interactive bits. Kind of like NeWS vs. Display Postscript, ironically.)

Even Flex is so 1990s. There have been several significant improvements on that old MVC-ish model from the pre-web client/server days. Most of them in fact *pre-date* the 1990s adoption of Smalltalk's 1970s MVC. Still Flex is way better than HTML/Ajax for developer countenance though.

An Open Source Twitter Clone

I am not a twitterer, er, one who twits. Well some might argue with
that last bit.

But one thing I've wondered is why the fascination with this service
that as far as I know is run by one outfit, has some "issues" (e.g. I
clicked on Mike Herrick's twitter URL the other day and each time saw
nothing but apologies from twitter for failing to work properly), and
is not federated across multiple providers as with, well, the web.

And I've wondered why people don't just use XMPP which is a federated
messaging platform with pub/sub topics etc. Or why not create a
twitter-ish system that can be federated.

So someone just published the source to an erlang-based twitter clone...

http://twoorl.com/
http://code.google.com/p/twoorl

I don't know if this specific clone is the answer, but it begs the question.

McCain the Maverick

I'm old enough to remember the savings and loan scandal as an adult. I'm jaded enough to be tired of this "McCain is a straight-talking maverick" hoodwink. Let's not forget he is from an "elite" family, and his questionably unethical interfering on behalf of rich beneficiaries goes back decades and involves activities that has cost us real money, and some worse. From the wikipedia article on the Keating Five, of which McCain is one...

"Gray testified that several U.S. senators had approached him and requested that he ease off on the Lincoln investigation. It came out that these senators had been beneficiaries of $300,000 (collective total) in campaign contributions from Keating. McCain received $112,000 by 1987 from Keating and Keating's relatives and employees to McCain's Senate campaign, more than any of the other Senators. In September 1987 National Thrift News was the first media outlet to break the story. In October 1989 The Arizona Republic reported that in addition to campaign contributions, McCain's wife and her father had invested $359,100 in a Keating shopping center in April 1986, a year before McCain met with the regulators. The paper also reported that the McCains, sometimes accompanied by their daughter and baby-sitter, had made at least nine trips at Keating's expense, sometimes aboard the American Continental Corporation (parent of Lincoln) jet. Three of the trips were made during vacations to Keating's opulent Bahamas retreat at Cat Cay. McCain also did not pay Keating for some of the trips until years after they were taken, after he learned that Keating was in trouble over Lincoln. Lincoln Savings and Loan's collapse is said to have cost taxpayers $3.4 billion."

Animated Graffiti

Via Signal vs. Noise, the following video is incredibly inventive. The theme becomes tedious for me before halfway, but still... I'd like to see more like this with a better story to tell...

Erlang and the Facebook Chat Architecture

Seen on InfoQ re: the Facebook Chat Architecture...

Another challenge was delivering messages in real time. Facebook choose a technique whereby the client pulls updates from the server, similar to Comet's XHR Long Polling Process...

Facebook choose a combination of C++ and Erlang to implement clustered and partitioned subsystems. The C++ module is used to log chat messages, while Erlang "holds online users' conversations in-memory and serves the long-polled HTTP requests". epoll, a new system call introduced in Linux 2.6, was used to drive the Erlang module. Eugene states why the decision was made to go with Erlang...

because the problem domain fits Erlang like a glove. Erlang is a functional concurrency-oriented language with extremely low-weight user-space "processes", share-nothing message-passing semantics, built-in distribution, and a "crash and recover" philosophy proven by two decades of deployment on large soft-realtime production systems...

The secret for going from zero to seventy million users overnight is to avoid doing it all in one fell swoop. We chose to simulate the impact of many real users hitting many machines by means of a "dark launch" period in which Facebook pages would make connections to the chat servers, query for presence information and simulate message sends without a single UI element drawn on the page.

I am kind of amazed to see the attention Erlang has been given over the last couple two three years. OSCON in July here in Portland will have at least a three Erlang related sessions: a couple of tutorials and a presentation.

Mike Herrick on CSI and on InfoQ

InfoQ interviews Mike Herrick re: using Rails and JRuby for the CSI. Mike also blog'd about his project recently...

When deployed, UT-NEDSS project will directly contribute to the prevention of sickness and death by effectively collecting, identifying, tracking and trending information gathered about infectious diseases and bioterrorism attacks. Additionally, this unique partnership is providing lessons for public health informatics

Steve Vinoski on RPC and Messaging

There's been a stream of messages on the erlang list about RPC. Steve Vinoski's put an explanation of RPC and his position on his blog. Now he's followed up on the email list with a position on integration generally, and pub/sub messaging in particular...

One of the most effective forms of enterprise integration I've seen over the years is publish/subscribe messaging. I worked many years for CORBA vendors, and we'd often lose potential deals to messaging systems. Message queuing systems work well because (in no particular order):

• they don't pretend to be programming language procedure or method calls, so they avoid the associated impedance mismatch problems
• they don't try to hide distributed systems issues
• coupling is low -- drop a message into a queue here, pick up a message from a queue there
• queues can be persistent, or more generally, delivery guarantees can be varied as needed
• asynchrony
• payloads need not conform to some made-up IDL type system
• getting two different messaging systems to interoperate is easier than getting two different RPC or distributed object systems to interoperate

The problem with messaging systems, though, is that traditionally they've been quite expensive. Thankfully, I believe AMQP (http://amqp.org/) solves that issue nicely, and of course Erlang is the perfect way to implement it, which Alexis, Tony, and the rest of the RabbitMQ guys have already done (http://www.rabbitmq.com/).

I agree with Steve's take. The best pub/sub mechanism I have experience with has been Tibco's rv. All the Tibco layers above it have been enterprisey bits to make a sale. The basic rv bus is super-easy to just use from C, Python, Java, anything. Horribly expensive back in the day, and I assume it still is.

AMQP should have helped address this. But I've not seen much on the internets about AMQP over the last many months. I wonder if this is going to go anywhere? Or am I just missing the traffic?

I'm not sure XMPP pub/sub in the enterprise is going anywhere too quickly either, but that's another candidate for getting beyond vendor lock-in for enterprise messaging.

Why on earth Cisco is defining a new RPC protocol is beyond me.

And to any AMQP folks: the redirect from http://amqp.org/ seems broken.

Enterprises should be moving toward HTTP first and foremost, figuring out what's happening with XMPP for various other kinds of messaging, and/or AMQP in situations where HTTP and XMPP don't seem to cut it.

Matthias Felleisen speaking at Portland State University on June 2

Just rec'd the following announcement on the pdxfunc email list...

Matthias Felleisen is speaking at PSU a week from Monday (June 2). The
talk is open to the public.

"The First Year in Computer Science"
Matthias Felleisen
Northeastern University
2nd June, 16:40–18:00
Location: Hoffman Hall Rm 109, Portland State University
Abstract

My team and I have developed an alternative approach to the first-year courses on programming and computing. Unlike conventional approaches, ours focuses on designing programs in a systematic manner. The syntax of the chosen programming languages (both Java and Scheme) is only discussed as needed to support design principles.

Field tests with over 500 high schools and a dozen colleges have shown that the approach produces better students than conventional approaches that use a single language.

In several controlled studies, we could also show that students find our curriculum more appealing than the AP curriculum.

In my talk, I will provide an overview of the project, especially its intellectual premises and principles. My goal is to encourage you to think about the first year in a different way. The old ones are of questionable value. If we want our beautiful discipline to survive, we must find a good way of teaching it."

http://web.cecs.pdx.edu/~colloq/felleisen-firstYear.html for more information.

More Ted

This is part two of my response to Ted Neward's latest misunderstanding of Erlang. Ted wrote to Steve Vinoski...

Sorry, Steve-O, but I think you're out in left field on this one. I'm happy to argue it further with you over beer, but if you want the last word, have at it, and we'll compare scores when we run into each other at the next conference.

Everything I've read of Steve's tells me he's well informed by actual experience. Ted's writing exhibits the most basic misunderstandings. I wonder why Ted didn't spend a little beer time with some experts before attempting to write with supposed authority.

More from Ted...

any time we incorporate something directly as part of the language, there's all kinds of versioning and revision issues that come with it. This, to my mind, is one of Scala's (and F#'s and Lisp's and Clojure's and Scheme's and other composite languages') greatest strengths, the ability to create constructs that look like they're part of the language, but in fact come from libraries.

Most of Erlang's power comes from libraries, but Ted wouldn't know that. Erlang the language is very small and expressive, but Ted wouldn't know that. Erlang's not really suffered from revision issues over the nine years I've been following the language, but Ted wouldn't know that. And Ted apparently didn't bother to ask anyone or read anything that might inform himself.

I agree with Ted's observation that extensible language syntax is desirable. Erlang's extensibility is cumbersome and not-too-beautiful. That's not the goal of Erlang though.

So you have to decide whether the benefits of Erlang outweigh issues like this. By the way, Gambit Scheme has Erlang-like concurrency performance, although shared-everything. But since Scheme can be used to define new languages such as Termite, it can be used to create syntactically-extensible Erlang-ish systems.

If you decide to go with Erlang the decision should be based on what Erlang is now, and not expect Erlang to change much. It's been largely stable as a language for years, as opposed to the arms race in Java and Java-like worlds. Language extensibility should not be as important to you as the simple Erlang programming model. This is a reasonable approach to take. Erlang is a known quantity, well proven. Not every language has to be as extensible as Lisp, in fact most are not.

Beware of languages like Scala and Clojure that intend to be safer concurrently than Java - they still run on the JVM and integration with Java code. So by definition their safety only can go so far.

I do criticize Erlang for having its own VM (though I think it's not a VM, it's an interpreter, which is a far cry from an actual VM)

Maybe we should criticize Java for having its own JVM? Why doesn't it use Smalltalk's or Erlang's which pre-date the JVM? (Trick question. The JVM, and Hot Spot in particular, is based on ideas developed in Smalltalk, etc. Essentially the JVM is a dynamic language VM plus layers of cruft.)

Not much digging would have told Ted that Erlang compiles to native code or to an intermediate compiled code. Ted's misleading claims are heading into the dangerously ignorant. Why should I believe anything Ted writes about any topic? When Ted writes "I think that..." I think that we should read it as, "I'm guessing that..."

The JVM and the CLR have (literally) thousands of man-months sunk into them to reach high levels in all those areas. Can Erlang claim the same?

Well, yeah. I don't know how many "man months" it stacks up to be, but Erlang has been around for decades in some very demanding situations. Did Ted bother to look into any of these VM questions?

Best part is, the IT department doesn't have to do anything different to their existing Java-based network topology to start taking advantage of this. Can you say the same for Erlang?

If the architecture is going to change from "big, honkin' J2ee behemoths" to highly concurrent, lightweight interaction, then the topology should change. This is true whether moving from J2EE to Erlang or J2EE to JRuby/Rails or J2EE to Project Zero or J2EE to anything reasonably resembling modern best practices.

Moving to Erlang can take advantage of existing Java bits using the various network protocol standards or CORBA or SOAP or the JInterface class library which makes a Java application act like an Erlang node.

Frankly, whether the application you're monitoring hooks into the monitoring infrastructure is not really part of the argument, since Erlang doesn't offer that, either. I'm more concerned with whether the infrastructure is monitoring-friendly... if Erlang ties into SNMP out of the box with no work required by the programmer, please tell me where that's doc'ed and how it works!

Apparently Ted's world of monitoring begins and ends with Java and dotnet's JMX and WMI. How would you monitor both Java and dotnet? SNMP likely. Both JMX and WMI can be hooked into SNMP. SNMP is the internet standard, more mature, and extends beyond Java and dotnet.

I've never used it, but just doing a quick check shows that Erlang has a pretty extensive SNMP api and Reference Manual, and User Guide. Hmm. Did Ted even bother to look?

Yes, the JVM could easily adopt the multi-process model if it chose to. (Said work is being done via the Java Isolates JSR.) The CLR already does (via AppDomains).

I addressed Erlang vs. AppDomains in my previous post. These are clearly two different mechanisms with two different intents: AppDomains provide application isolation; Erlang processes provide finer-grained isolation of concurrent activities ("actor"-like) within an application.

Within an application neither the JVM nor the JVM-ish CLR provide Erlang-like processes. Scala and Clojure go a good bit of the way, but the advantage of Erlang is in its simplicity and combination of concurrency programming _and_ management. This goes well beyond a little bit of syntax around Java threads within a single JVM.

Barry Kelly has a really good blog post explaining more about these various VMs and Scala's Lift framework and implementation of Actors.

Toward Finer Tuning of Definitions of Processes and Languages, Reliably

Here's an attempt to define terms more finely (finally?) after reading the latest (final?) response from Ted Neward to Steve Vinoski's attempts at helping Ted to clarify differences between Erlang and other languages and run-times, esp. those based on Java/JVM-ish. At a superficial level of understanding what Ted writes has some meaning. Just enough to be troublesome. So here's my pass at making clarifications.

The first terms to narrow in on are "process" and "thread" as they are used in various languages and operating systems. There are similarities for sure, but the subtle differences are the important aspects when making comparisons. Ted writes a few things using these terms...

Processes in *nix are just as vulnerable to bad coding practices as are processes in Windows or Mac OS X

Aside: Mac OS X _is_ Unix. But that's for a different debate on the merits of Windows per se.

Anyway. I don't expect anyone to argue with Ted on this point, so let's dig deeper into "bad coding practices" and processes. Erlang is a "mostly-functional" language and there is very little shared memory in its concurrency model. This removes many opportunities for bad coding. So can we say that programming an operating system process in Erlang is less vulnerable than programming an operating system process using most popular languages? I would say yes, generally.

the robustness and reliability of a system is entirely held hostage to the skill and care of the worst programmer on the system

The emphasis is Ted's, not mine. There are many fine points to be made about this claim. One is the term "entirely" and another is the term "system". But first is the observation that this claim is about development practices more than anything else. To argue about language and mechanism shouldn't we assume that the teams are skilled at least in their own languages and mechanisms?

Let's say no and look deeper. You won't read much Erlang literature before you come across terms like robustness and reliability, and explanation of language constructs and framework mechanisms that support these traits.

On the other hand one can program in Java and similar languages for years before realizing that attention to robustness and reliability are even the developer's concern.

Back to "system" and "entirely" being at the mercy of the worst programmer. Perhaps the most important point made by the Erlang experts is to expect the worst and to design your system to recover from that. The Erlang culture does its best to help you isolate bad parts and the Erlang mechanisms make that isolation and recovery easier than with most other systems.

Erlang's model is similar to the CLR's AppDomain construct, which has been around since .NET 1.0, and Java's proposed "Isolate" feature which has yet to be implemented.

I'm not going to presume to know anything more about AppDomains than Ted know's about about Erlang, but this comparison seems true only barely. Here's some information I found about AppDomains...

The main purpose of an Application Domain is to isolate our applications from other applications. Application domains run on a single Win32 process... Objects in the same Application Domain communicate directly while Objects that exists in different Application Domains interact with each other by transporting copies of objects to each other or by using proxies for message exchange (by reference).

Any somewhat-skilled programmer familiar in the least with Erlang should be able to tell you significant differences between the AppDomain described above and Erlang's process model. I'm bothered that Ted Neward has such a large stage for making superficial statements that these mechanisms are in any way meaningfully the same. This is a significant disservice to programmers following his writing, trying to learn useful information. If Ted does not have (apparently) even a few days worth of reasonable Erlang experience or even reading, why should he presume to make such significant and horribly misleading statements to his large audience?

Here are the significant differences between these two mechanisms (Erlang processes and dotnet AppDomains)...

• AppDomains appear to be used to put distinct applications inside the same Windows OS process.
• Erlang processes are used to decompose a single application into many, small, independent, shared nothing processes.
• Objects in different AppDomains are considered separate and rarely-interacting.
• Processes in Erlang are roughly the same "size" as an instance of a major role-playing object in a single application. They collaborate with other processes frequently and are highly interactive.
• Objects in an AppDomain are "shared-everything", running concurrent, shared-everything threads.
• Erlang processes (the equivalent of objects in dotnet) in an application are "shared-nothing".
• AppDomains pass copies of values or references to objects (as far as I can tell from this one article).
• Erlang processes always pass copies of values. Erlang and its frameworks per se do not provide references, and so it's "shared-nothing" all the way down. The equivalent of "reference to an object" in Erlang is "identifier of a process".
• AppDomains appear to be a large-grain, compile-time, declarative mechanism. Any Windows OS process would have just a few AppDomains. I would suppose these number in the dozens per OS process.
• Erlang processes are a fine-grain, run-time, dynamic mechanism. An Erlang node (essentially an OS process) would have many hundreds of processes or more.

Ted continues...

if the argument here is that Erlang's reliability comes from its lack of shared state between threads, hell, man, that's hardly a difficult architecture to cook up. Most transactional systems get there pretty easily, including EJB, though then programmers then go through dozens of hoops to try and break it.

Ted does a pretty good job of hinting that EJB is a broken model. EJB isolation is more like an AppDomain. EJB separates one application from another, with fairly laborious mechanisms required to get them to communicate with each other.

Within an EJB application there is supposed to be no concurrency at all. And so again this is the opposite of Erlang. Erlang encourages high concurrency and ease of interaction. But these points are apparently lost on Ted. Usually one cannot read a single page of Erlang literature without having this made absolutely obvious.

The next post will get into Ted's arguments about language and common run-times.

April 2008 Game Console Sales

Wii ------------ 714,200 units
Xbox 360 --- 188,000 units
PS3 ---------- 187,100 units

(NPD Group)

http://blog.seattlepi.nwsource.com/microsoft/archives/138983.asp?source=rss

A Little Knowledge

Steve Vinoski is in a bit of back and forth with another blogger about
Erlang vs. other languages, and how easily one could adopt the best
bits of the other. Steve implies, correctly, IMO, that to perform this
kind of assessment requires a fairly good understanding, or at least a
fair bit of research, into the various candidates.

http://steve.vinoski.net/blog/2008/05/09/thinking-in-language-but-not-clearly/

I've seen several attempts to build "erlang in x" where x = common
lisp, python, java, dotnet, etc. To the extent I've followed them they
implement a fraction of the message passing and then don't go much
further. Scala, etc. on the jvm and those on other conventional
runtimes have a problem. Integration with Java instances within those
runtimes is not shared-nothing. You have to remain aware of what is
and what is not following the actor paradigm.

Erlang's JInterface is probably a better integration mechanism in the
long-run than running in the same shared-memory OS process.

As for monitoring, certainly a quick glance would have shown Ted there
is SNMP, as Steve points out. But also isn't the entire
process-linking and supervisor hierarchy an important kind of
"monitoring", i.e. "monitoring for availability"? This is something
all these other platforms lack.

So the more I think about it, the less suited these other platforms
are for messaging amongst themselves *or* for monitoring!

Then I came across Avi Bryant's post on Strongtalk, a high-performance
implementation of Smalltalk, correcting an apparently common
misunderstanding that the speed has something to do with the optional
type checker...

"Why do people make this stuff up? ... Strongtalk was that much faster
whether you used the optional static type system or not. Strongtalk's
optimizing compiler completely ignored the types, and it [types -ed.]
made your program run not one iota faster to add them."

http://www.avibryant.com/2008/05/those-who-misre.html

Your Architecture's Agenda

Phil Windley writes...

"Designs are full of defaults--some explicit and most of the implicit.
Programmers don't pay enough attention to defaults. Rail's "convention
over configuration" is a great example of a system that carefully
thought through defaults. 37 Signals calls this concept opinionated
software."

http://www.windley.com/archives/2008/05/whats_your_architectures_agenda.shtml

Isn't That What The Internets Are For?

Joe Wilcox watches Microsoft and wonders...

"Mesh is the only thing that really makes sense out of a Yahoo
acquisition to me. Yahoo has rich content services—and they're
everywhere. If Microsoft could plug Mesh into that infrastructure,
fast, and flip the switch "Wow!" Imagine, for example, Mesh making
Flickr photos instantly available to all your PCs, cell phones and
TVs. Software plus hardware plus services."

But isn't that what the internets are for?

http://www.microsoft-watch.com/content/web_services_browser/yahoo_between_a_rock_and_a_hard_place.html?kc=MWRSS02129TX1K0000535

Carl Hewitt Blog and the History of Logic Programming

Carl Hewitt has a blog and an interesting History of Logic Programming. Also an interesting essay re: Wikipedia at http://wikicensored.info/

Another Cool Smalltalk Tool

In addition to being such a simple and expressive language, Smalltalk should be known for its great collection of tools. Here's another one that was recently developed at UIUC, a Smalltalk Mecca...

Algernon was developed by Erik Hinterbichler and Joey Hagedorn with their CS598 classmates, working with Professor Ralph Johnson of the Software Architecture Group at UIUC. The developers say that Algernon provides lightning quick access to categories, classes, morphs, and global variables in your image, and is inspired by tools such as Quicksilver and LaunchBar for Mac OS X. Like Quicksilver, it learns from your behaviours, and will prioritise options that you use most often.

Good Bits, Bad Bits

Michael Nygard posted about SAP's SOA capabilities. I wasn't sure if he'd appreciated their complexity, but he does emphasize that in a comment on this post here.

We're talking really complicated, WS-*-ish bits. My advice from hands-on experience two years ago would be: stop; walk back slowly, slowly; now turn and run; faster.

Not the kind of good system attributes I think of when reading Michael's book.

The Second Right Answer

Roger von Oech via Gary King...

My mantra is “Look for the Second Right Answer.” This has been my guiding principle for over thirty years. Much of our educational system tries to teach us to look for the “one right answer.”

I find that looking for the second right answer is an incredibly easy way to open my mind. For example, when I’m looking for information, this mantra tells me to go beyond the right answers that have worked in the past and look for others. When I’m trying to be creative, it playfully advises me to put my ideas in unusual contexts to give them new meanings.

Between a Rock and a Hard Place

Update: Keith Gaughan in the comments points out Yahoo is a FreeBSD shop primarily. I stand corrected. However I assume any "cloud" OS services would provide Linux as their primary platform. End.

The downside of MSFT dropping the bid is we don't get to see what their strategy would have been. Would they have attempted to migrate everyone over to the Windows platform? Really?

Would they have dropped the reported Yahoo move into providing cloud computing services? (I assume these would/will be based on Linux and open source generally. Who would do or could afford otherwise?)

How could MSFT purchase on open-source Linux based company without migrating to Windows or altogether dropping the open source Linux bits? I suppose they could move all the Linux bits to Novell / Suse. But that's still an endorsement of Linux generally, which in spite of the Novell arrangement MSFT still detests.

Do they just pull a 180 and hope to become a true Internet / Linux / OSS company over night? That would be *something*.

Sit back and think about it. Wow. Is Microsoft stuck in a bad place with the Internets or what?

"There won't be much differentiation between consumer and enterprise architectures"

From Dan Farber on CNet News: Google CEO Eric Schmidt and IBM CEO Sam Palmisano getting all, well, Googly-eyed with each other...

IBM is taking some of the learnings from the project and plans to operate a cloud that will allow partners to house their Web-based applications and sell them to customers, Palmisano said. "It is the first time we have taken something from the consumer arena and applied it to the enterprise," he said.

Schmidt said that over time there won't be much differentiation between consumer and enterprise architectures. The major difference is that enterprise customers will pay for software and services, with required security and other features, and consumers won't...

Google's YouTube captures 10 hours of video every 60 seconds, and IBM might like that business if it could figure out how to make money at it. But eventually, IBM, Microsoft, Sun, Google, and other big players will look more similar in their technical architectures and business models...

Currently, Salesforce.com is selling Google Apps as an integrated part of its platform. It's not far-fetched to think that Google would seek out IBM's help with its business partners to spread the Google word in the enterprise.

"Pull the other one. It’s got bells on."

Gary King on the horribly underreported conflicts of interest with former military leaders simultaneoulsy taking positions as Big Media war analysts and going on the take as contractors and investors in the military-industrial complex. Does anyone care that the Pentagon's coaching these slimy bastards went up to at least Rumsfeld (at the time) and porbably well into the Whitehouse itself?

Billions of USD are being wasted (and in fact borrowed from China, et al.) EACH MONTH lining the pockets of the people who go on TV and play up the war in order to attract advertisers and instill fear in the general population. Meanwhile baby-faced young men and women are losing limbs, their brains are shelled into jelly, and losing their lives.

I'll vote for any presidential candidate who promises to reinstitute the draft, without exceptions. That is the only action I can see ending this boondoggle in the next year or so. That is the only thing I can see getting enough people's attention to really care about the endless death and destruction and blatant robbery and profiteering. As long as the vast majority of our children are safe, we seem unable to care about those who chose to protect our _constitution_ and have wound up losing their lives and limbs doing anything but.

Why we've not gone up to the door of the Whitehouse and dragged the bastards directly into the courts ourselves is beyond me. Now excuse me, American Idol is on.

Oh and remember: the next president will likely nominate three supreme court justices. And you want McBush, er, McCain, to be the one for that? As long as you like the concept of the "unitary executive". W's men are already on the court putting this theory into practice.

Objectively

"Rubinius switched from C to C++ to implement it's core VM"

For the life of me I cannot understand why projects use C++ rather than Objective-C. Hmm.

Catching up on comments to this post...

Yeah, I can be too brief sometimes. Here's the essence of what I like about ObjC vs. C++. ObjC attempts to keep the Obj and the C distinct, while C++ attempts to combine them. As a result the Obj in ObjC is very much like the Obj in Smalltalk. And the C on Obj C is very much like the C in ANSI C.

The Obj in C++ is significantly more complicated than the Obj in Smalltalk or in ObjC. The C in C++ is also significantly more complicated, to the point where I don't think it can be called "C". People will talk about the expressiveness of C++ and how much it has evolved over the years. I still very much prefer the simplicity of ObjC.

I am also surprised the ObjC has portability issues. With the GNU implementation?

And I am surprised about the Ruby kernel issue as well. I also thought this would be so small to warrant just C or even better, a subset of Ruby that compiles easily into C. This is what Squeak uses for its kernel. Gambit Scheme does something along the same lines, allowing a very C-ish dialect of Scheme that translates directly.

The Manchurian Minister

In reference to James Snell's recent post and subsequent comments
about Obama and Jeremiah Wright. This is a Big Media misdirection.

What about McCain's follies, e.g. singing "Bomb, bomb, bom, bomb. bomb
Iran" to the tune of the Beach Boy's "Barbara Ann"??? Do you want this
person to lead the free world???

What about McCain refusing to stand down from John Hagee's
endorsement??? Hagee being the self-proclaimed "Christian" who to this
day believes the flooding of New Orleans after Hurricane Katrina is
due to homosexuality.

Politics. I'm not crazy about Obama or any candidate, but I'll take
anyone who's a bit more on stable ground than McCain. Holy cow.

In a comment on James' post, about Obama, the comment claims that…
"Every day a new skeleton comes out."

Do we have a list of these skeletons somewhere? Every day since when?
How many skeletons are there?

Does Obama have a track record in government of legislating along the
lines of what this supposed Manchurian Minister would dictate?

Is the Big Media doing a service or disservice on this topic? It seems
not to be digging deep about anything, certainly not about Obama's
actual track record and certainly not anything about the untouchable
McBush, I mean, McCain. The Big Media does seem determined to keep the
wound open and never find a way to close it one way or another.

People get the government and the media they deserve, to borrow a
phrase from Phil Windley.

Update:

I'll tell you what, looking at the comments over on James' post, it's *rul* scary how different people's opinions are from mine. There is a *huge* divide in our thinking. I don't know if I'm right or not, but I have my guesses.

Huh?: "The fact that it runs on its own interpreter, bad."

What is this obsession with some virtual machine being the one, true byte code? The Java Virtual Machine, the CLR, Parrot, whatever. Give it up.

I agree with Steve Vinoski...

The fact that it runs on its own interpreter, good; otherwise, the reliability wouldn’t be there.

We need to get over our thinking about "One VM to bring them all and in the darkness bind them". Instead we should be focused on improving interprocess communication among various languages. This can be done with HTTP and XMPP. And we should expecially be focused on reliability, deployment, starting and stopping locally or remotely, etc. XMPP's "presence" provides Erlang-process-like linking of a sort as well.

With Erlang's JInterface for Java then a Java process can look like an Erlang process (distributed or remote). Two or more Java processes can use JInterface to communicate and "link" reliably and Erlang virtual machines and libraries, save this one single .jar, do not have to be anywhere in sight.

To obsess about a single VM is to remain stuck at about 1980 and UCSD Pascal's p-code. It just should not matter today, and certainly not tomorrow. The forest is now much more important than any given tree.

Pay attention to the new JVM from IBM in support of their lightweight, fast-start, single-purpose process philosophy embodied in Project Zero. It's not intended to be a big honkin' run everything forever virtual machine. It will support JVM languages and the more the merrier in the sense that such a JVM will enable lightweight pieces to be stiched together dynamically. However the intention is to perform some interprocess communication and then get out of the way. Exactly the right approach for any virtual machine.

Jini clearly is *the* most important thing about Java, ever. But it's lost. Gone. Buh-bye. Pity.

Flash Now Even Openier

Adobe continues to open up more in the world of Flash. Sure, if you like the browser as an application platform, go for it.

This one's better and doesn't hurt the web any more than anything else unless you, the developer, choose to. You can hurt the web just as badly in the browser without hardly trying.

Update for Ed:

2nd Bar Camp Portland - May 2-4, 2008

http://barcamp.org/BarCampPortland

Wallhoo or Yahmart?

I stand by my very brief post a few weeks ago: yahoo and wallmart should merge. Yahoo has the internet savvy and infrastructure. Wallmart has the sales and distribution. Yahoo needs to get out from under search and ads as their model. Wallmart needs to get on the web better to compete with Amazon, Apple's itunes, etc.

Who else is going to compete with Amazon? I wonder if a Google / Wallmart merger would go through? Not as likely as Yahoo, and Google does not (yet?) feel the need like Yahoo should.

Brevity

I am sure I give plenty of people good reasons not to read this blog. One positive attribute I generally try to hold to is brevity. (Let me know if I don't meet that criteria in your book.)

And so I agree with Steve Dekorte's assessment of bloggers that write entries that are just too long for my patience. Steve names one blogger in particular who I unsubscribed a good while ago after a brief trial period. There are a couple others I could name.

I'm not saying they have nothing worthwhile to write about. The people who come to mind for myself do tend to write interesting entries, do tend to have those entries much more widely quoted than anything I've ever written.

All I am saying, before this gets too long, is that I cannot take the time to read long entries that stay on target, let alone long entries that take too long to reach a point or even a subject. And if I don't stop now...

Video: Introduction to Erlang

Bob Ippolito at the C4[1] conference a while back, introducing Erlang and how his company is using it...

Simple is as simple does or some such.

Open Is As Open Does Or Some Such

At ONLamp, Todd Ogasawara points out some potentially harsh consequences associated with, if not attributable to, Sun's $$$ purchase of MySQL...

If the model of selling services does not justify something like a billion dollar price tag, what then? For MySQL and Sun, the answer is to provide more value-added features for a price and closing the source.

Am I happy about this? Not hardly! But, I saw this coming and have been preparing for it. I’ve been looking at PostgreSQL since the day Sun announced buying MySQL. And, recently, it was pointed out to me that Ingres (which I used back in the 1980s) is now an Open Source product. I’m not going to suddenly stop using MySQL or recommend that people switch away from it. But, I think it is prudent to take a look at alternatives.

Simple Works Best

Dustin Puryear on memcached at ONLamp...

memcached was written to serve one basic role: cache database request. It wasn’t written to provide a massively redundant service. Or to distribute load across memcached nodes. Or to provide a secure proxy to a database service. It just takes a query and returns whatever is in the cache. And this is done using a simple hash, meaning that at its core memcached uses a set of algorithms that you’ll find on every second year Computer Science exam in college.

Microsoft's Mess

Readers of this blog could have predicted my title, I'm sure. But I have support from Joe Wilcox...

Live Mesh is so messy to explain, I can't cover everything in this post...

It's the most anti-Web 2.0 technology yet released by any company. Microsoft is building a services-based operating system that transcends and extends Windows and also the function of Web browsers. It's bold, brilliant and downright scary.

Actually, it's typical and to be expected. Microsoft regularly comes out with huge proposals to own everything, world wide.

Meanwhile Google, Amazon, and others nibble off bits and pieces of the world-wide web without altogether hurting the web. That is, they *get* the web, and they *get* how to contribute to and profit from the web. They continuously and incrementally add more to the web. Microsoft periodically says, "Here, swallow this huge horse tranquilizer and you'll be all right."

Ozzie apparently sees the web as a "given", but not something to embrace. Rather he apparently sees the web as an inhibiter to the success of Groove. And so his grand design seems to be to wrap the web inside of Groove. Consider Ozzie's own terminology: the web to him is "a hub".

Good luck with that.

Or go back to implementing WinFS, that should keep you busy while the internets slip further from your grasp.

Read BobWarfield in-depth analysis at SmoothSpan. Especially if you are from Microsoft. Especially if you are Ray Ozzie...

There are 100 engineers at work on Live Mesh already, and lots of key functionality (like version control) nowhere in sight. Aside from the Tactics of Monopoly, the other Fail mode is creating a giant monolith of software. Vista is a painful example of how far things can go wrong. Mesh is, at its core, another attempt to rework the document and folder file system. Microsoft promised this in Longhorn for years but never delivered.

My clojure talk at pdxfunc

Amy Farrell wrote up some impressive notes from a talk last week that Adam Jones and I gave on the clojure programming language to the pdxfunc functional programming interest group in Portland. My part of this was presented as a long-time Lisper but new to Clojure. I'm not sold on Clojure yet - I like the idea of a mostly functional, concurrent Lisp, and I like the idea of Lisp for the JVM. But is it a better choice than JScheme or SISC or the other more (IEEE and ANSI) standard Lisps? Maybe for those reasons above, but it also seems gratuitously unlike the standard Lisps.

(Update: Rich Hickey (Clojure's author and impressive dynamic force when you consider what he creates per unit of time) responds in the comments here to some of my observations in my talk. I should also say when you consider a new Lisp for the 21st century, Clojure is probably the better fit than Arc. Probably a better fit than Scheme or CL per se. Although Gambit Scheme and an evolution of Termite would be up on my list.)

This talk was patterned after the BOF on Erlang I held at last year's OSCON -- I like talking through things while evaluating them "live". At OSCON I typed everything live and that went fine. Last week I had a new laptop, and even though I like the keyboard very much (MacBook), I thought I might fumble-finger too much. So I set up all the expressions in a file beforehand and using emacs all I did was talk and type c-x-c-e to evaluate each expression in the clojure sub-process.

Not sure what's in store for the May meeting but in June pdxfunc may have a good discussion on a new set of open source web libraries for Haskell from Galois. Several Haskell enthusiasts (or more) attend pdxfunc, so maybe I'll get back into that whole typing, lazy functional thing, and learn from experts at some point.

Decentralization at a Cross Roads

Linux Journal has a great interview by Doc Searls with Bob Frankston in the May edition. From Searls' introduction...

"Telecom and the internet have always been strange bedfellows... we have an industry that's been around for 171 years... and... we have... an "end-to-end" model that doesn't require telecom... carriers only want us to think only in terms of familiar and expensive services such as television."

On a seemingly unrelated note, NPR had a radio piece this morning on solar power. A couple large companies are trying to build giant plants in the desert to boil water, to turn large turbines, to sell the electricity over the grid. Another tendency of large corporations wanting us to think only in familiar, hugely expensive terms. At least they're not trying to sell the idea of splitting atoms to boil the water.

Not to mention we're running out of water, another problem that can only be solved on a massively small scale. Certainly boiling water in the desert in huge quantities is not inline with our water future.

If any one thing will save civilization that thing will be drastically decentralized. I'm not convinced we have good odds to pull that off. I think our psychology and sociology is not adapted to the level of our IQ.

Watch out for big. We are at an interesting cross roads, and the big organizations are not going to make the right decisions. They are not designed to. It's not their fault, they're just dinosaurs.

Smalltalk and Seaside at FISL 9.0

Randal Schwartz is in Brazil at FISL 9.0. He gave a Seaside talk...

...the biggest news is that based on the preliminary interest in Seaside because of my talk, the FISL conference organizers offered an entire room for next years conference (the full three days with 12 hours per day), as well as four or five main-track hour talks, if I could help organize the subconference details! This is quite a gift, because it will mean that we can expose the 7000 conference attendees to a variety of Smalltalk programs, without paying for rooms or badging or promotion. The conference asked if I could get some corporate sponsors on board, and I immediately fired off email to James at Cincom and Monty at GemStone, and thank goodness they read email on Saturday, because they offered their support quickly. Of course, we have many details to work out, but everyone agrees that we will move forward!

Brazil has been fairly active with Smalltalk over the years. And Randal is a great evangelist for Smalltalk and Seaside. Having talked with him a couple of times recently at Portland-area groups for Smalltalk and open source development, he comes across as an intelligent, enthusiastic evangelist for whatever he's interested in.

He's going to do wonders for Smalltalk and Seaside, just watch.

Covers

Peter Saint-Andre writes about some favorite cover songs. Here are some of mine that come to mind:

• Dave Mason's "All Along the Watchtower" is a cover of Hendrix's cover.
• Peter Frampton's "While My Guitar Gently Weeps".
• The Pretenders' "Needle and the Damage Done"
• The Pretenders again: "Angel of the Morning"

And my list goes on, but I'll end it for now with some cover albums (as opposed to album covers)...

• Not Fade Away (Remembering Buddy Holly) - various artists
• Common Thread: The Songs of the Eagles - various country artists
• Influence - Shaw and Blades
• An Acoustic Tribute to Prince - various female artists, I found this on itunes.
• Run Devil Run - Paul McCartney covering artists who influenced him

And the list could go on further. Of course I'd include every cover the Beatles made of Chuck Berry, Buddy Holly, Carl Perkins.

Most covered song on my ipod: "Sweet Jane" at seven different artists. I have almost as many covers (6) of "Within You Without You".

Clueless in Seattle

Ray Ozzie speaks, and he speaks about Open Source Software. I don't even know what to quote from his statement. They're just not in the right ballpark. The only thing Ozzie communicates here is that he just does not understand the open source community, which _is_ the community for building out the internets.

This comes from the guy who built Groove out of proprietary Microsoft bits and proprietary bits from his developers. By the time they were shipping Groove, those bits had rotted. COM this and that, custom proprietary object database foo.

Had the protocols and the bits within Groove been even somewhat open, they might have grown a community around it, and kept it alive. Maybe some of those bits are living inside some Microsoft product. Running COM this and that???

Microsoft's "groove" looks more like a rut and face it, they've flipped the bozo bit. What a long, slow, excrutiating slide into oblivion they're on unless they can figure out what's really happening on the internets these days.

Update: This is an example of what BobWarfield at SmoothSpan calls: "Microsoft's Expensive Rift with the Web."

Another update: James Robertson ties this into the Smalltalk community. Seaside seems to be taking Smalltalk to another level. Smalltalk is an interesting community re: open source. As with the Lisp community there's always been a significant "openness" to Smalltalk, predating the actual open source licensing of various implementations. However there's no way the various vendor's Smalltalk GUI's could ever have been unified into an open implementation, but Seaside is on its way toward that end.

Intuit's Flex-able Cloud

Catchy title, eh?

Bob Warfield has some interesting "pre-news" about a new cloud
computing platform...

http://smoothspan.wordpress.com/2008/04/16/intuits-radical-new-flex-quickbase-cloud-platform/

"Someone mentioned that the standard for Cloud Platforms was to bring
up an app that everyone in the audience can log into within 5 minutes
or less. No problems here. We had the good old "Hello, World" up and
running very quickly."

But he wonders about Microsoft's cloudiness (cloudlessness?)...

""
Doesn't the Big M have SQL Server Data Services:
http://www.microsoft.com/sql/dataservices/default.mspx

Please to be wanting Vista now!

OMFG

Vista: "Career Ending"

Joe Wilcox of Microsoft Watch notes...

"Poole's departure pretty much marks the end of an era. Nearly every
Microsoft executive associated with the Windows Vista launch has left
the company. Vista has proven to be a career-ending enterprise, in
stark contrast to bygone days when big promotions followed the release
of a new Windows version.

I wonder: Would Poole have stayed, if not for those e-mails released
as part of the Windows Vista Capable lawsuit? Vistagate isn't over
yet. "

http://www.microsoft-watch.com/content/corporate/poole_realizes_his_unlimited_potential_elsewhere.html?kc=MWRSS02129TX1K0000535

The Airlines and the News Channels

Why is the FAA all of a sudden so concerned about the spacing between plastic clasps on wiring in all of these airplanes? Why are they shutting so many planes down this week, when the immediate danger is almost nothing? Why was the FAA unconcerned at best, deliberately harmful at worst, only a few weeks ago?

Well. Why is this headline from the AP getting so little coverage? "Cheney, Others OK'd Harsh Interrogations" (where "Harsh Interrogations" really means "Torture").

Diversion. Every politician uses diversionary tactics, e.g. releasing bad press late on a Friday. In this case maybe the administration thought they needed to truly clog up the news channels with something benign but something that dramatically affects thousands of citizens. It's not unimaginable.

Any congressman or woman who does not support immediate criminal investigations of the adminstration at the cabinet level should not be re-elected.

I truly get the impression that the current action by the FAA has nothing to do with the FAA or the safety of air travel. This may be an irrational theory, but are there any _rational_ theories for the current FAA fiasco?

Who should buy Yahoo?

WalMart.

IBM's New Reality Runtime

Jerry Cuomo writes about IBM's "New Reality Runtime" - a JVM implementation, etc. designed for lightweight, RESTful web services, developed using simple, dynamic languages.

We Lose A Little On Every Loan But Make It Up In Volume

(via Steve Dekorte) a slide show explaining how "Collateralized Debt Obligations" full of bad loans became AAA rated. Sad, but true, and further essential reading for all citizens and investors.

If you've ever wondered how and why corporations can carry potential debt "off balance sheet", this explains it using stick figures. Even politicians could understand it. Share with your congressmen and women.

"But crap is crap, isn't it? I don't get it.

Economic Crisis Backstory

Mark Watson points to a Fresh Air episode that is essential listening for all citizens of the U.S.A. and anyone following the current crunching sound you hear in the financial markets.

Crunch

From the Washington Post on the continuing crunch for credit...

"This is the equivalent of a hurricane coming, and staging food and water at the site," Education Secretary Margaret Spellings said in an interview. "Our responsibility is to be prepared for every eventuality."

Some colleges are already saying that their students' expected financial needs for the fall semester are outstripping what lenders are willing to provide...

The tumult is likely to push hundreds, if not thousands, of firms out of the student lending business, said Mark Kantrowitz, publisher of FinAid, a Web site that provides financial advice for students. Of particular concern is the number of firms leaving the business of making federally guaranteed loans, which, besides parents, are the primary source of financing for students' higher education.

Ripple, quake, crunch, and shake... there's a point in the movie "Titanic" where passengers are playing with the broken ice on the deck after the collision. Not long afterward, reality sets in: the ship is sinking.

Large institutions today cannot tell you whether they've just grazed the iceberg, or whether it's ripped through their hulls. In the case of higher education for next year, it looks like a big gaping hole and there's not enough life boats.

Airplanes

In a matter of weeks we (the usa at least) go from senior FAA politicos ordering whistle blowers to stand down, to the airlines shutting down nearly entire fleets. Umk.

I have reservations to make. Which airlines?

Platform Wars

Here's what I like about this new era of cloud computing: rather than the useless language wars of the old era (i.e. sun/java vs. msft/java-ish), we are now heading toward platform wars. Amazon will add some of the "full stackness" of Google's, and Google will expand as well. And these battles are aimed at adding real value.

Microsoft, Yahoo, whoever else, will have to off similar capabilities, and new advantages. And they all have to appeal to several "stacks". Dotnet won't do it, at least not solely. Java won't either.

The stack starts lower than the virtual machine, just like in the old days. Only funner.

Portland's West-side Open Technology Center

From Steve Morris at the OTBC:
=========================================================

WOTC is a facility at the OTBC in Beaverton (http://www.opentechcenter.com/)
for open source and open technology developers to use without charge
(i.e., not for proprietary development).
We can use it as a quiet work environment or as a meeting place. The OTBC
presentation room is also available when it's not scheduled for other uses.

We are having an organizing meeting for WOTC on Tuesday, April 8, 2008,
at 6:00pm at OTBC. Pizza and soft drinks will be served.
Please RSVP so that we will have plenty of that available.
~~~~~~~~~~~

See http://www.opentechcenter.com/drupal/?q=contact for directions to OTBC.

Topics for discussion include but are not limited to:

- name
- attracting interest/users
- measuring success
- usage guidelines
- access/hours/security
- what do we need in the lab
- TBD/other

Mass transit to OTBC/WOTC:

Trimet bus 67 travels from Merlo MAX Station to PCC-Rock Creek (by Cornell
Oaks business park). Walk from the corner of 158th Ave./Greenbrier Parkway
to OTBC (0.1 mile).

We have a very beginning wiki at http://wiki.cs.pdx.edu/ootc/ . The
account creation password (just to inhibit spammers) is portlandia.
The mailing list is at http://wiki.cs.pdx.edu/mailman/listinfo/ootc .

Thanks. Hope to see lots of you there.
Bart Massey, Randy Dunlap, Steve Morris

John McCarthy

Paul Graham describes John McCarthy as one of his heroes. Hopefully computer science will remember certain people like McCarthy the same way we remember the significant contributions of scientists from other disciplines.

John McCarthy

John McCarthy invented Lisp, the field of (or at least the term) artificial intelligence, and was an early member of both of the top two computer science departments, MIT and Stanford. No one would dispute that he's one of the greats, but he's an especial hero to me because of Lisp.

It's hard for us now to understand what a conceptual leap that was at the time. Paradoxically, one of the reasons his achievement is hard to appreciate is that it was so successful. Practically every programming language invented in the last 20 years includes ideas from Lisp, and each year the median language gets more Lisplike.

In 1958 these ideas were anything but obvious. In 1958 there seem to have been two ways of thinking about programming. Some people thought of it as math, and proved things about Turing Machines. Others thought of it as a way to get things done, and designed languages all too influenced by the technology of the day. McCarthy alone bridged the gap. He designed a language that was math. But designed is not really the word; discovered is more like it.

Several other of Graham's hero biographies are worth reading too.

Standards: Take Your Pick

I don't know much about this process or its result. I've seen bad
standards for no good reason. This sounds like enough money may have
been scattered around to "buy" a standard.

I don't really care too much. Even as standards, the days of MS-Office
formats are numbered, even of those numbers can seem large.

John Dowdell (of Adobe, to be sure) writes: "One of the most shameful
things I've ever seen in technology."

http://weblogs.macromedia.com/jd/archives/2008/04/a_perversion_of.cfm

Modular Data Centers

James Hamilton of Microsoft Research writes...
====
Over the last couple of years, the modular datacenter approach has
gained momentum. Now nearly all data center equipment providers have
started offering container based solutions

* IBM Scalable modular data center
* Rackable ICE Cube™ Modular Data Center
* Sun Modular Datacenter S20 (project Blackbox)
* Dell Insight
* Verari Forest Container Solution
====
Link: http://perspectives.mvdirona.com/2008/04/02/FirstContainerizedDataCenterAnnouncement.aspx

AIR Linux

From Adobe regarding AIR for Linux...

We have posted the first publicly available alpha build of the Adobe AIR runtime installer and SDK for Linux. In addition, we are linking to several sample applications that currently run on AIR. You should consider these builds only alpha quality and not yet feature complete...

It is our intention to release future versions of Adobe AIR simultaneously on Linux, Mac and Windows

As chromatic pointed out in a comment to a previous blog post, this is not completely open, and the term "linux" will probably continue to mean x86, 32-bit, and certain distributions of Linux selected for testing. But we'll see. Maybe that should be submitted as a bug?

But there's more to read about, e.g. this from Artima...

Adobe also announced today that it is joining the Linux Foundation with the aim to make rich user experiences easier to develop and deploy on Linux desktops. Among the initial platforms that will see the benefit of Adobe's contributions in this area is Ubuntu, already among the most UI-rich and user-friendly Linux distros.

Artima referred to the release notes which in turn refers to a discussion forum for AIR for Linux.

I'll tuck this tidbit down here at the bottom of the post: Wrox has some AIR books coming out. The Beginner AIR book is already shipping from Amazon.

The 60's

Following snow at the _beach_ this weekend, and more snow at home this morning, temperatures are getting back into the 60's this week. No fooling.

My last weather report for a while, we hope.

Malaysia's Web Lesson

An AP report (link TBD) quotes the Malaysian PM... "a serious misjudgement" to rely on government-controlled press in recent elections. Opponents used open blogs, texting, etc.

Let's hope the National Front has decided to be open, and that they've decided not to attempt shutting down the internets.

Made to Stick

I've been making it stick all these years, and now I find it's already been Made to Stick!