On Messages and Files, and the Rest of the Future of Data

Mark Nottingham's questions on messages and files...

Why do messages — which in developers’ minds, inevitably means “short-lived” — have an advantage over files? Making your OS message-based seems to just add complexity, not make data more portable and ubiquitous.

...have me thinking about many things which I will boil down to a small hypothesis.

When we think of messages, we should say "messages are documents, typically shorter in length and shorter in duration". Then "message passing" is like "document passing". When we think of longer lived documents and messages, we should think merely of "passing" them to a service that has the responsibilty for the life of that document. Of course this should be independent of OS or file system. (Those are implementation details only. Our applications should rise above them.)

From working on systems in a half dozen domains I have come to the conclusion there are three simple patterns we should lean on for making documents and messages persistent.

• Tuples Spaces (think Ruple Forums) for in-progress, state machine-like transactions and collaboration.
• Versioned document trees (think Subversion's file system) for long-lived, shared, document editing.
• Star Schema-based storage (think Sybase IQ, a simple, low-maintenance, scalable database technology) for read-only transaction history and analysis.

I deeply believe (it is my hypothesis) that these are the only patterns you need, and the implementations going forward can become far simpler and adaptable than all of today's cruft. As for "messaging" (the gerund, i.e. SOAP and its ilk), WS-xxx is a band wagon approach toward more complexity. Messaging protocols are a means to an end (i.e. getting a message into one of these persistent "end point like" locations). All collaboration can take place via these "persistence" mechanisms. They are actually coordination mechanisms, and persistence is a by-product since collaboration is frequently different-time and/or different-place, and even when computing is solo or collaboration is same-time and same-place, you often would like a history.

This is the rest of the future, so to speak. The present vision of rest is blinded by HTTP. Although HTTP is an application layer mechanism, it does not address application semantics in a deeply meaningful way. Consider that any of the above persistence mechanisms, and many others, can be defined with or without HTTP. Saying HTTP is sufficient is like saying the verbs "go", "stop", "turn left", and "turn right" are sufficient for defining any children's playground activity. The real meaning is in the decisions of when to take these actions and how to interpret the response of the collaborators in the activity.

These three persistence mechanisms match the needs of the kinds of decisions our systems make for us, and the kinds of decisions we wish to make with them. You can tell me how crazy I am on this wiki page.

Them's Fight'n Words

Anders Hejlsberg: There is clearly a performance aspect to it. One possible solution would be to say, "There are no value types. All types are heap allocated. Now we have representational identity, and so we're done, right?" Except it performs like crap. We know that from Smalltalk systems that did it that way, so something better is needed.

This is really one more thing generally not worth worrying about. *This* is the real lesson of Smalltalk in the real world. I have seen precious few Smalltalk (or Lisp, or ...) applications that required this kind of minutiae. Large matrices of doubles in these cases, usually, are the culprit. In these cases the work arounds are not enough to justify complicating the language for everyone.

A good 64-bit data representation will make this point moot in just that many more scenarios. But when Smalltalk and other simple dynamic languages are cruising with 64-bits, the complicated languages like C# will *still* be burdened with all their interfering mechanisms.

What to do when your language is not dynamic enough?

When a language is formed by piling feature on top of feature, you run out of gas sooner rather than later. Such is the case with "partial classes". Could one have foreseen the desire for "partial methods"?

What do you do when your language is not dynamic enough, but the next feature may be the straw that breaks the camel's back? You resort to code generators.

dotnet *demands* a good dynamic language ASAP.

Why Societies Collapse: Jared Diamond at Princeton University

If you enjoyed Jared Diamond's Pulitzer Prize-winning Guns, Germs, and Steel, you might be interested in reading this transcript of his talk on the collapse of past societies.

If Montana were an isolated country, Montana would be in a state of collapse. Montana is not going to collapse, because it’s supported by the rest of the United States, and yet other societies have collapsed in the past, and are collapsing now or will collapse in the future, from problems similar to those facing Montana. The same problems that we’ve seen throughout human history, problems of water, forests, topsoil, irrigation, salinisation, climate change, erosion, introduced pests and disease and population; problems similar to those faced by Montanans today are the ones posing problems in Afghanistan, Pakistan, China, Australia, Nepal, Ethiopia and so on.

Answering a question on whether technology is the answer to envionmental collapses of the future...

The second thing is that the lesson we’ve learned again and again in the environmental area is it’s cheaper, much cheaper and more efficacious to prevent a problem at the beginning than to solve it by high technology later on. So it’s costing billions of dollars to clean up the Hudson River, and it costs billions of dollars to clean up Montana, it would cost a trivial amount to do it right in the beginning. Therefore, I do not look to technology as our saviour.

A Good Read

The industry needs encouragement to make modest, but significant, advances, like the kind advocated for by Jon Udell.

Bravo. And encore.

A Lesson in Rhetoric: Bush Wraps Self in Flag

Bush is speaking now in a clip on CNN. He is using phrases like, "America did the right thing."

Why defend yourself and your administration when you can defend America?

Next phrase: "I could take the word of a madman or I could protect Americans."

Is this implying George Tenant, Dir. CIA, is a madman? 8^)

Has rhetoric ever been as blunt in an administration?

Deep Problems with Model Driven Architeture

My take on Model Driven Architecture as it is currently being defined is this:

The current definition, whether it is Microsoft's or the OMG's (PDF), is that the approach assumes no improvements on the underlying technologies. In particular, MDA assumes no *simplifications* on the underlying technologies. Rather in these approaches, MDA *is* the simplification.

The problem then is that an MDA is like a compiler in the way compilers were typically described in classrooms about 25 years ago:

1. The compiler generates assembly code.
2. The assembler generates binary code.
3. The linker generates an executable.
4. The loader loads it and initializes the PC (program counter).

The Microsoft approach professes to be more "agile" (love that word) than the OMG approach. Still it does not seem to be an attempt to improve our tools so much as contain them and force them to submit themselves to the MDA interface.

Should they be excused?

Phil Windley interprets Jim Flowers to be saying that...

Jim's fundamental point, I think, is that getting real people with real problems (i.e. elections office staff) in the debate will add significantly to level of discussion and move us closer to real solutions.

But this seems to ignore that so called real people with real problems *already* purchased and implemented unbelievably corruptable electronic voting solutions in order to replace the existing corrupted mechanical voting solutions.

Jim's position also seems to ignore that at least in some cases (e.g. in California) the use of such systems is not only deplorable, but it is explicitly *illegal*.

Yet more evidence that computer programmers are effectively writing the laws in many cases through the implementation of systems that fail to obey the law. At least in California the situation appears to be improving. Whether machines will be corrected before they're used again, and what retributions that would entail, is yet to be determined.

A Matter of Course

The state's school superintendent has proposed striking the word evolution from Georgia's science curriculum and replacing it with the phrase "biological changes over time."

Cox repeatedly referred to evolution as a "buzzword" Thursday and said the ban was proposed, in part, to alleviate pressure on teachers in socially conservative areas where parents object to its teaching.

I say why bother. Let evolution take its course.

Risk Analysis and Decision Making

Ted Neward attended a talk on risk analysis and decision making. This is the gem for technology facing (and technology fascinated) developers. Heed...

Cost-benefit analysis is a skill, not an arcane art, and more technical leads and architects need to spend more time working on it. But we don't, because it's not NEARLY as cool as working with the latest O/R mapping layer or tool, despite the fact that using the latest O/R mapping layer or tool in of itself represents a risk that should be carefully examined using a cost/benefit analysis.

If only we got as excited about adding 25 points of business value to a system.

The Psychology of Query Programming

The intent is there, the accomplishment is there. The only bummer is the notation of the query language itself. Who wants to write code like that?

Taking a Stand for Our Public Airwaves

Whatever your political stripes, if you are an American who believes the airwaves are still a public resource, do all of us a favor and participate in this brief boycott of CBS.

Remember, the airwaves are licensed and renewed by the public to CBS. They are not privately owned.

Descriptive Temporary

Michael is examining some code in Smalltalk and C. Here it is...

" The original Smalltalk example. "
canvas displayLineFrom: (topPoint x - gibDistance @ topPoint y) to: (topPoint + 1).


/* The original C example. */
canvas.displayLineFrom_to( Point.asPoint(topPoint.x() - gibDistance, topPoint.y()), topPoint.MoveBy(1,1));

I would not hesitate to use the "Descriptive Temporary" pattern. (Is there one? Maybe there should be.) Taking this to the extreme, I might end up with something like the following in Smalltalk and C, respectively. Keeping to short methods, I would not have much more code than this in a single method or function. I think this style, while longer, is easier on my brain...

" The Smalltalk example with descriptive temporaries. "
x := topPoint x - gibDistance
y := topPoint y
lineBegin := x @ y
lineEnd := topPoint + 1
canvas displayLineFrom: lineBegin to: lineEnd.


/* The C example with descriptive temporaries. */
int x = topPoint.x() - gibDistance;
int y = topPoint.y();
Point lineBegin = new Point(x, y);
Point lineEnd = topPoint.MoveBy(1, 1);
canvas.displayLineFrom_to(lineBegin, lineEnd);

Now, this also has the effect of making Smalltalk more readable than C since there is less line noise. But I learned Smalltalk before I learned C.

Addressing DLL Hell with 40 year old technology

Ted Neward anticipates solutions to runtime downloads, DLL hell, and other nightmares, realizing Java is in a similar boat...

And lest anybody start to think otherwise, the JVM presents the same problem as the .NET Runtime, and when we solve the .NET Runtime versioning issues, we'll have solved the JVM ones, using the same solution.

The linking problem has been solved for Java... there is the GNU Compiler for Java, as well as commercial native Java compilers. JNLP is intended to solve versioning problems. I have used it, but never in the presence of versioning problems.

The bigger answer is time. Systems will thrash less as they mature. This is for better or worse, since the ability to change even undesirable features is inversely proportional to the number of dependencies applications have on them.

Now is a good time to bring up a theme of mine: the software world has a few examples of highly dynamic systems; the rest are evolving to become more like them, truly. Now is a good time to point out that highly dynamic systems like Lisp and Smalltalk have a long, proven history of backwards compatibility:

1. Because they are mature. They've been around for decades.
2. But also because they allow themselves to be amended and redefined down to their core.

If your application requires a feature that comes out of the box in another dialect or from a previous version, chances are good you can take that feature with you to a new dialect or version. If you wish the root class Object had a feature the vendor did not think to include, you can add it yourself just in the applications that you designate.

The Right Stuff

this stuff is being integrated at the platform and framework levels on both the MS and non-MS side so that developers can no only take advantage of it without having to understand all the nitty-gritty details, but also gain maximal benefits with minimal code.

There be dragons.

I'm getting punchy. I better go.

Messages and Objects, Again

Good points on all sides of the ongoing debate on messages and objects and distributed systems.

I will just add one observation...

Dropping a feature is not the same thing as fixing a feature.

More on Inheritance Being Evil

Much lamenting of the perils of using inheritance in an OO language. But these perils are part of a well worn path. Take heart and remember to learn from the mistakes of others,

White Box and Black Box frameworks... it's a matter of evolution.

Who's the Loser?

I can only whine a bit in response to this desire to create new flow of control sytax.

Yesterday I posted a new way to define functions and their tests contiguously using a new syntax in Scheme. It took about 10 minutes and less than 10 lines of code.

Likewise in Lisp as well as Smalltalk defining new control structures is childs play, mainly because the control structures in those languages are defined using the very same mechanisms that are available to you, the dear programmer.

Yes, I know, I am a loser to even bring this up.

Is COM interop more stable than dotnet's?

I'm recalling something for the the dotnet reality check...

Because dotnet is a more "complete" object model than COM (inheritance and shadowing of public and private aspects) and because different languages (VB.NET and C# to name two) have somewhat different rules for inheritance, it may be that dotnet has more interop problems than COM. I couldn't say because I don't know COM.

A problem with dotnet inheritance was documented some time ago. I don't believe it has been fixed.

Minimum Requirements for One Runtime, Multiple Languages? Less is More

Dare Obasanjo follows up on Jon Udell's investigation into all things dotnet with a piece on language neutrality.

Two points:

• The dotnet framework to date is "language neutral" within a very narrow definition of neutrality.
• Should we care?

What is really needed in language neutrality? A few thoughts:

• Shared nothing, scalable threads. Let our languages run in the same address space, but force them to say at an arm's length from each other.
• Short-circuited message passing. Let our language collaborate the same way whether they are together or apart. Just make it more efficient if they are together.
• I/O and other system services. Let them use a common set of system services, virtualize the limitations of the host OS.

There is no reason why one could not build a Longhorn on this "less is more" model, and the result would have more potential (flexibility, scalability) than the one that appears on the horizon, as far as I can see.