Java and XUL

A recommendation to combine Java with XUL.

Remix: The Defense of Whitey

Update: Jon Stewart on the Daily Show made a similar statement (almost word for word) about the definition of an activist court. He apparently said it first in New York, but I didn't get the show until an hour+ later on the west coast. I was hoping his staff reads my blog. Obviously a vain idea, considering the timing factor. 8^)

BTW couldn't the claim realistically be made that the most "activist" decision any US court has made in recent memory is the Supreme Court decision to declare Bush the winner in 2000? Water under the bridge, right? But I don't recall this decision ever given countenance by consitutional scholars.

From one book on the matter: "Digging deeply into their earlier writings and rulings, Dershowitz proves beyond a reasonable doubt that the justices who gave George W. Bush the presidency contradicted their previous positions to do so."

Reinforcing the point. "Activism" is clearly in the eye of the beholder.

The Defense of Whitey...

Simply brilliant.

By the way, an "activist" court is one that makes decisions *you* don't like. And a "state's right" is a decision that you *do* like. (For any definition of "you", left or right.)

Making language translations stick

The automated french translation of "Making it stick" comes out something like "He's making the stick".

Well, that's not what I intended for the title of this blog. A better translation might be...

"Rendant quelque chose de dernier de mani?re permanente."

...which is not as short as "Making it stick" but means something like "Making something last permanently."

Explaining ifTrue:ifFalse:

The challenge is expressed this way...

try explaining to someone how Smalltalk's ifTrue:ifFalse: works sometime

The message looks like this...
someObject ifTrue: [someCode] ifFalse: [otherCode].

See if my attempt makes sense to you:

If someObject is true, then someCode is executed. Otherwise otherCode is executed.

Is that basic enough for non-Smalltalkers?

The Concepts are Hard

Fred Brooks, via Chad Dickerson, displaying ancient industry wisdom...

I believe the hard part of building software to be the specification, design, and testing of this conceptual construct, not the labor of representing it and testing the fidelity of the representation. We still make syntax errors, to be sure; but they are fuzz compared with the conceptual errors in most systems.

Yes!!! Consider the recent Mars Spirit Rover problem... it was an unconsidered scenario, not an ill-specified proof, that disabled the Rover.

We need *simple* tools that stay out of the way for us to have more time to consider problematic scenarios.

And we need better simulation tools and conceptual analysis tools. For example was Spirit's flash file system modeled at any level as a concurrent state machine? What kind of analysis would have caught the missing transition from a failed transmission to expunging the leftover files?

Note that this is also what Longhorn architects should fear the most... the unknown unknowns, as Rumsfeld would put it.

Duck Typing an Invoice?

Has the nature of data modeling changed in the era of the Semantic Web?

All around the world, as I write this, developers are struggling to create models for invoices and other "simple" business documents into IT systems. All around the world, multiple efforts new and old continue to attempt to zoom in on a definitive model of what it is to be an invoice...

There is *no such thing as an invoice* in the classical data modelling sense... There seems to me to be a fundamental mismatch here between the classical software model of an invoice and the reality of real world invoices.

Think back to how you recognized that invoice as an invoice. You found a bunch of attributes which you associate with invoices. You found enough of them in your analysis of the piece of paper to conclude that it was statistically speaking, more likely than not to be an invoice. It if walks like a duck...

I think this is a case of both/and rather than either/or. Financial systems should not be expected to "scan a document looking for invoice-y-ness".

Yes, there are umpteen definitions of an invoice. No, formally modelling any one of these is not an endless excercise.

You see, any given accounts payable system only has to understand *one* of these. So the one I use understands SAP's definition of an invoice. The one you use may understand Great Plain's definition.

Will there ever be a universal definition of "Invoice" for the Internet? Or will the Semantic Web just allow us to send bits that resemble invoice-y-ness?

I'm not sure we'll need universality. I kind of anticipate the arrival of a marketplace for "adapters" and "translators" for documents, and some of those will employ fuzziness. There is room for formal documents to become more "semi-formal" making them more "social". But most of today's formal bits for formal systems, like subledgers of purchasing systems, must remain formal.

My AP system does not need a universal model, nor does it need to understand umpteen invoice models, but neither does it need a neural net or a rule base. It does need at least one formal definition of an invoice, which obviously is not insurmountable.

Maybe I'm a quack.

Things aren't as easy as one suspects

The good news is that people inside MSFT are considering the complexities of WinFS.

Dare Obasanjo writes...

things aren't as easy as one suspects at first glance at WinFS

Insert appropriate "I told you so" from here and here.

Unfortunately the notion that a solution could in any way resort to "even if it is just coming up with application design guidelines" is *completely* unacceptable for a 21st century file system or database.

There is a philisophical schizophrenia taking place inside the head of WinFS. Is it a file system? Is it a database? Is it cheap and simple? Is it transactional and robust?

I personally do not believe WinFS should be cast as a file system at all. Rather WinFS should be a complete rethinking of what a database is. WinFS should eat the lunch of SQL Server, not NTFS. But the technical problems are deep, and the business of introducing a replacement for SQL Server are enourmous compared to a replacement for NTFS.

However, any practical use for WinFS by *definition* makes it also a replacement for many current uses of SQL Server. Unless you just want a way to store pictures. That's trivial, and that alone is not the heart and soul WinFS being presented to us.

As for the security problems, the answer there is also obvious yet painful. A real innovation in Longhorn would be to use "trusted computing" in the hardware and software to implement a capability-based operating system.

XAML, camel. Don't just play at being engineers. Create something really useful.

Camp Smalltalk 2004 - Yes

Blaine asks who else will be at Camp Smalltalk 2004. I'll be there.

Dynamic loading in Lisp, Smalltalk, and dotnet

Gordon Weakliem speculates on dynamic loading in dotnet...

I don't know of anything like this for .NET, though AppCenter may do that for you. On .NET, you can replace assemblies in a server application, but that forces an application restart, so you lose any state stored on the server. It sounds like in James' scenario, he relies on using loadable components along with an administrative API to get the server to reload components, which makes me wonder if you could do something similar for a .NET web application, where the application itself is just a driver that loads components from elsewhere. It certainly wouldn't be a standard design though.

I am sure I know less about dynamic loading in dotnet than almost anyone, and MSFT has the issues in hand. I wonder though how language model of dotnet complications may wreak havoc on a simple solution to the problem.

One issue that comes to mind is stack-allocated objects...

Imagine module A defines a stack-allocated and some system should load a newer version of A. Now imagine the newer version of A redefines the stack allocated object, e.g. gives it a new "shape". What does this do to the objects allocated on the stack and the code in other modules that manipulates them?

This is one of the key differences between the "everything is an object" as defined by dotnet, and the "everything is an object" as defined by Smalltalk.

What Happens to Java/Linux When Longhorn Ships?

Wesner Moise asks this question. Clearly MSFT is a force to be reckoned with. They are making a huge investment to bring even more developers more solidly into their API camp, which is not easily emulated.

What I hope happens to Linux is that significant developers continue to realize that the linux core is solid, and that good systems should still be built in layers. Microsoft is bundling so much into their "core" for business reasons, it's a bet that may not pay off as well as hoped.

It's not good software, but it is great business.

And speaking as a software developer, as dispassionately as I can speak about something I am passionate about, there are precious few gems in what I have seen so far.

The two real gems to me seem to be Indigo and the Business Framework class library. Ironically these are two components that are the most independent of the "core".

The two primary losers seem to be WinFS and the whole XAML, Avalon stuff. Not that they are not aimed in a worthy direction, I just don't see a lot of bang for the buck.

The Linux and the Java worlds (they are still separate from each other) have the benefit of more freedom to innovate on *top* of those platforms.

The Longhorn core is too big and will suck too much energy just to begin to grok the complexity of the singular vision.

But I have never been good at business math. The strategy from a business perspective is sound. It has obviously worked in the past.

Economic forces to get or keep a piece of the rest of the pie should remain strong enough to promote innovation on the non MSFT platforms. I don't see others being plowed under so easily. There is too much wrong with Longhorn and too much incentive to compete.

Complexity in the Enterprise: Same as it ever was?

Phil Windley makes a case about SOA and complexity. I'll buy the conclusion, but the argument needs more realism. Here goes...

First, no one's IT architecture looks like the first figure. That figure is too neat. The real picture is somewhere between the first and the second, but the integration lines are supported by an evolution of technologies from previous-generation message queues to file transfer to re-entering data by hand from one application to another (or worse, by hand from a print out).

Second, a future SOA architecture is unlikely to look like the second figure, for several reasons. One reason Phil makes indirectly: that picture is so complex as to be unmaintainable. So while *someone's* IT architecture may end up this bad, the typical architecture almost certainly won't.

Other reasons that figure is unrealistic: business processes provide a natural organizing force for large grained services. I have trouble believing the architecture of the future will be made of hundreds of interacting components without some large grained (logical, if not physical) hubs of service organized by purchasing, inventory, manufacturing, sales, general ledger, etc.

A third reason I have not seen mentioned much in the past year, but I believe will hold even as the business cycle improves. That reason is vendor consolidation, two forms of it. On the IT side, even Phil's argument points out that an SOA benefits from organizing forces. One of those forces is for an IT architecture to drive toward fewer vendors supplying products into the architecture. On the vendor side, well, vendors will want to be on those select lists, and so will continue to merge and make overall sense of the resulting product strategy.

On to the three problems Phil enumerates for an SOA...

First, "No one team understands or controls all the moving parts". But that is true today without a doubt.

Second, "Change management is more complex". Maybe. But almost by definition a useful SOA has to evolve toward fully embracing the parameters of being "loosely coupled". Change management is complex today, to a large degree because there has been little enterprise architectural guidance and pieces thrown together have resulted from little attention to "loose coupling" principles.

Third, "Separation of concerns is more difficult". Unlikely, for a few reasons. Concurrent, even ahead, of the movement to an SOA, is the movement of IT organizations themselves to an Enterprise Architecture alignment. More attention than in the past is being paid to enterprise concerns prior to deployment, as far ahead as the capability roadmap, the project concept, or at least the project planning stages. The people who support the IT architecture, pay for it, and even those who design it, are sensitive already to the problems inherent in such a picture and are already showing signs of addressing "separation of concerns" at the enterprise level through architecture and through budget (Enterprise Program Management). A challenge is to find enough balance between avoiding constant interaction and indirect impacts of minimizing communication. (Plug goes here for new collaboration tools like Wikis and RSS to lower the effort of finding "just enough communication".)

Lastly, a default argument could be made that a large movement toward an SOA is inherently dependent on SOA vendors addressing these problems effectively. This leads to somewhat of a Catch-22, but we can expect the usual curve of early adopters to begin working out the kinks.

I don't believe many of these problems will be addressed by "web service intermediaries" per se. Although the infrastructure is still immature, the real problems for the enterprise are at the boundary where the services architecture meets the business architecture. The real problems of heading toward a world of rich services is how to get them to play at the business level. How do you keep them agreeing on a correct chart of values, a correctpart and product hierarchy, roles and responsibilities, up-to-date with engineering change notices, etc.

These are not insurmountable. To reiterate I believe the movement toward an SOA will be preceeded by a movement to realign IT for the enterprise architecture and go hand in hand with vendor and product consolidation, and less custom development. Without these steps, then yes, the future looks unmanageably complex. *With* these steps the future is still complex because the changes will be gradual and IT will continue to deal with the as-is and the to-be concurrently. But at least there is hope if the SOA products mature and rational principles are in place for their adoption.

When will WS == Working System?

WS-xxx is supposed to be composable. Great. How do we tell the potentially solid connections that *work* from the illogical, or simply broken, connections that don't work?

I suppose that is where Indigo will give you Microsoft's working combinations based on some declarative attrbutes. (And those attributes themselves may work in Microsoft's C#, but not in some other language provider's dotnet compiler.)

How do I figure out in some Java implementation of WS-xxx what to provide to line behaviors up with the Indigo implementation? This is probably more of a mess than necessary.

And where are all those web services, anyway? Does their apparent absence have anything to do with the WS-xxx soup? Shouldn't one just use HTTP[S] after all?

Note the Subtext

Sean McGrath echos Don Box, but isn't the subtext obvious?

Whence the network is the computer?

Ironically(?), the PC is still an API battle ground. APIs are not as good as the wire is for interop, but never doubt their importance to the industry.

Just great

One of the security firms doing business in Iraq is named CusterBattles.

Nope. It is named for the two principles, Scott Custer and Mike Battles.

It's Morning in America: So Let's Wake Up!

There are just a few basic points the coming presidential campaign should be based on. Every other issue should be brushed aside with just enough rhetoric to play to the center and get the conversation back on track.

It's called the center for good reasons. Most people are not far right neo-cons (even most true conservatives are not neo-cons). Likewise most people at best don't care about gay marriage. Sorry to those of you with strong opinions on either side of that one, which is simply a messy intersection of civil laws and socio-religious customs. The Democrats have to stick to the simple message this year.

No, the point of this election year is to get back to the center, or something resembling it. Let's call it the "moral majority".

The moral consequences of destroying countries and not rebuilding them is bad enough. But the practical consequences are worse. The Bush administration's foreign policy is undermining US security.

What makes a secure system?

As people debate the risks of the Windows code being stolen, and people debate the merits of open surce for making code more secure, I am wondering about a few things:

• Writing code in a language that allows buffer overruns is not safe.
• Writing office applications for a LAN that assumes all particpants are associates and then putting them on the Internet is not safe.
• Writing operating systems that run every application with full user rights (or worse) is not safe.

Open source that is poorly designed is not safe. Closed source that is poorly designed and then released in the open is even less safe.

More Thoughts on the Future of Persistence

Jay Han picks up the discussion on the future of persistence and poses some questions...

What are the experiences from orthogonally persistent OSes? Did they make say IPC any easier?

I can't answer from the OS perspective, but I can answer from the Gemstone Smalltalk perspective (since Smalltalk has been accused of being an OS, and indeed has demonstrated itself to be a pretty complete one).

Did Gemstone/S make IPC any easier? Of course the answer is yes and no.

Yes, because it implements a transactional (ACID) shared memory, so all applications are coordinated in the loosely coupled manner of a database. And yes, because the persistence between one app and the database is transparent, i.e. that relationship is nearly identical to a Smalltalk app and its image file.

No, because it is specific to Smalltalk. Integration with other languages is no better, i.e. you need to use COM or CORBA or C or SOAP or... Next question...

Amoeba had more RAM then disk and it didn't get virtual memory until late. Obviously there must have been some problems making data persistent there. Was there ever another system that had larger primary memory then secondary?

All I can say here is that Gemstone/S applications perform best when most of the persistent pages of objects fit in a Gemstone shared page cache and most of the applications run on that node that has a cache.

Palm PDA and cell phones have transparent persistence. What are other examples of a set of applications utilizing transparent persistence today?

Gemstone applications in insurance, banking, transportation, manufacturing, etc. are all based on transparent persistence. This is not entirely true because by the time a large multi-user application gets into production there is a good bit of code managing transactions, conflicts, etc. so the persistence is no longer entirely transparent. But there is no O/R mapping and there is no concern about the objects not fitting in RAM. Instead of persistence per se, the emphasis is on coordination.

Now I would argue strenuously that Gemstone/S does not have the right coordination model (i.e. a shared transactional object space). In this model *everything* in RAM is transactional (at least everything strongly connected from a persistent root). And everything in an external database is also transactional, so every connection between the transactional RAM and the transactional relational database requires a two phase commit transaction, or some confidence that the two phase rules can be relaxed.

A better model is not to make everything in RAM transactional, rather the application should use specific coorindation mechanisms, in particular one of:

• tuple space
• versioned tree
• star schema dimensional facts

These mechanisms are kinds of "databases" that could be implemented even more simply in MRAM. An application that may itself be in MRAM should still coordinate its activities with these explicit coordination mechanisms that may or may not have multiple, distributed client processes.

Compiling Efficient Python

[Update: Michael Salib's PyCon 2004 session will be addressing exactly this topic --- "Faster than C: Static Type Inference with Starkiller"...

This dynamism makes programming in Python a joy, but generating optimal code a nightmare. Yet while the presence of such abundant dynamism makes traditional static optimization impossible, in most programs, there is surprisingly little dynamism. For example, in most Python programs:

• all class and function objects are created exactly once
• class inheritance relationships do not change at run time
• methods are not added after a class object has been created
• most expressions have exactly one type; the vast majority of those that have more than one type have only a few types

The flip side is that what little dynamism a particular program makes use of is often absolutely vital.

I have developed a type inference algorithm for Python that is able to resolve most dispatches statically. This algorithm is based on Ole Agesen's Cartesian Product Algorithm for type inference of Self programs. I have built a type inferencer for Python based on this algorithm called Starkiller. ]

As folks are discussing, C Python's simple implementation has a cost/benefit. Compiling efficiently may require some kind of "optimistic with fallback" approach. Assume the internal representationss are not messed with, and so compile the code to be optimistically efficient. If the internals become messed with then flip the bit and fallback to the simple implementation for that object.

This is similar to optimistically compiling for efficient data representations. For example when the code is about to do some math, compile in-line a test for the data type. Branch to multiple paths of code based on the type. Create a branch for in-line integer math and/or a branch for in-line double math and also include a branch for the fully boxed math.

Another approach that should not be overlooked is whole/partial program analysis. When moving from development to production, whole program analysis could be employed to optimize the specific application or groups of packages. This approach has been employed for Scheme with a good deal of promise...

Stalin has been tested on a suite of benchmarks whose length ranges up to a thousand lines. Stalin outperforms Scheme->C, Gambit-C, Chez, SML/NJ, and even handwritten Ada, Pascal, Fortran, and C code, on most of these benchmarks.

It's About Time

I've read some criticism of Victoria Livschitz's interview mostly correcting the usual misunderstandings of the average software developer or manager. But consider this...

Consider a few common concepts that people universally use to understand and describe all systems -- concepts that do not fit the object mold. The "before/after" paradigm, as well that of "cause/effect," and the notion of the "state of the system" are amongst the most vivid examples... The sequence of the routine itself -- what comes before what under what conditions based on what causality -- simply has no meaningful representation in OO, because OO has no concept of sequencing, or state, or cause.

She makes a good point here, and this applies not just to OOP. We build the concepts of time, sequence, and causation into applications from scratch when they are required by the customer. We have essentially no support for these at the language level.

How does a developer typically denote that some event has occurred? Like this...
has_occurred = True

How does a developer typically recall that some event has occurred? Of course sprinkly these liberally...
if has_occurred: ...

We have the State pattern to manage behavioral differences based on the current state, but we have almost no common patterns for behavioral differences based on the history of state.

How much runtime should a runtime need?

How much runtime should a runtime need? The OS and frameworks developers have to work with today are so much bigger than they need to be. Let's remind ourselves of what it takes to port an OS that has a vector graphics GUI, an OO file system, garbage collection, etc. to bare metal...

The background is the Mitsubishi has a nifty single-chip RISC computer that integrates RAM on the same chip as the CPU. But they didn't have any software for it. So they got a bright summer intern named Curtis Wickman to port Squeak to it.

Curtis had to write all the device drivers from scratch, including a display driver, the mouse and keyboard handlers, a Flash RAM file system, a loader, and sound output.

This took four to six weeks, I think. However, we were then able to put a generic Squeak image onto it and it looked and behaved exactly as it does on a PC or Macintosh. Even though we knew intellectually that this would be the case, it was somewhat mind-bending when Alan grabbed the mouse during our demo began doing an unrehearsed demo and everything worked perfectly!

The amount of code required for this "bare machine" implementation is quite modest; 2000 lines of C and a hundred or so of assembly code, as I recall. -- John Maloney

Meta Stuff and WinFS

I think automated meta stuff is the least of issues to be concerned about with WinFS.

I've been watching the PDC video presentations on WinFS. Having worked in the distributed object oriented database industry for a number of years, I can foresee potential nightmares that are being portrayed as dreams in these videos.

Why not make a real database more manageable for applications and end users? As it stands WinFS seems about half baked and certainly schizophrenic.

Conceptually WinFS is near the top of appealing Longhorn concepts. Practically, WinFS comes across as a needing a good bit more reality infused to get into production.

When Everything is Persistent...

Another recurring thought on databases as a coordination mechanism...

On messages, files, and persistence: when everything in your runtime is transparently persistent, and you've stripped away all the mechanisms that have only to do with making a transaction ACID, what you are left with is merely a coordination mechanism.

In this future there will still be a use for a tuple space, a versioned document tree, and a dimensional model.

Messages and its Gerund: Reliable Messaging on the Cheap

Mark continues on messages and files with a crucial observation...

Filesystems, after all, are one of the few things that define and hold together operating systems; they allow for unintended uses of data. If you expose everything through a specialized API, everyone who comes into contact with the system ? developers, administrators, and users ? has to learn a new means of accessing it.

True, but I would generalize this to any kind of database. A filesystem is simply one kind of database. In its typical form, it is outmoded. But Mark continues...

Put another way, I can’t think of any good reason why you wouldn’t want to expose persistent state as a file. that doesn’t mean that there can’t be other interfaces, but why lock it up in them?

P.S. Sean, I’m not sure specialised RPOST/RPUT methods are necessary; I think it can be done with a pattern, or maybe a few extra headers.

Mark's click-submit-only-once pattern is thought provoking.

java.util.concurrent

The java.util.concurrent package is a great asset for Java. (JSR 166). The package has been around for a long time from Doug Lea. Hopefully most developers are using it already.

This package is far more valuable than any new syntax that could be added to the language.

Collections, messages, and blocks... and Databases

Blaine Buxton rightly wants to delay his collection calculations, i.e. use objects as a means for lazy evaluation. This is something they're good for, but doesn't get much attention.

In fact this is just what Avi Bryant does in ROE (Relational Object Expressions). Consider that a relational database is essentially a collection, and often a large one, that you would like to operate on efficiently. And so his collection messages are simply delayed as long as needed.

I would expect some of the implementation ideas behind ROE would be applicable to Blaine's intention.

On Messages and Files, and the Rest of the Future of Data

Mark Nottingham's questions on messages and files...

Why do messages — which in developers’ minds, inevitably means “short-lived” — have an advantage over files? Making your OS message-based seems to just add complexity, not make data more portable and ubiquitous.

...have me thinking about many things which I will boil down to a small hypothesis.

When we think of messages, we should say "messages are documents, typically shorter in length and shorter in duration". Then "message passing" is like "document passing". When we think of longer lived documents and messages, we should think merely of "passing" them to a service that has the responsibilty for the life of that document. Of course this should be independent of OS or file system. (Those are implementation details only. Our applications should rise above them.)

From working on systems in a half dozen domains I have come to the conclusion there are three simple patterns we should lean on for making documents and messages persistent.

• Tuples Spaces (think Ruple Forums) for in-progress, state machine-like transactions and collaboration.
• Versioned document trees (think Subversion's file system) for long-lived, shared, document editing.
• Star Schema-based storage (think Sybase IQ, a simple, low-maintenance, scalable database technology) for read-only transaction history and analysis.

I deeply believe (it is my hypothesis) that these are the only patterns you need, and the implementations going forward can become far simpler and adaptable than all of today's cruft. As for "messaging" (the gerund, i.e. SOAP and its ilk), WS-xxx is a band wagon approach toward more complexity. Messaging protocols are a means to an end (i.e. getting a message into one of these persistent "end point like" locations). All collaboration can take place via these "persistence" mechanisms. They are actually coordination mechanisms, and persistence is a by-product since collaboration is frequently different-time and/or different-place, and even when computing is solo or collaboration is same-time and same-place, you often would like a history.

This is the rest of the future, so to speak. The present vision of rest is blinded by HTTP. Although HTTP is an application layer mechanism, it does not address application semantics in a deeply meaningful way. Consider that any of the above persistence mechanisms, and many others, can be defined with or without HTTP. Saying HTTP is sufficient is like saying the verbs "go", "stop", "turn left", and "turn right" are sufficient for defining any children's playground activity. The real meaning is in the decisions of when to take these actions and how to interpret the response of the collaborators in the activity.

These three persistence mechanisms match the needs of the kinds of decisions our systems make for us, and the kinds of decisions we wish to make with them. You can tell me how crazy I am on this wiki page.

Them's Fight'n Words

Anders Hejlsberg: There is clearly a performance aspect to it. One possible solution would be to say, "There are no value types. All types are heap allocated. Now we have representational identity, and so we're done, right?" Except it performs like crap. We know that from Smalltalk systems that did it that way, so something better is needed.

This is really one more thing generally not worth worrying about. *This* is the real lesson of Smalltalk in the real world. I have seen precious few Smalltalk (or Lisp, or ...) applications that required this kind of minutiae. Large matrices of doubles in these cases, usually, are the culprit. In these cases the work arounds are not enough to justify complicating the language for everyone.

A good 64-bit data representation will make this point moot in just that many more scenarios. But when Smalltalk and other simple dynamic languages are cruising with 64-bits, the complicated languages like C# will *still* be burdened with all their interfering mechanisms.

What to do when your language is not dynamic enough?

When a language is formed by piling feature on top of feature, you run out of gas sooner rather than later. Such is the case with "partial classes". Could one have foreseen the desire for "partial methods"?

What do you do when your language is not dynamic enough, but the next feature may be the straw that breaks the camel's back? You resort to code generators.

dotnet *demands* a good dynamic language ASAP.

Why Societies Collapse: Jared Diamond at Princeton University

If you enjoyed Jared Diamond's Pulitzer Prize-winning Guns, Germs, and Steel, you might be interested in reading this transcript of his talk on the collapse of past societies.

If Montana were an isolated country, Montana would be in a state of collapse. Montana is not going to collapse, because it’s supported by the rest of the United States, and yet other societies have collapsed in the past, and are collapsing now or will collapse in the future, from problems similar to those facing Montana. The same problems that we’ve seen throughout human history, problems of water, forests, topsoil, irrigation, salinisation, climate change, erosion, introduced pests and disease and population; problems similar to those faced by Montanans today are the ones posing problems in Afghanistan, Pakistan, China, Australia, Nepal, Ethiopia and so on.

Answering a question on whether technology is the answer to envionmental collapses of the future...

The second thing is that the lesson we’ve learned again and again in the environmental area is it’s cheaper, much cheaper and more efficacious to prevent a problem at the beginning than to solve it by high technology later on. So it’s costing billions of dollars to clean up the Hudson River, and it costs billions of dollars to clean up Montana, it would cost a trivial amount to do it right in the beginning. Therefore, I do not look to technology as our saviour.

A Good Read

The industry needs encouragement to make modest, but significant, advances, like the kind advocated for by Jon Udell.

Bravo. And encore.

A Lesson in Rhetoric: Bush Wraps Self in Flag

Bush is speaking now in a clip on CNN. He is using phrases like, "America did the right thing."

Why defend yourself and your administration when you can defend America?

Next phrase: "I could take the word of a madman or I could protect Americans."

Is this implying George Tenant, Dir. CIA, is a madman? 8^)

Has rhetoric ever been as blunt in an administration?

Deep Problems with Model Driven Architeture

My take on Model Driven Architecture as it is currently being defined is this:

The current definition, whether it is Microsoft's or the OMG's (PDF), is that the approach assumes no improvements on the underlying technologies. In particular, MDA assumes no *simplifications* on the underlying technologies. Rather in these approaches, MDA *is* the simplification.

The problem then is that an MDA is like a compiler in the way compilers were typically described in classrooms about 25 years ago:

1. The compiler generates assembly code.
2. The assembler generates binary code.
3. The linker generates an executable.
4. The loader loads it and initializes the PC (program counter).

The Microsoft approach professes to be more "agile" (love that word) than the OMG approach. Still it does not seem to be an attempt to improve our tools so much as contain them and force them to submit themselves to the MDA interface.

Should they be excused?

Phil Windley interprets Jim Flowers to be saying that...

Jim's fundamental point, I think, is that getting real people with real problems (i.e. elections office staff) in the debate will add significantly to level of discussion and move us closer to real solutions.

But this seems to ignore that so called real people with real problems *already* purchased and implemented unbelievably corruptable electronic voting solutions in order to replace the existing corrupted mechanical voting solutions.

Jim's position also seems to ignore that at least in some cases (e.g. in California) the use of such systems is not only deplorable, but it is explicitly *illegal*.

Yet more evidence that computer programmers are effectively writing the laws in many cases through the implementation of systems that fail to obey the law. At least in California the situation appears to be improving. Whether machines will be corrected before they're used again, and what retributions that would entail, is yet to be determined.

A Matter of Course

The state's school superintendent has proposed striking the word evolution from Georgia's science curriculum and replacing it with the phrase "biological changes over time."

Cox repeatedly referred to evolution as a "buzzword" Thursday and said the ban was proposed, in part, to alleviate pressure on teachers in socially conservative areas where parents object to its teaching.

I say why bother. Let evolution take its course.

Risk Analysis and Decision Making

Ted Neward attended a talk on risk analysis and decision making. This is the gem for technology facing (and technology fascinated) developers. Heed...

Cost-benefit analysis is a skill, not an arcane art, and more technical leads and architects need to spend more time working on it. But we don't, because it's not NEARLY as cool as working with the latest O/R mapping layer or tool, despite the fact that using the latest O/R mapping layer or tool in of itself represents a risk that should be carefully examined using a cost/benefit analysis.

If only we got as excited about adding 25 points of business value to a system.

The Psychology of Query Programming

The intent is there, the accomplishment is there. The only bummer is the notation of the query language itself. Who wants to write code like that?

Taking a Stand for Our Public Airwaves

Whatever your political stripes, if you are an American who believes the airwaves are still a public resource, do all of us a favor and participate in this brief boycott of CBS.

Remember, the airwaves are licensed and renewed by the public to CBS. They are not privately owned.

Descriptive Temporary

Michael is examining some code in Smalltalk and C. Here it is...

" The original Smalltalk example. "
canvas displayLineFrom: (topPoint x - gibDistance @ topPoint y) to: (topPoint + 1).


/* The original C example. */
canvas.displayLineFrom_to( Point.asPoint(topPoint.x() - gibDistance, topPoint.y()), topPoint.MoveBy(1,1));

I would not hesitate to use the "Descriptive Temporary" pattern. (Is there one? Maybe there should be.) Taking this to the extreme, I might end up with something like the following in Smalltalk and C, respectively. Keeping to short methods, I would not have much more code than this in a single method or function. I think this style, while longer, is easier on my brain...

" The Smalltalk example with descriptive temporaries. "
x := topPoint x - gibDistance
y := topPoint y
lineBegin := x @ y
lineEnd := topPoint + 1
canvas displayLineFrom: lineBegin to: lineEnd.


/* The C example with descriptive temporaries. */
int x = topPoint.x() - gibDistance;
int y = topPoint.y();
Point lineBegin = new Point(x, y);
Point lineEnd = topPoint.MoveBy(1, 1);
canvas.displayLineFrom_to(lineBegin, lineEnd);

Now, this also has the effect of making Smalltalk more readable than C since there is less line noise. But I learned Smalltalk before I learned C.

Addressing DLL Hell with 40 year old technology

Ted Neward anticipates solutions to runtime downloads, DLL hell, and other nightmares, realizing Java is in a similar boat...

And lest anybody start to think otherwise, the JVM presents the same problem as the .NET Runtime, and when we solve the .NET Runtime versioning issues, we'll have solved the JVM ones, using the same solution.

The linking problem has been solved for Java... there is the GNU Compiler for Java, as well as commercial native Java compilers. JNLP is intended to solve versioning problems. I have used it, but never in the presence of versioning problems.

The bigger answer is time. Systems will thrash less as they mature. This is for better or worse, since the ability to change even undesirable features is inversely proportional to the number of dependencies applications have on them.

Now is a good time to bring up a theme of mine: the software world has a few examples of highly dynamic systems; the rest are evolving to become more like them, truly. Now is a good time to point out that highly dynamic systems like Lisp and Smalltalk have a long, proven history of backwards compatibility:

1. Because they are mature. They've been around for decades.
2. But also because they allow themselves to be amended and redefined down to their core.

If your application requires a feature that comes out of the box in another dialect or from a previous version, chances are good you can take that feature with you to a new dialect or version. If you wish the root class Object had a feature the vendor did not think to include, you can add it yourself just in the applications that you designate.

The Right Stuff

this stuff is being integrated at the platform and framework levels on both the MS and non-MS side so that developers can no only take advantage of it without having to understand all the nitty-gritty details, but also gain maximal benefits with minimal code.

There be dragons.

I'm getting punchy. I better go.

Messages and Objects, Again

Good points on all sides of the ongoing debate on messages and objects and distributed systems.

I will just add one observation...

Dropping a feature is not the same thing as fixing a feature.

More on Inheritance Being Evil

Much lamenting of the perils of using inheritance in an OO language. But these perils are part of a well worn path. Take heart and remember to learn from the mistakes of others,

White Box and Black Box frameworks... it's a matter of evolution.

Who's the Loser?

I can only whine a bit in response to this desire to create new flow of control sytax.

Yesterday I posted a new way to define functions and their tests contiguously using a new syntax in Scheme. It took about 10 minutes and less than 10 lines of code.

Likewise in Lisp as well as Smalltalk defining new control structures is childs play, mainly because the control structures in those languages are defined using the very same mechanisms that are available to you, the dear programmer.

Yes, I know, I am a loser to even bring this up.

Is COM interop more stable than dotnet's?

I'm recalling something for the the dotnet reality check...

Because dotnet is a more "complete" object model than COM (inheritance and shadowing of public and private aspects) and because different languages (VB.NET and C# to name two) have somewhat different rules for inheritance, it may be that dotnet has more interop problems than COM. I couldn't say because I don't know COM.

A problem with dotnet inheritance was documented some time ago. I don't believe it has been fixed.

Minimum Requirements for One Runtime, Multiple Languages? Less is More

Dare Obasanjo follows up on Jon Udell's investigation into all things dotnet with a piece on language neutrality.

Two points:

• The dotnet framework to date is "language neutral" within a very narrow definition of neutrality.
• Should we care?

What is really needed in language neutrality? A few thoughts:

• Shared nothing, scalable threads. Let our languages run in the same address space, but force them to say at an arm's length from each other.
• Short-circuited message passing. Let our language collaborate the same way whether they are together or apart. Just make it more efficient if they are together.
• I/O and other system services. Let them use a common set of system services, virtualize the limitations of the host OS.

There is no reason why one could not build a Longhorn on this "less is more" model, and the result would have more potential (flexibility, scalability) than the one that appears on the horizon, as far as I can see.

A thread of conversation on the agile-testing yahoo group is speculating on the ability to define a function and its tests "contiguously" to reduce the effort to make the contextual switch between "coding" and "testing".

Here is a simple Scheme macro to approximate the idea...

; The following is a macro that allows you to supply test input and
; test results when you define a function. After the function is
; defined, the tool automatically runs the function on the test input
; and checks to see if it matches the expected result. There are a
; number of enhancements (list of test inputs, define test function,
; etc.) to consider. 

(defmacro define-eg (name-result parameter-input . body)
  (let ((name (car name-result))
 	(eg-result (cadr name-result))
 	(parameters (map (lambda (p) (car p)) parameter-input))
 	(eg-input (map (lambda (p) (cadr p)) parameter-input)))
    `(begin (define (,name ,@parameters)
 	      ,@body)
 	    (equal? ,eg-result (,name ,@eg-input)))))
 
(define-eg (product-of-sums 108) ((a 5) (b 7) (c 9))
  (* (+ a b) c))
 
> (load "define-eg.scm")
> (define-eg (product-of-sums 108) ((a 5) (b 7) (c 9)) (* (+ a b) c))
#t
> (product-of-sums 5 7 9)
108
> (define-eg (bad-product-of-sums 108) ((a 5) (b 7) (c 9)) (+ (* a b) c))
#f
> (bad-product-of-sums 5 7 9)
44

PERT and Agility

Robert Martin (home, blog) in Software Development magazine provides a simply useful introduction to PERT charts, Critical Paths, and a couple of simple tools to replace them for software project management.

Catfish in the Memepool

Catfish in the Memepool... I have been enjoying Brian Foote's writing for almost 15 years, I guess, and now he has a blog (with a feed).

A Method for Design By Contract

At the BCS OOPS Resource page, the slides from Richard Mitchell's December session on a "Method for Design By Contract" (zipped ppt).

This presentation is worth reading on two levels:

1. This is one of the best presentations of Design By Contract, what it is and how to do it.
2. This is just a good example of an expository presentation in general.

BTW the ideas in Design By Contract are useful for defining services in general.

Programmer Tests vs. V&V Tests

...when up to half of the output of a full-blown TDD-style project can be test code, we’re going to want to find ways to automate and streamline the effort.

We have to make a distinction about what kinds of tests we're writing and what kinds of tests we want to automate.

This goes back to an exchange I had with Jarno Virtanen earlier this month. In that exchange I wrote...

The TDD tests should be just enough to get to a satisfactory design. Depending on the system being constructed, you should still consider acceptance tests, including performance tests and more complete suites for functional coverage than were needed just for the design process.

We don't want to "over automate" the TDD tests since they are programmer tests, i.e. a programmer's tool for thinking.

We do want to automate the validation and verification tests because we want to eliminate boredom and error, allowing the V&V team to spend their time on activities like creative exploratory tests.

Idiomatic Lisp and Idiomatic Scheme

I guess I agree with Brian Marick's code reading style more than Richard Gabriel's. Rather than using an optional private parameter in a public function, though, I prefer to use the "named" let, with an accumulator there, for tail recursive functions.

Is it idiomatic? It's fairly common in Scheme. I am not sure why it has not been adopted more widely into Common Lisp.

;; Richard Gabriel's Idiomatic Common Lisp
(defun fact (n)
  (labels ((f (n acc)
	      (if (<= n 1) acc (f (- n 1) (* n acc)))))
    (f n 1)))

;; My (Idiomatic?) Scheme
(define (fact n)
  (let loop ((n n) (acc 1))
       (if (<= n 1)
	   acc
	 (loop (- n 1) (* n acc)))))

;; "Old School" Common Lisp
(defun fact(n &optional (acc 1))
  (if (<= n 1)
      acc
    (fact (- n 1) (* n acc)))

Turn a Handheld into a Desktop

As PDA hardware gains more GHz and MB, attaching large monitors, keyboards, and mice directly to the cradle makes a great deal of sense.

I would think this makes much more sense than a table PC for a large number of users. Well, me.

Bluetooth, anyone?

Composing (Avalon) and Partial Classes (XAML)

Why does Avalon at once promote composition (slides (PPT)) *and* simultaneously base XAML on partial classes, a new language construct for weaving a single class from multiple sources?

Why is composition not good enough? Or if not, why is inheritance not the next best thing? I have yet to see a rationale for introducing another mechanism for organizing object-oriented code.

Answers can go here.

Business Week: "Waking Up From the American Dream"

Paul Krugman, on Business Week's article titled "Waking Up From the American Dream"...

Suppose that you actually liked a caste society, and you were seeking ways to use your control of the government to further entrench the advantages of the haves against the have-nots. What would you do?

"Managed Runtime"

From the things that make me go, "Hmmm", department...
Years from now will programmers wonder what the heck the "managed" in "managed runtime" means?

Inversion of Control

Folks are trying to push back the history of the notion "Inversion of Control".

I can't recall the phrase itself, but the notion goes back at least into the late 1980s when the object-oriented community was talking about "libraries" (you call it) vs. "frameworks" (it calls you), e.g. Designing Reusable Classes, Johnson and Foote, 1988.

I bet the essence of IoC was also expressed by Peter Deutsch in 1983's "Reusability in the Smalltalk-80 Programming System". I can't remember if this paper actually used the term "framework". But that was a main point, and draws on the Smalltalk work they did at PARC in the 1970s, including the first "famous" inversion of control, Model View Controller, with Trygve Reenskaug

Catch 22: Atom is available, but it is all or nothing

Update: This from Blogger in response... There is not a want to publish both RSS/Atom. If you do not want to yet switch to Atom, you can use your RSS feed until you choose to switch to Atom.

So if you want to publish both RSS and Atom from Blogger, let them know.

I can publish Atom if I change a setting in my blogger configuration. But I won't, at least not yet.

I've sent an email to Blogger/Google to explain this. Apparently, (from what I can see in the UI), I have to choose between RSS 1.0 and Atom. I cannot have both, unless I write it myself, and I have no interest in that.

How can the Blogger base switch to Atom this way until all the universe supports it first? How will the universe be encouraged to support Atom unless the Blogger base publilshes it? Catch 22.

Leave one test failing

One thing that Kent Beck mentioned in Test Driven Development was that if he's not finished test-driving a chunk of code, but has to leave it for a while, he'll leave one test failing. When he gets back to pick up where he left off, he runs the tests, sees the failing one, and by making that test pass, gets back into the mindset needed for that design problem.

I do this all the time. This is especially handy when you are programming at home and are susceptible to frequent family interruptions. It's also handy when you are done for the day. When you come back in the morning, the test is there ready to tell you exactly where you left off.

What you need to know about the next piece...

...is contained in the last piece...

The ceramics teacher announced on opening day that he was dividing the class into two groups. All those on the left side of the studio, he said, would be graded solely on the quantity of work they produced, all those on the right solely on its quality. His procedure was simple: on the final day of class he would bring in his bathroom scales and weigh the work of the quantity group: fifty pound of pots rated an A, forty pounds a B, and so on. Those being graded on quality, however, needed to produce only one pot -albeit a perfect one - to get an A.

Well, came grading time and a curious fact emerged: the works of highest quality were all produced by the group being graded for quantity. It seems that while the quantity group was busily churning out piles of work - and learning from their mistakes - the quality group had sat theorizing about perfection, and in the end had little more to show for their efforts than grandiose theories and a pile of dead clay.

Art is human; error is human; ergo, art is error. Inevitably, your work (like, uh, the preceding syllogism) will be flawed.

What you need to know about the next piece is contained in the last piece.

I have nothing to add. Buy the book.

dotnet events and external language bridges

Update: Note that the Python.NET described here is *not* a CLR implementation of Python. This is "regular" C Python. You drop the bridge DLLs into a Python folder, then import CLR, and at that point have access to all the dotnet classes, events, etc. There's been some confusion of this with the more experimental IronPython, which is an all-new CLR implementation of Python and not ready for production use.

The VisualWorks Smalltalk dotnet bridge does not yet handle dotnet events...

But registering for events in .NET was not part of the version 1 plan. It is extremely difficult and there is no solution whatsoever at the moment (AFAIK for any *-.NET bridge).

The problem is that you need to give .NET a delegate object to call (type safe callback) and that has to be part of the managed world. We are thinking about synthesizing such objects (their classes) and having an extra callback mechanism with dedicated marshaling

For the record, the very nice production quality Python.NET does support dotnet events in the manner described above. I have not looked at the code for how, but it is open source.

Here is an example of registering for a dotnet event in CPython...

    def add_menu(self):
        self.Menu = MainMenu()
        self.tool_menu_item = MenuItem()
        ....
        self.ellipse_tool_menu_item = MenuItem()
        self.ellipse_tool_menu_item.Text = "Add &Ellipse"
        self.tool_menu_item.MenuItems.Add(self.ellipse_tool_menu_item)
        self.ellipse_tool_menu_item.Click += EventHandler(self.pick_ellipse)
        ....
    def pick_ellipse(self, sender, args):
        self.ellipse_tool_menu_item.Checked = True
        self.canvas.current_tool(self.canvas.ellipse_tool)
        ....

Writing Run-time Optimizers in High Level Languages

Miguel de Icaza is crazy about this...

One of the fascinating things about Jikes's JIT engine is that it was completely written in Java. Absolutely mind-blowing. The upside is that many things can now be done in a higher-level language than C like JITing and garbage collection, the downside is that these systems are fairly heavy during compilation time and are best suited for server use and not desktop use.

And I am crazy about the following, and wonder how they compare...

This talk summarises two decades of work on Smalltalk and Self compilation and virtual machine technology and describes a novel attempt at an adaptive optimizer for Smalltalk that is written in Smalltalk and to a meaningful extent, portable across implementations... the explicit representation of code and execution state make it convenient to implement a hybrid architecture where an adaptive optimizer written entirely in Smalltalk, guided by type information obtained from the VM, produces optimized bytecoded methods that are executed by a more-or-less conventional VM.

Smalltalk about the enterprise

Roy Osherove is asking about Smalltalk in the enterprise. Gemstone Smalltalk was the first object-oriented application server to my knowledge. The core architecture has been in development for about 20 years. The systems developed include manufacturing floor, billing, banking, insurance, shipping, and logistics applications.

There are a surprising number of new web applications running Gemstone Smalltalk too. I would probably choose Cincom's VisualWorks over Gemstone today since it now has a lot of server capabilities, better web tools and objects, as well as better product support.

AS/400 and Smalltalk

Richard Demers connects the IBM AS/400 (which has an object/capability based OS) and Smalltalk.

The AS/400 is an interesting animal. Actually, seven years ago Dave Thomas (founder of OTI, bought by IBM) gave a keynote talk to Gemstone's company meeting. There he suggested that Gemstone Smalltalk be ported to the AS/400.

I am not sure the suggestion was seriously discussed at the executive level. The company shortly after decided to take on IBM, Sun, and Oracle in the Java/J2EE market. (This was before Weblogic made a name for itself.) And the rest is history... Smalltalk is still their cash cow.

Why Smalltalk?

What makes Smalltalk that much more productive?

Jim hits all the right notes. I would add a couple other notions...

There is no main. Rather than writing programs, you construct objects. Rather than running programs, you test objects. A Smalltalk environment will have several browsers up at a time. Some for browsing, others for editing. Another typical scenario is to have several *workspaces* up at the same time. A little code here, a few tests there.

This ties into my other notion, that Smalltalk is essentially *testable*. Before the onslaught of XP, Smalltalk culture was even then based on writing a little and testing a little.

Plus common practice (trivially achieved in the Smalltalk environment) is to keep this "image" of multiple windows and multiple, incremental tasks flowing between sessions. Smalltalk has trivial persistence that supports the development environment greatly.

Adventures in 64 bits

Michael Lucas-Smith expands his adventure with registers to the 64-bit world.

The 64-bit world is interesting for another reason: data representation. Not only does the IA-64 have a large number of registers, and not only can you "rename" them so that they don't always refer to the same physical register, but since they are 64-bits wide you can fit a lot of data into them as "immediates" in a dynamic language like Smalltalk.

How would you represent a 64-bit floating point number in Smalltalk?

In the 32-bit world, you would have a 32-bit object reference where some number of bits would indicate the "tag" or the data type and some number would indicate the memory location where the data actually resides. The tag may be zero bits if all data is stored in memory and never immediately in the reference itself. Another option is to use a 2-bit tag, say the lowest two bits, to indicate the data type and then align the allocated memory to account for the "wasting" of the lowest two bits. (They indicate tags instead of pointing to byte aligned allocations.)

These two bits are then 00 or masked off to get to the allocated memory. Otherwise some other pattern (11, 10, 01) indicates the data is immediately in the reference itself. This may be a 30-bit integer (again, the tag is wasted), or a character (some other byte or 2-byte integer in the upper 16-bit word) or some other design. (Google should turn up some creative designs, this is off the top of my tired head.)

But for 64 bits the world of encoding gets *very* interesting. Back to the 64-bit floating point number...

In the 64-bit world, just represent a 64-bit FP number as... hey, a 64-bit FP number. Use all the bits!

But then how do you represent everything else if all the bits are used for FP? Well, there are a large number of 64-bit arrangements that are not actually FP numbers, rather they are "Not a Numbers" or NaNs. But not all 64 bits are used to indicate some arrangement is a NaN. Actually a small number of bits are used to indicate NaN (something like 10 or 11).

A very large number of integers can be represented as 64-bit FPs. Only the *really* *really* large need to be memory allocated. A lot of other interesting data can be represented immediately in the 64-bits (and so passed around in registers, stored immediately in vectors and matrices, etc.) These data might be characters (easily up to 32-bits of character data, more bits could represent common fonts, styles, etc.) 32-bit color, the same.

Dynamic languages like Smalltalk become significantly more efficient for number crunching (engineering, finance) because they're no longer using memory allocated numbers for floats and large integers.

So much for science...

Back to politics: the Bush administration wishes to replace this time-tested, widely respected process of evaluating scientific research and proposals for funding of research, with a process that places control in the hands, not of experts in the field in which the research is pursued, but of the White House and the OMB...

The administration says that policy should be based on scientific truths so well-established as to be beyond question. But there is no scientific truth beyond question; that is the very nature of science. This is not about basing policy on better science; this is about interfering with the decision of what constitutes good science... to the financial advantage of various industries that contribute to Bush's campaign, and to the political advantage of the Bush administration as they throw a bone to their religious "conservative" (read: radical fundamentalist) supporters.

The war in... is but a symptom of a far deeper malady within the American spirit

...communism is a judgment against our failure to make democracy real and follow through on the revolutions that we initiated

Ian Bicking has a commemoration of MLK, Jr. and he chose to quote from one of his last speeches.

What a beautiful choice. The mainstream media tends to choose quotes and clips from many years earlier when his focus was still primarily civil rights for African Americans.

King was beginning to shine the Light on society's even greater ills. These are neither as obvious nor as easy to resolve as were civil rights. And so they're still with us and he was assassinated not long after this speech.

"Why are you speaking about the war, Dr. King? Why are you joining the voices of dissent?" "Peace and civil rights don't mix," they say. "Aren't you hurting the cause of your people?" they ask. And when I hear them, though I often understand the source of their concern, I am nevertheless greatly saddened, for such questions mean that the inquirers have not really known me, my commitment, or my calling. Indeed, their questions suggest that they do not know the world in which they live. In the light of such tragic misunderstanding, I deem it of signal importance to try to state clearly, and I trust concisely, why I believe that the path from Dexter Avenue Baptist Church -- the church in Montgomery, Alabama, where I began my pastorate -- leads clearly to this sanctuary tonight.

OO Contracts

This financial contracts specification would make an interesting Smalltalk library. The original claim was that a functional language should be used because combinators, unlike objects, can be easily composed.

But combinators are as easy to build with objects as they are with functions.

Riding Giants

"Dogtown" was such a good documentary, I'm a definite for "Riding Giants" when it arrives in a theater near me.

I was skateboarding in the same era as the Z-Boys. Unfortunately I was in Ohio. Our biggest stunt was to make it down a long hill on a sidewalk that ended in small gravel, or grass if you could steer well enough to get to the grass.

More than a few times we'd recover from rides by plucking out very small rocks deeply embedded in the calluses of the palms of our hands. These were the bonding moments for young adolescents, and so I guess that's why they're the things I remember.

On the Health of "Offshoring"

Google is opening a Bangalore engineering office...

The office will operate in an identical manner to our other engineering groups, with the same scope of work, hiring standards and unique Google culture.

I wonder if they'll offer the same health and insurance benefits.

GarageBand and the Garage Band Method

GarageBand from Apple gets a references from Jon Udell on turning "consumers" into producers.... this might be a good time to plug "The Garage Band Method". Unfortunately the book appears to be unavailable from Amazon as well as Powells.

I'll have to cherish my copy even more. The Garage Band Method is a no nonsense approach to making *interesting* music quickly while learning piano, guitar, or sax. (Moreover, all three are covered in the one book.) The method is not "learn on your own". You're expected to have a teacher, but the book tells you what to look for in a teacher and how to work with one. The other key point is to get with other people who want to play and may also be beginners in their instrument.

Toysight, Eye Toy

The Toysight for iSight for the Mac seems kind of like the PS2 Eye Toy, but I don't know the details for either. My 11 year old got the Eye Toy for Christmas.

It's a lot of fun and even more potential than realized so far.

An Object Lession: XPathDocument2, Is the 2 Noise or Information?

Objects are for communication. A key to good design is to choose good names. This Longhorn class should be renamed now: System.Xml.XPathDocument2 to set an example.

What does the name XPathDocument2 tell me, the reader? Well, it basically tells me there is an XPathDocument (or maybe an XPathDocument1 and perhaps an XPathDocument3).

It also suggest that the 2 class was developed some time *after* the first one. It might tell me not to use the first one. I don't know. The 2 is noise that wants to be information.

A better name would tell me that XPathDocument2 can be disconnected from the source, not tied to a DOM, and used for read/write, and provides change notifications. The numeral 2 cannot tell me that. Neither does it tell me that the original DOM-bound XPathDocument still may be a good choice in some scenarios.

I don't know exactly what the better name is, but it is out there in the conversations that led to its creation, and (unfortunately) in the conversations that will take place among its consumers. Maybe ReadWriteXPathDocument is a good choice, but only real conversations (as opposed to the ones going on in my head right now) will tell.

This post has nothing to do with XML and XPath or Longhorn. It has everything to do with why objects are important and will remain useful for their original purpose: code organization. Good names are a key to good organization.

Well, this post *does* have something to do with Longhorn... the developers at Microsoft are creating now the language many developers will use for probably a decade. Please take the time to provide meaningful names. Two years before the release date is *not* the time to affix numerals to the end of classes for relatives that provide alternate functionality!

Blitz is an open source JavaSpaces implementation

Freshmeat...

Blitz is an open source JavaSpaces implementation designed to ease development and deployment of JavaSpaces technology. It is Jini 2.0 enabled, and uses established VM principles. It also implements smart indexing, tuneable persistence, and active/passive lease cleanup. It is designed with experimentation and expansion in mind.

Python and groupby

For Python's list comprehensions comes groupby...

Guido inspired SQL-like GROUPBY class that also encapsulates the logic in a Unix-like "sort | uniq".

class groupby(dict):
    def __init__(self, seq, key=lambda x:x):
        for value in seq:
            k = key(value)
            self.setdefault(k, []).append(value)
    __iter__ = dict.iteritems

# -------------------------- Examples -----------------------------------

>>> letters = 'abracadabra'
>>> [g for k, g in groupby(letters)]                # grouped
[['a', 'a', 'a', 'a', 'a'], ['r', 'r'], ['b', 'b'], ['c'], ['d']]
>>> [k for k, g in groupby(letters)]                # uniq
['a', 'r', 'b', 'c', 'd']
>>> [(k, len(g)) for k, g in groupby(letters)]      # uniq -c
[('a', 5), ('r', 2), ('b', 2), ('c', 1), ('d', 1)]
>>> [k for k, g in groupby(letters) if len(g) > 1]  # uniq -d
['a', 'r', 'b']

RDBMS: Inherently Loosely Coupled?

Carlos Perez writes...

In general, a relational database provides a fixed, queried, self-describing, lazy evaluated system that is inherently loosely coupled. It's surprising that its perceived to be more tightly coupled than a component based system.

I agree 100 percent with this statement. I think the current popular SQL databases are archaic for a number of reasons, yet they are a mechanism for loose coupling. What's more, they are widely understood by developers. An unfortunate aspect of relational databases is there is typically so much ceremony involved in putting one into production. There is a time and place for ceremony, and likewise for agility. We should be practicing agility with databases, and vendors should be listening for ways to improve them.

AI Never Really Worked At All?

AI never really worked at all, says Tim Bray in his latest installment in an interesting technology winners and losers assessment. I'm enjoying this series so far, but the AI line item struck me funny.

First of all, looking at this list, some of these technologies are very specific, such as Unix/C and SQL/RDBMS. But "AI" is a broad, 40-year-and-continuing, label for several families of hard computational problems. How can these be compared in terms of "hard to build"?

Digging into various AI categories, clearly some have been very successful, such as expert systems, and various forms of search and planning. Other categories of AI research have been "successful" in terms of learning exactly what the problems are in that category. Not all successful research results in working products.

Test Driven Development

Jarno Virtanen writes...

My personal gripe about test-driven development is the complete lack of the so-called white box testing methods. In white box testing, you use the source code in order to create test cases that cover the whole functionality of the testable unit. I would go so far as to claim that the trickiest bugs are in the edge- and corner-cases of the code which you can reveal with white box testing. But, of course, tests in advance of coding are better than no tests at all. (I am too often guilty of writing no tests at all.)

I think it is important to emphasize that test-driven design does not imply that the tests you write toward a design are not necessarily the only tests you should write to get to a production quality release. The TDD tests should be just enough to get to a satisfactory design. Depending on the system being constructed, you should still consider acceptance tests, including performance tests and more complete suites for functional coverage than were needed just for the design process.

Can you learn YAML in five minutes?

Can you learn YAML in five minutes? I did.

Then I read through a lot of the YAML articles, web, and spec this weekend. I was kind of aware of YAML some time ago, but this was my first in-depth look.

YAML appears to be just popular enough to have a future, but of course XML is a juggernaut.

I like that YAML maps to simple data structures found in essentially all modern languages. There is no separate "DOM". YAML also appears to have a good set of base data types with extensibility. On top of all this, the syntax is significantly simpler and more human readable, which also means it can be hand edited more readily.

YAML might make a good syntax for a build system, as opposed to ANT. It would also be a good candidate for "preference" files and other text documents that people read and write by hand, but also process with applications.

There is a simple mapping between YAML and a subset of XML. YAML is a good deal more compact than XML, although I wonder of that would significantly reduce the benefits of HTTP compression on large files. Depends on the markup to content ratio I guess.

Register Allocation

Michael Lucas-Smith calculates the methods that can use register allocation effectively...

I put in italics the methods that would require a spill object. Only 0.37% of methods in the system will spill!. Clearly this indicates that the technique will work even on a system with 8 registers like an x86.

I think that is to some degree optimistic since no registers will be left on the x86 for intermediates.

Data+Algorithms

Keith Ray responds to my recent item on coordination via databases...

As long as databases only store data, and don't store whole objects (data+algorithms), they will always risk disassociating meaning and values.

I understand there is this risk. But the scenario we're comparing to is the distributed messaging paradigm where messages have no algorithms associated and the various end points use different languages. People have decided they don't want the "whole objects" scenario.

I think there is a tremendous benfit to whole objects in either scenario, if good choices are made about the language. The majority has not gone along with that. Ever. Back in the days when CORBA and OODBs were hot topics at OOPSLA, there was a clear message being presented from an aggregated look at the experience reports. Namely, distributed Smalltalk and the Gemstone Smalltalk OODB were relatively easy wins, while distributed CORBA and other OODBs would work OK but were a pain in the ass.

Coordination via Databases

Jay Han responds to my item on using databases for coordination...

We know that databases provide concurrency control and transaction management. These features let applications share data -- you can call this coordination at low level. But what about coordination at high level? How can they exchange semantics of data? e.g. "9999-12-31 in date field means now? never? forever in the future?" or "20 point means 20 basis point." (Schemas and constraints can check syntax of data but not the "validity" of data.) Because I don't see how databases provide meaningful (hence ad-hoc) coordination, I don't quite understand Patrick's last sentence above especially the second half.

Databases have no advantage nor disadvantage in this respect (the "meaning" of data) to any other coordination mechanism.

Edd Dumbill on XML and Databases

Edd Dumbill makes an interesting observation about XML and databases based on submissions to XML Europe 2004...

Databases. Though there's a reasonable amount of interest in the W3C XML Query language, there's not much to say about XML and databases. It doesn't seem to me that the integration of XML with relational databases has taken off in the way we once thought it might. Is this just a temporary lull in the convergence activity? Probably.

BigCo Dynamics

Jon delivers a great sermon on dynamic languages and wraps it up with an observation...

Somebody asked me yesterday why platform vendors like Microsoft and Sun are never at the forefront of dynamic-language innovation. I don't know why that's so, but it does seem to be true.

Sun has been at the forefront in the labs, e.g. the Self language was partially a Sun effort. The Java HotSpot VM is essentially a productization of many dynamic language implementation innovations.

I think an answer to the question above has more to do with the psychology and sociology of programming than it does with anything else. The big companies are almost by definition the ones whose value chain ends with giving the majority what they want. What they want is not always what works good in laboratories, nor is it necessarily what is "best for them".

Looking at language evolution in the long run, especially as plotted against a graph of Moore's Law, clearly the trend is to become more dynamic. A good indicator is a major industry journalist writing about such things.

I don't know what the name will be for the ultimate dynamic language, but it is almost certain to have {curly braces} in the syntax and come from Microsoft. Well, sociology may determine the syntax to have <angle brackets>, but it almost certainly will come from Microsoft. 8^)

We *Have* Magic

Tim Bray on the tripod of objects, messages, and tables...

...for me, this essay brought into focus the fact that anyone who isn’t comfortable with object-oriented design, and with relational data modeling, and with wrangling XML messages; anyone not comfortable with all three, I say, just isn’t fully trained.

Well said, for developing today. But what about tomorrow? When he writes...

Lots of architects have learned, painfully, that you usually can’t magick relational rows away behind object/class abstractions. The right way to think about a database is as a set of normalized tables that are designed to be addressed with SQL strings.

...I can only suggest that we need to upgrade our languages to provide more magic.

Lisp, Smalltalk, and Python come to mind. These are the magic we have for now, but there's more to come if we stay the course. Our databases are still based on early 1980's implementation ideas. Let's keep the good concepts and upgrade the rest. Our messages are struggling with nascent infrastructure and representation issues.

Our languages (i.e. "objects"), our databases, and our messaging are all struggling to become more dynamic. As they do, they will also become more unified, or at least "unify-able".

Will it be Cow Paths or Paved Roads?

Jon Udell... For me every document is a database, and every database is an assembly of documents.

Fair enough, I like the convergence too. Hopefully our tools will provide some examples and guidance toward good structures for documents for various scenarios. There are best practices in the decision support arena that are not always employed even by us professional nerds. New query tools and calculators would do well to steer the evolution toward the paved roads or we may be processing XML on the cow paths for longer than necessary.

Keith Ray writes about multiple return values in Python and Smalltalk...

# Python example...
beforeNotAfter, afterNotBefore = foo.findNonOverlappedElements( before, after )
# do stuff using beforeNotAfter
# do stuff using afterNotBefore

"Smalltalk example..."
foo findNonOverlappedElementsOf: before and: after 
    doing: [ :beforeNotAfter :afterNotBefore |
                "do stuff to beforeNotAfter".
                "do stuff to afterNotBefore" ]

The other aspect of the Smalltalk solution here is "tell, don't ask". We tell 'foo' to find the non-overlapped elements, and we tell foo what to do with the resulting lists of elements (execute our block). This forces a certain cohesion to our code. The Python example is violating the spirit of the Law of Demeter, if not the actual Law (1) because we ask and then work the results of what we asked.

But I don't see how the Law of Demeter is involved in this. Neither the Python nor the Smalltalk examples divulge anything about the implementation of foo. There are no undue dependencies.

In fact this is one use of a Block in Smalltalk that obfuscates the code. Python's simple syntax for returning tuples into a multiple assignment is the simplest mechanism for this scenario.

If only you believed in miracles, we'd get by

Americans increasingly [tell] pollsters that they believe in prayer and miracles, while only 28 percent say they believe in evolution.

Isn't evolution a miracle?

Snow, Snow, and More... Freezing Rain???

New Year's Day we had 5" of snow and great sledding. The only good sledding snow in the last four years down in the valley.

Last night the snow was even better. But I had an all day meeting in the office. Checking in with my family throughout the day, my boys spent all day outside. The sledding was great. For them. 8^(

Before I even left the office, the landscape had been receiving a coat of freezing rain. The fun would be over. But wait...

Up ahead in the driveway there is a snow ramp and kids are falling down hard in the road. The ice coating is almost perfect and the roads are empty.

Several of us make the full trek up the road. Again and again. There was plenty of sledding left this night.

Tomorrow the muscles will ache and I have Part 2 of the two day meeting. 8 am.

More Songs about Objects and Data (and Pattern Matching)

On DevHawk the discussion continues about objects and data. Herewith are some thoughts...

Objects are first and foremost for code organization. Computers could do their work just fine, thank you, without objects. And so if the "data" or "messages" or "documents" are not going to map to objects, then will we be better off?

I think the salient, yet somewhat conflicting, points include these:

• Objects are for code organization
• Commands, messages, and documents can be organized as objects
• Different languages have widely different implementations of "objects"
• Interoperability should not assume anything about a specific object model
• Still, mapping data exchanges into the objects of your language are desirable
• Therefore, attempt to model interoperable data exchanges as objects to the extent it will help you with your code organization

In addition to objects, the other code organization mechanism that will lessen your burden is pattern matching. The better pattern matching is integrated into your language, the less code you will have to write when passing semi-structured data around.

Erlang is a good example of a language that does not have "objects" per se, but Erlanf does have a very convenient pattern patching syntax for interprocess messages. Languages that have both objects and pattern matching are even better off.

Microsoft Business Framework

Update: What happened to IBM's San Francisco framework for Java? Was it too ambitious? Will MBF be more practical? After a half decade of EJB and J2EE, is there anything comparable? In any language? See MicrosoftBusinessFramework.

From the PDC, maybe the best presentation I have view thus far... the Microsoft Business Framework.

Continuations: Python, Haystack

From the Chandler email list, some references to the use of continuations in the Haystack system and the Python language...

In September, Hamish Harvey described some of the neat things about Haystack, including UI continuations. (Harvey's message was posted Fri, 26 Sep 2003; there's an article describing Haystack's continuations at http://haystack.lcs.mit.edu/papers/uist2003-uicont.pdf.)

I just wanted to add a reference to a tutorial article on continuations using Python: "Continuations Made Simple and Illustrated" by Denys Duchier at http://www.ps.uni-sb.de/~duchier/python/continuations.html.

I also want to second Harvey's enthusiasm for the idea. When the design work on CPIA gets into high gear, I strongly recommend reviewing this technique.

Regards,
Don Dwiggins

Databases for coordination

Jim writes...

The plain truth is, not that many projects really need complex distribution mechanisms. Most apps look a lot like this:

• Read data from a store (db, files, etc)
• Modify data
• Dump data back to the store

I think we have gotten too far away from this wisdom. The typical database has distribution and coordination built in. That means for the most part you get to write your application in isolation. The database also to serves to coordinate the application's own actions.

We need to get back to the future and look at how to simplify coordination and not just message passing. Databases are a coordination mechanism for one or more applications, and one mechanism is better than two.