SOAP Opera: There's really not much mystery about this stuff

Update: John Robb's FUDeflection. Or redirection. Too soon to tell. The SOAP opera continues.

As someone with zero investment in RSS or Echo, I have enjoyed the past week or so from afar. Mostly waiting for Dave Winer to speak up, which he did, and did well. Now Jon Udell adds his take, which is not surprisingly insightful, but is surprisingly sarcastic.

He pulls it off. Here's the crux, though, in his more typically pragmatic style...

Anyway, Dave showed everybody how to use RSS. That's his crowning achievement in my book. Not the format...

Now that the dam has broken, Dave has endorsed the new effort. It must have been an incredibly hard thing to do. I have a teenage daughter and when it's time for her to leave the nest, in a couple of years, I hope I'll handle that transition as graciously as Dave is handling this one. Meanwhile, the Echo designers are -- not surprisingly -- converging on a core that looks a lot like RSS. So far they've discovered that a blog entry has a link, an author, a publication date, and one or more semantically-equivalent content items. Any day now, they'll conclude that it also has a description. There's really not much mystery about this stuff.

I did not realize what was coming with Echo when I reacted positively to Steve Gillmor's take on Microsoft and RSS a couple weeks ago. Of course when I was suggesting RSS might be supplanted over time, I did not think it would be for apparently political reasons. I certainly did not think the effort would begin any time soon.

My reaction to Echo is along the lines of many others: RSS (regardless of its many forms and politics, I mean the collective sense of "RSS") has just in the last months taken a spot in the zeitgeist. My god, it takes ages for these concepts to make their way into large corporations. And now we have to explain Echo? I'm not going to rush that one.

"We need to be more like the auto industry." (SAP)

Henning Kagermann, CEO of SAP, says "We need to be more like the auto industry..."

Prague: Judith Owen (piano, vocals) and Harry Shearer (Le Show, Spinal Tap, A Mighty Wind)

Harry Shearer...

Just in case you know anybody in Prague:

"Teddy Bears' Picnic" plays this Sunday, Monday and Tuesday at KINO OKO

Frantiska Krizka 15
170 00 Prague 7
Czech Republic

Judith Owen plays, with Harry Shearer on bass, Monday night at Zelezna Jazz Club

Zelezna 16
Praha 1
Czech Rep

Chris Sell's Applied XML Conference

Speaking of speaking at conferences, I will be doing that again at Chris Sell's Applied XML Conference in a couple of weeks. This time I'm using Python, which I consider a kissing cousin to the Lisp family along with Smalltalk, Ruby, and some others. Look at the code a couple messages below. You could wrap some parentheses around those statements and never blink.

Corrupting Influence

Gordon Weakliem swears I turned him on to Lisp...

What led you to try Lisp?

I heard Patrick Logan speak at a conference about building web services with Scheme, which sounded interesting, so I read "The Little Schemer". Later, Paul Graham's writings, esp. "Beating the Averages", but also because I was feeling the limitations of the Java / C# family of languages.

Wow, Gordon. If only it were that easy with everyone else!

Jython and JXPath

I recently started using JXPath, an XPath implementation for multiple underlying data models in Java. It works out of the box for DOM, JDOM, java.util.Map, servlet contexts, and Java Beans. Probably others.

I wanted to use XPath to query Jython objects. The really cool thing is the amount of code it takes to do so. It's so short I include it right here...

from java.lang import String
from org.apache.commons.jxpath import JXPathIntrospector
from org.apache.commons.jxpath import DynamicPropertyHandler
import jarray

class JythonPropertyHandler (DynamicPropertyHandler):
    def getPropertyNames (self, py_object):
        keys = py_object.__dict__.keys()
        return jarray.array(keys, String)
    def getProperty(self, py_object, property):
        if py_object.__dict__.has_key(property):
            return py_object.__dict__[property]
        else:
            return None
    def setProperty(self, py_object, property, value):
        py_object.__dict__[property] = value

def registerJythonPropertyHandler():
    JXPathIntrospector.registerDynamicClass(PyInstance, JythonPropertyHandler)

And so...

>>> context = JXPathContext.newContext(some_object)
>>> context.getValue("frob[abc > 1]")
<__main__.Foo instance at 3278971>

Virtual Databases and Enterprise Integration

Enterprise IT environments are a mess primarily because they've grown piecemeal by function and technology. Every business unit and indeed every project is "empowered" to make their own decisions. And so they rightly make the best decision for their purposes. Every vendor has some hook in their technology to bring you back to the trough.

Even if some Enterprise Resource Planner (ERP) met all your needs, how long before you could actually get to "All Your Base Are Belong To Us" nirvana? And so we have "Enterprise Application Integration" (EAI) and (new to me...) "Enterprise Information Integration" (EII). One approach to EII in the past has been to build a massive Enterprise Data Warehouse (EDW).

Another approach, Phil Windley writes in his blog item on virtual databases, "The biggest hurdle... is creating the... data model." and includes some useful references.

Another pragmatic source of inspiration is Ralph Kimball. First some history...

A significant contributor to the Xerox Star in the 1970s (which influenced the Mac, which influenced...), Kimball and peers at Metaphor in the 1980s pioneered data integration, iconic capsule programming, and end-user data analysis. Metaphor was sold to IBM and languished under the shadow of OS/2 politics.

In the 1990s Kimball went on to pioneer data warehousing design principles and the Red Brick OLAP database (also sold to IBM via Informix). He is now an independent consultant and popular author on data integration topics for OLAP.

His keys to an effective virtual database (or data bus architecture in his words) is conformed dimensions, smallest grain facts, etc. Most of these principles apply to the largest data warehouses or the smallest databases, and would benefit any Enterprise Information Integration effort.

Microsoft and RSS?

Steve Gillmor on Microsoft and RSS.

Interesting ideas. But what is interesting about RSS? What is interesting about any protocol?

Maybe it's just that developers are using RSS and Microsoft likes developers. The protocol itself is nothing special, but that in itself is its advantage... witness the number of applications that implement it. It was designed by developers.

So simple protocols are good. Has RSS been through the scalability ringer yet? Does it matter? Another protocol will come along soon if necessary. The interesting questions are what to do with the protocol. That's what Steve Gillmor is getting at, perhaps.

Driving in Boston

Driving in Boston... Dave Winer has excellent advice...

That way a wrong turn, while it will still take you miles out of your way, won't waste hours of your time.

My first night in Boston (a Friday) my buddy and I spent driving all over town. Not deliberately. We were trying to find something, a specific restaurant I think, near Fenway, because we also wanted to see Fenway Park.

The next morning I looked at a map to see where we had been. Ommagod.

I learned pretty quickly to use the T.

I learned the next best tip when I met the woman who is now my wife. In particular this: get a '62 Ford Falcon and practice getting around the Fresh Pond rotary as quickly as you can.

She named hers "Buzzy".

Lots of discussion here and there and seemingly everywhere about Oracle and Peoplesoft. Of course this is just the tip of the iceberg in business software consolidations.

...they basically force users to comply to certain process and end-user has very little control to 'customize' the application.

Yes, these systems have evolved over decades. In SAP's case, R3 is a descendant of their mainframe product from the 1970s by way of their client/server R2 in the 1980s. R1 was incredibly innovative back then, and it helped SAP get to where it is today, because they had one of the first production-quality machine independent virtual machines.

Too bad the language was ABAP, aka "German COBOL". But it was the 1970s after all.

In any case the future will be interesting since the value in these systems to the vendor is their proprietariness. The value to the consumer is their automation first and their openness second.

They're all trying to open up and still maintain enough proprietary handcuffs to turn a profit. Their historic margins will fall. They're all going after the SMBs (Small and Medium size Businesses).

All the ERPs but Microsoft will probably end up with increasingly open source components and try to make their profit on services. Give IBM the odds here. But IBM will have to make some strategic purchases first to get to a higher business-level software foundation. (Anyone remember "San Francisco"? Before and After)

What's Next Discussion

I posted a high level outline of a What's Next discussion. It's a Wiki page so anyone can mold to their liking. If you're interested, add and modify, but you may want to make the pages more like a "discussion in progress" before refactoring into a "finished document".

See also the What's Next home page.

To do: read the page from Bob.

The Blog of War

From IEEE Spectrum, "The Blog of War"...

It's not exactly clear where blogs fit into the established media universe. The blogverse may be shaping up to be a kind of shadow media.

Trouble Ahead

Trouble ahead
Trouble behind
And you know that notion
Just crossed my mind

The US Patent Office has so many trains running in opposite directions on the same tracks.

Switchman sleepin
Train hundred and two
Is on the wrong track and
Headed for you

Boston Accents

Funny discussion about accents.

Here's my two cents from spending four years there 20 years ago...

I'd say you have acclimated to the language when you find your pronunciation of "Worcester" naturally rolls off the tongue as "Wuster" and "Peabody" has no logical pronounciation other than "Pea'bdy".

To be a true master you also have to be able to say with a straight face...

"Put that wicked drooring of the tree in my top draw, wudja?"

Tiny C Compiler:

TCC is a small, fast, optimizing C compiler, linker, and loader. And the cool thing for dynamic language implementations...

With libtcc, you can use TCC as a backend for dynamic code generation.

This is very interesting. About 8 times faster than GCC. Why spend time on a virtual machine instruction set?

TCC can also automatically generate memory and bound checks while allowing all C pointers operations.

What's Next: Wiki Page

A Wiki page now exists for people interested in What's Next, especially but not exclusively if you can attend OSCON in July.

What's Next: Some ideas

Quick thoughts before the weekend about what's next. After a web services foundation is in place, what do you set out to do with it? What metaphors guide your vision?

• The structured and the semi-structured are merged.
• Everyone is able to fully participate.
• How one participates gradually evolves and influences others.
• Trust is associated with information and services.
• Trust evolves.

One metaphor is formulating: Conversations. Maybe in one sense something like Conversation for Action.

Another question, what current systems and applications most closely embody these thoughts, or your own thoughts about what's next?

What's Next: Call for Discussion

Sam Ruby responds to my interest in WS-xxx...

[W]hat I most would like to explore is what is next. After Web Services.

I occasionally hear talk of an Internet Operating System. And as a group we obsess on what is the optimal design of an individual neuron.

Something nagging in me tells me that these are the wrong metaphors and that we are [on] the wrong meta-level. There is no brain operating system. The emergent behavior of a brain can't completely be understood by looking at how individual neurons operate.

The future is multi-cellular. The cell walls aren't necessarily distance, but represent trust boundaries. It will never be the case that all data will be directly addressible. We wouldn't want it to be.

I like this. For anyone inspired by this description, I'd like to organanize some conversations around this. There are all kinds of dots that can be connected.

Let me know if you'll be in Portland the second week in July (OSCON, Applied XML DevCon) and I can set up a "What's Next?" discussion if there's interest. Meanwhile I'll be thinking out loud here and I'll be looking for other responses to this query.

SOAP, WS-xxx, and CORBA (but not sockets)

From Sam Ruby's comments pages, on SOAP, WS-xxx, and CORBA...

Sam writes: What I like about the WS stack is that if all you need is SOAP document literal, all you need to implement is extremely small.

Reply: Yes, a great boon for adoption.

Dave Winer writes: SOAP was created by people who develop applications, so it didn't suffer from a lack of applications to pull it through.

Reply: Yes, a great boon for adoption! 8^) You folks were not about to create a CORBA first and then build apps later.

And this is a problem, as far as I can see, with the WS-xxx stuff taking place today, which is more like CORBA than it is like XML-RPC. Shouldn't the WS-xxx folks be working with OMG on that stuff? Why isn't "Enterprise Web Services" simply the next CORBA 4.0 release with SOAP optionally (but efficiently) slipped into the stack? There are high performance, open source implementations of much of CORBA 3.0.

I can think of at least one reason why not, but it's not a good reason.

View-centric reasoning for linda and tuple space computation

From The Institution of Electrical Engineers (IEE) Proceedings Software, "View-centric reasoning for linda and tuple space computation" (PDF)...

Abstract: In contrast to sequential computation, concurrent computation gives rise to parallel events. Efforts to translate the history of concurrent computations into sequential event traces result in the potential uncertainty of the observed order of these events. Loosely coupled dis- tributed systems complicate this uncertainty even further by introducing the element of multiple imperfect observers of these parallel events. Properties of such systems are difficult to reason about, and in some cases, attempts to prove safety or liveness lead to ambiguities. We present a survey of challenges of reasoning about properties of concurrent systems. We then propose a new approach, view-centric reasoning, that avoids the problem of translating concurrency into a sequential representation. Finally. we demonstrate the usefulness of view-centric reasoning as a framework for disambiguating the meaning of tuple space predicate operations, versions of which exist commercially in IBM's T Spaces and Sun's JavaSpaces.

Simpler and Cheaper: Lessons from Ethernet

From an Infoworld interview with Bob Metcalfe...

Ethernet didn’t seem to have the problems that bedeviled IBM's Token Ring and ArcNet. Did this play a role in Ethernet's success?

Yes. Ethernet was simpler than its alternatives. It didn’t, for example, rely on the circulation and maintenance of the token. And Token Ring mathematically and, I think, sometimes unrealistically had advantages over Ethernet as long as the token was humming along. But if that token ever got lost or duplicated to create a storm, then things went to hell in a hand basket pretty quickly. The simplicity also allowed the implementations of Ethernet to be simpler and cheaper and come to market sooner. I mean, Ethernet beat Token Ring to market by five years. So that’s quite a lead. That’s probably what saved Ethernet from the IBM juggernaut.

Rules

Jon Udell highlights a technology lost in the late 1980's "A.I. Winter", but which kind of hung around and is now coming back into its own. I'm a veteran of the A.I. Winter and was disappointed when very practical tools like KnowledgeCraft and GoldWorks were lost in the hype of those days. Looks like GoldWorks is still available.

Some more resources to add to Jon's list...

• CLIPS - the grandparent of almost all current rules engines, public domain, written in C with good integration possibilities to other languages.
• Business Rules Applied - a very good book that covers a wide variety of issues around the management of business rules.

Facility

Phil Windley draws some good points out of the interview with Grady Booch. But, and there's always a 'but' when I read something, my philosophy is there's no point in not being a critical reader. If I end up in the worse case being wrong or mistaken, I've still learned something.

Grady Booch writes just after Phil's last quote... One of the advantages of platforms such as .NET is that they codify a lot of those hard things so that global decisions can be made intelligently, then carried out locally by the individual developers.

I think this is untrue. I don't know if Booch is trying to flatter Microsoft, or the readership of the dotNET magazine, or if he really believes this.

I wish the journalist would have pushed for more detail. Why does Grady believe this to be so? Wouldn't that answer help the readers of the interview with how they use dotNET?

For one thing, I don't believe this is true for any current toolset. I'm not singling out dotNET. There is too much complexity in software development. Yes, we're building more complex systems than in the past, but that's no excuse... we should be taking unnecessary complexity out and putting more facility in. Facile --- most software is not.

Glossy Interviews

From an interview with Grady Booch in dotNET magazine...

Q: What is the future of Web services and GXA in conjunction with traditional development?

Booch: I'm pretty jazzed with the direction of Web services. We look at organizations that are already effective at building component-based systems. Meaning they know how to componentize their stuff and deploy them that way. For them the move to Web services is actually a small step. Plus it's not a difficult step, because it represents a different packaging for the things they're already exposing. But there's value in it because now it allows them to transcend a particular platform.

I'm working with a group I think I can mention publicly: MedBiquitous, a project at Johns Hopkins led by Dr. Peter Green. The essence of the project is to try to tie together the various medical research agencies and data across a variety of different specializations in medicine, so that you break down the barriers of specialization. That's a gross simplification of what they're doing but it's pretty cool stuff. The essence of their architecture is based upon Web services, which enables them to make key architectural decisions that transcend particular platforms and technologies.

How helpful is this for you? As a reader and developer, should you expect more from an interview with an industry leader?

This seems to me to be filler in between the ads. I'm disappointed.

Tons of ideas ripe for the picking

Ray Ozzie writes...

I strongly recommend that people join the ACM and use its Digital Library and read some of the things that influenced all of us in the CSCW discipline, e.g. Turoff's early work in conferencing, Flores' work on speech acts/workflow, etc. It's never been more relevant. Now that people have realized that we need to move beyond the Web, there are tons of ideas ripe for the picking in terms of innovative architectures for interpersonal communications & cooperative work e.g. awareness & event notification architectures, real-time & replicated architectures, etc. Explore SIGGROUP papers; head down to Group '03 this fall, and blog what you hear.

This probably is obvious: I couldn't have said it better myself.

I've been a reviewer for several major conferences over the years. The striking reaction I get over and over is how few connections people make to the work that has taken place in our community. Software is forever changing on the surface, but when you get more than skin deep, there are so many common themes, sources of inspiration, and even words of caution.

Meanwhile, what to do?

Jon Udell writes...

In an earlier posting, I characterized one axis of the debate as robustness versus agility, and wondered how we can have our cake and eat it too. Sam Ruby pointed to WS-Transaction, to which Patrick Logan replied "I think we need a rethinking of databases, messages, and coordination." I'm sure that's true. Meanwhile, what to do?

Exactly the right question to ask. Here is the kernel of my response.

1. Design collaborative business processes with partners (internal or external) for exchanging (secured) XML documents independent of connection technology.
2. Emphasize exception processing and failure recovery in those designs. Again independent of connection technology.
3. Apply the simplest technology that could possibly work for connecting and exchanging those documents. Keep the documents independent of the connection technology.
4. Apply the simples technology, or whatever you have available, for processing those documents internally. Keep this technology independent of the connection technology and keep the documents independent of this internal technology. (i.e. your partner should not know or care that you are using MySQL, J2EE, dotNET, Python, etc.)

Something further. Consider passing documents using the following grammar in an XML namespace as a header to the exchanged document.

• "put document D in shared space S with time to live T"
• "take one document matching pattern P from shared space S or return special document N if no match"
• "peek at shared space S for all documents matching pattern P and return copies of those documents or special document N if no match"
• "put copy of document D in space S1 (local) when D is added to space S2 and D matches pattern P"

Metaphor Computer Systems

Peter Nolan writes about Metaphor Computer Systems from the listserv at datawarehousing.com. If you have information about this company or its products you can add that to a wiki page at MetaphorComputingSystems. Endorsements for simple systems like this should be an inspiration to developers. I'm not sure if Meta5 is all or a subset of the original product.

If I said M3 it was a typo...Metaphor Computer Systems (M4) was founded in the early 80s (might have been 1982) and released a product called Data Interpretation System (DIS) in 1984 if my memory serves correctly. Ralph Kimball was one of the co-founders.

I'm not sure there are many places where 'the history of Metaphor' is documented. I wish there was, it was quite a company. In 1997 I got the opportunity to personally say thanks to Ralph for his part in M4 because it changed the direction I was heading in.

In 1991 I had one of those 'ah-ha' moments when I saw DIS demonstrated to a customer of mine. As I watched it took about 5 minutes to realise that what I was seeing was the way end user computing was 'meant to be'. I was completely 'sold' on DIS as 'the way' data analysis/analytical applications would be built in the future. So was the customer. They bought the product.

In another one of those industry 'if-onlys'. If only IBM knew what they had it could have been the office desktop we all use today. (IBM bought Metaphor in 1988. So in 1988 IBM owned an 'office desktop' far superior to MS Office which would not exist for a few years yet. Instead of pushing DIS they pushed Office Vision, which died.)

The list of features in DIS was endless. The folks at M4 broke so much new ground. They had some 400+ customers by 1993 and their customers were household names.

I wish there were some demos or screen shots still around to explain to people what it could do.

The most important thing about DIS was that the IT staff were no longer required to develop analytical applications. The biggest benefit of this was that the business users did not need to 'externalise' or 'communicate' their requirements to anyone. Working on the DIS desktop they could try out their ideas and if they turned out to be good they could be 'packaged' into an analytical app. Yes, I am talking 1986 here. And, as per my previous comment, the 'intellectual capital' of the analyst to who created the application was captured with the application to a very significant extent because any other business analyst could read it. They barely needed any training to be able to read even a quite complex application.

One time an actuary customer of mine borrowed the manuals and from scratch, with no training and no help from anyone else, wrote a 30 year death experience analysis application in 3 weeks. It was that easy and that good to use DIS.

DIS was 'so cool' you didn't even need a keyboard. We used to put out keyboards on top of the screens and they would stay there for weeks on end. Try writing an analytical app today without a keyboard!!!!

Unfortunately, at the time IBM bought Metaphor the future of the world was PS/2, OS/2, MicroChannel, 8514A graphics adapters and Token Ring (at least according to IBM). When moving the software from proprietary to IBM hardware, IBM specified ONLY IBM hardware. There were a lot of other reasons why Metaphor experienced trouble operating inside IBM. (They were not the only ones, who remembers 'Rolm' telephones today?)

DIS was all 32 bit and it was the first fully 32 bit app that was released on OS/2, but OS/2 was 'doomed' and by the time windows 95 came out the opportunity to get DIS out into the marketplace had pretty much gone away and MS Office had a stranglehold on the windows 3.1 desktop.

Eventually the product was stabilised and then finally withdrawn from marketing. I think that was around 1997/8.

But there are many of us out there who look at what we could do then and what we can do today and wonder how it can be that there is so much that is still so hard to do now that was so easy then. Call me nostaligic!!! ;-)

EJB and CORBA

From Jon Udell's item on J2EE/EJB...

You need your business logic available to multiple applications. What if those other applications aren't Java-based?

EJB is designed for CORBA clients.

The problems include, though, firewalls and interop. Unfortunately just as CORBA was rounding third and heading toward home on those issues as well as the price/performance issue, SOAP became a monkeywrench. Too bad really that CORBA could not have been the foundation of an "Enterprise SOAP" that evolved toward some of the flavor of XML messaging.

Also, CORBA 3.0 incorporates its own "Component Model", an improved EJB model that is language neutral. Maybe too little, too late.

Web Services Architecture Usage Scenarios

I just stumbled upon these WS usage scenarios. I've not seen them before, and have read through only a couple of the discussion messages. (Someone probably pointed me to this list some time ago. I "stumble upon" things this way now and again.)

Apropos to my interests, the latest message in the discussion is "REST, uniformity and semantics". Actually REST shows up about every 10-20 messages. When I am awake tomorrow I'll wonder if the conversation ever goes anywhere, and what else is discussed.

More Inflection Points: RAM-based Databases

From the thoughts of Monty Widenius (MySQL)... What I am looking forward to is diskless computers, a little similar to PDAs today, but with much better capacity. Hard disks will be used more as very large backup devices for things that you don't need regularly.

Gemstone's largest customers usually had their entire OODB running on very large systems with the entire database in RAM via copy-on-write shared pages.

Battery-backed RAM would be an incredible boost to performance and simplification to implementing ACID transactions. The other big win for large memory databases is to distinguish read-only data, where you could get rid of most semaphores as well. That's not done enough today.

For habit's sake we insist on running OLAP from OLTP systems designed 20-30 years ago. Newer designs can fit far more RO data in RAM too.

(This is a good time to be a computer scientist. Lots of change over the next five years.)

Wiki and User Identity

Dave Winer believes...

[U]nless I'm mistaken, [Wikis] don't have a concept of user identity...

Some do. Some don't. Sometimes it's an admin option.

Scaling Tuple Spaces

One of the masters of simplicity in software, Graham Glass, writes...

another awesome bit of software technology is Linda, the tuple spaces architecture by David Gelernter. like Lisp, it provides a simple set of primitives with great expressive power. but i don't think that the Linda approach scales well. the remove() primitive requires some degree of locking that is not suited for distributed environments.

Voyager itself was a simple and awesome software technology. Congratulations to Graham for that as well as Glue and the upcoming Gaia. I'd be interested in his thoughts on Linda-like systems for the Internet.

The most significant adjustment in scaling Linda to the Internet that comes to my mind is to redefine blocking operations to be non-blocking. So 'in' (i.e. remove) would not block until a match is found. Maybe there would be a timeout, but the window would be fairly short and so may just be meaningless at Internet scale.

For these non-blocking polling operations, some rationale has to be assigned to an ethernet-like (but larger distance and time scale) backoff policy. Not unlike the RSS situation with polling. But the surprise with Ethernet is that the pipes continue to get faster and conflicts are not the problem people envisioned.

The alternative is to use a cloud-like mechanism for scalable trigger/notification. Eric Hanson has been designing these for relational databases at UFL.

Another aspect of this is the number of simultaneous users of a single tuple space. Scaling out a tuple space is fairly simple... create more spaces. Work has been done to replicate spaces, but in general I'd look for opportunities to simply partition scenarios and participants across distinct spaces. That must be the 80 percent or greater situation at least.

Reliability of messaging would have to take into account that the connection itself does not (and probably should not) implement guaranteed delivery of just one copy of the message. This is where some wisdom has to go into message design and higher-level domain-specific protocols. Less of an issue as the Internet evolves into its next incarnation, but this is the simplest approach for the next N years.

You can add your thoughts to the TupleSpaceScalability wiki page.

R-E-S-P-E-C-T

Sam Ruby writes, referring to the original Smalltalk refactoring browser...

[The] key question in my mind is not when such things converge in the laboratory, but when will such things converge in the mainstream?

I know it's a relatively small market, but Lisp and Smalltalk have been out of the lab for years. IBM's direction before Sun's introduction of Java was clearly aimed at Smalltalk and they were having a good deal of success with it.

There is no doubt that the Smalltalk refactoring browser was first. (Reading Martin Fowler's book will back the claim.)

But when the RB came out, it was aimed at, used, and ultimately made the de facto choice for commercial Smalltalk development.

One Function to a Function

A great site for Lisp programmers and other sorts. Not just Lisp-specific wisdom, from Chris Riesbeck's programming notes, via Kevin Rosenberg's blog, via Lemonodor.

Smalltalk, Lisp, and syntax extension

From James' Cincom Smalltalk blog...

While Smalltalk doesn't have the macro system Lisp has, generics are a non-issue in ST, as is boxing/unboxing. Ditto the "simplification" things. One could summarize this whole thing as adding complexity to a language to make up for the glaring flaws.

While Smalltalk doesn't have syntax macros like Lisp, Smalltalk *does* have simpler block syntax and keyword arguments.

The effect is that new keyword messages can take block closures, which essentially delay evaluation until the method behind the message decides to evaluate them, and the message send looks an awful lot like new syntax, very clean.

Many uses of macros in Lisp, frankly, are simply created to hide the appearance of the lambda keyword. Of course there are more "legitimate" uses of macros, but these are less common.

So Smalltalk and Lisp are fairly evenly weighted in the syntax category in my book.

Ruby is pretty close, too, but has some wierd quirks where they didn't get closures quite right and very bad silent mistakes can be made. Python is further back in this category.

Straightjackets

Phil Windley on dynamic typing...

A strongly typed language merely creates an environment so that the type-checker can automatically check the type constraints for you at the cost of restricting some genuinely useful things you might want to do.

One thing you want to do that is easier with dynamically type checking is testing. Only time spent developing significant systems in dynamic languages will bear this out. Illustrations don't capture the reality.

People see that static types can catch things early. They don't see that those things aren't important since they get caught early in dynamic tests anyway. Yes, you have to write good tests. Can anyone tell me when that's not the case?

Neither is it apparent on paper that refactoring large systems, or just experimenting with large systems, goes much more quickly when you have to change less code to get a small test to run. You trust that the complete set of tests will catch everything else when the experiment is ready to be integrated back into a complete working system.

But during spike experiments you want to ignore the irrelevent, create mock objects, even entire "mock databases", and play what if on isolated bits of code, even within large currently running systems. Doesn't happen easily in Java-like languages. Compilers that have no clue of what you really care about in the moment. They make you dot that 'i' over there and cross this 't' over here before you can even get to the bits of interest.

Learn from this, like this

However, I think that it makes the student's choice crystal clear: become a computer scientist, not just a programmer, because if the IT locomotive does decide next year that some new thing is better, you'd better be as adaptable or you'll get left in the dust.

You want to learn computer science, or anything really, from people with advice like this. You may end up even contributing to the next new thing.

So what?

Stefan Tilkov writes about me...

Patrick Logan has a blog. So what?

Exactly.

Oh, he continues...

Apart from some very interesting posts, I somehow recall the name, and have positive assocations with it ... digging ... I found a patricklogan@gemstone.com

Well, thanks. Gemstone went through an upheaval at the tail end of the dot com fiasco. I left before the implosion...

"In less than a year, a 20-year-old [sic] company, with revenues and profits, (though not brilliant, it was a going concern), has been destroyed by another company that is itself only a few years old. That is sad."

Industry Standards and Emergence

So I am wondering how much should be put into specifications and what should be left for emergence?

I have had bad experiences participating in the market-driven rush to unproven and premature standardization. So I lean toward the minimal necessary and look for patterns and principles to emerge from experience.

Systems from the future

Giulio Piancastelli is very kind and also on the money speculating...

I quoted Patrick's words because I think they are the most original and interesting from the whole bunch. I see Prolog and tuple centers in them, and I am glad for what I see.

Nice connection and thanks for the references to these projects. Indeed I think the future is bright for logic/constraint languages and coordination through blackboard/tuplespace architectures.

Speculating: the Network is not the Computer

There is nothing like a controversial statement to draw attention to an issue, or at least traffic to a little-known blog.

Apparently, Patrick took a peek at WS-Transaction, didn't get any further than the author list and decided it wasn't any good. I agree that two phase commit is traditional... I hold out more hope for compensating transactions via asynchronous messages.

I did read further than the list of authors. But nothing like a controversial statement to get people interested in a topic.

I do think the compensating approach is better than the atomic approach at the distributed business level of "transaction". But I am not sure either should be a part of the connection architecture. Compensation "meta-data" if you will should be a part of the documents exchanged among business partners, no matter how those documents are exchanged.

A distributed system across trust boundaries is where I believe we are collectively heading. The internet shows us the beginnings of what is possible... for the subset of things that are generally always online and publically accessible. Expanding this to mobile and access controlled is the next big thing.

NeuroTransmitters

I love the NeuroTransmitters story. But Sean McGrath illustrates the problems that emerge when you overload your connection architecture over time.

The network is not the computer. My computer is over here and yours is over there. All we need is a simple way to exchange documents.

The documents are the messages. The computers are the cells. (Fortresses in Roger Session's metaphor.)

The connections should be as simple as possible and get out of the way.

Sam Ruby points to WS-Transaction. But look at the list of authors. The problem is IBM and Microsoft have a lot to preserve by maintaining more or less traditional databases and transaction protocols.

Jon Udell wonders about the LAMP vs. J2EE/dotNET inflection point. But LAMP is made in the same image. This architecture goes back thirty years or more.

I've written an essay on what I think the next database architectures should look like. If you read through my blogs you'll know it has something to do with tuple spaces and star schemas. But Jon goes on to ask, "How do you preserve the fluidity that the agile enterprise requires?" There is a little bit more to say, since enterprises are not nearly as agile as they need to be.

WS-Transaction is intended to preserve the status quo rather than improve agility. I'll publish more on this in the near future.

Dynamic Model Checking

Lambda the Ultimate has this piece on a Scheme program that analyzes Excel spreadsheets.

What is most notable about this? The essence of this system represents the future of software development and the need for more dynamic analysis than "compilers" traditionally give.

Note:

• Scheme is a dynamic language.
• The program analyzes spreadsheets.
• Spreadsheets are another dynamic language.

This represents the convergence of dynamic model checking and infered static type checking. The future is getting more dynamic and so "type checkers" will become more sophisticated (and background) theorem provers than they are now.

Additional features will include symbolic execution including concurrency model checking and semi-symbolic partial evaluation.

Over the last 10-15 years electronics design has taken more of a software design flavor. Over the next 10-15 years software design will take on more of a traditional hardware simulation and model checking flavor.

A quote from FDR via Mark Johnson's weblog...

For out of this modern civilization economic royalists carved new dynasties. New kingdoms were built upon concentration of control over material things.

From the Vistorian Age to the present it could be argued that dynasties were necessary to build the steel mills, shipping, the railroads, the telecommunications systems, and so on. Perhaps the benefits for society outweigh the cost of establishing the industrial dynasties.

But now we're in an age where extreme decentralization is at hand. It will be more of a challenge to get dynastical forces out of the way of progress than it will be to actually establish a decentralized age itself. Are dynasties necessary for societies to decentralize in the way they were necessary to centralize in the first place? I don't think so.

Dylan told the senators and congressmen not to stand in the doorways or block up the halls. The times are changing. Will the dynasties allow it? Will the people know enough to demand it?

This from Matt Gerrans ties in with the posts on structure and interpretation of computer programs I wrote about this week (and almost every week it seems).

OK. Another blogger to follow.

BTW when I was writing electronic design automation software way back when Matt's approach was the popular (and correct) technique. Scheme was the scripting language of choice for DA tools.

More Notable Bloggers - Distributed Systems

Jon Udell points out Bob Martin, Ward, and Guido van Rossum have entered the blog world. I'll pay particular attention to Ward's, he has a singular style and an uncommonly sensible approach to software.

Moreover two notable people in distributed systems have blogs as well: Ken Arnold and Jim Waldo.

Both have been instrumental with JINI and JavaSpaces, the two best satellites in the Java universe.

The blogsphere has achieved another level of benefit for software developers with the addition of this new handful. I'll have to check out the others in their sidebars.

Dynamite!

Objective-C has advantages of simplicity and optional dynamic types, neither of which is an attribute of C++. A lot of people don't realize that Objective-C is part of the GCC. So the compiler and library are available for Linux, Windows, and other Unix in addition to Mac OS X.

The language also has a reflection interface in pure C, so integrating scripting languages into Objective-C applications is a snap compared to C++ or even C. You don't have to generate stubs per API. Just call the reflection API dynamically.

Bob Martin's Weblog and Dynamic Languages

First, I am thrilled to have come across Bob Martin's weblog. His first book is still required reading as far as I am concerned, it is not a C++ book, it is a *design* book. His second is right up there as well, having won a 2002 Jolt award. He's co-authored, edited, etc. many others too.

Second, this weblog topic asks, are dynamic languages replacing static languages? He goes on to confirm what dynamic language programmers have been confirming to themselves for forty years: dynamic languages and their iterative, incremental approach to design obviates the need for static type checking.

The next interesting question addresses the future of type systems and will a more formal yet expressive static type system be more productive than either dynamic languages or the more primitive type systems of Java and C#?

My take on it is this having done a lot of dynamic language programming and a little modern functional language programming...

There is a convergence out there, maybe ten years out on the road to The Hundred Year Language, where better (less intrusive, more expressive) formal type systems and dynamic programming meet. After all, incrementally building a dynamic system of objects is not unlike incrementally building an expressive system of typed combinators!

Essentially type systems are theorem provers. The more expressive they become, the close they are to the problem domain, just as the more expressive dynamic languages allow us to speak more about the problem domain than the "compiling domain". In the future I expect to be able to dynamically build semi-formal systems and then have that system tell me things, i.e. prove theorems about itself. From there I can make adjustments not just through more tests and code, but through direct manipulation of the "theorems" I have stumbled upon.

Talk about refactoring! Organizations with interacting systems will be able to refactor the "theorems" of how they do business. It's one thing to share tests and dynamically update them, but it's another thing to share theorems and dynamically update those.

Post-modern Programming 100 Years from Now?

Phil Windley now has a great quote from Paul Graham's "The Hundred Year Language"... (The permalink to the message is broken for April 30 at this point.)

[Paul] Object-oriented programming offers a sustainable way to write spaghetti code. It lets you accrete programs as a series of patches. Large organizations always tend to develop software this way, and I expect this to be as true in a hundred years as it is today.

[Phil] The phrase accrete programs as a series of patches is so deliciously visual that it makes me smile.

So many juicy bits in Paul's keynote. I guess that's what makes a good keynote.

Since Phil will be sending some traffic my way, this is a chance to redirect you to a piece from OOPSLA 2002 related to the above quote...

Noble and Biddle's Notes on Postmodern Programming.

Parallel Computing

Paul Graham, in his by now famous "The Hundred Year Language" keynote at Pycon 2003, questions the future of parallel computing. I am a little more optimistic than he is, see if you agree...

I expect that, as with the stupendous speed of the underlying hardware, parallelism will be something that is available if you ask for it explicitly, but ordinarily not used.

In a hundred years I expect languages to express opportunities for parallelism better than they do today. I expect language compilers to plan for parallelism more than programmers using those languages.

I would use pH as an example of where this could go more maintstream.

This implies that the kind of parallelism we have in a hundred years will not, except in special applications, be massive parallelism. I expect for ordinary programmers it will be more like being able to fork off processes that all end up running in parallel.

• Certainly programming over the next five years, not to mention a hundred, will be more event driven and concurrent.
• Languages like pH, mentioned above, will enable some of these concurrent processes to have some implicitly parallel computations.
• Other set-oriented languages like SQL (ignoring sequential stored procedures) also imply parallel computation.
• The Teradata DB architecture is already parallel and performs best on set-oriented SQL.
• As concurrent hardware becomes significantly more affordable we will see parallel implementations of open source databases for OLAP-oriented analysis.

Except in special kinds of applications, parallelism won't pervade the programs that are written in a hundred years. It would be premature optimization if it did.

On the contrary I think we'll be accustomed to parallel and concurrent computing through the use of simpler mechanisms that we have today, by and large.

• OLTP computing will take place through event-driven concurrent processes coordinated by simple message exchange patterns using simple "autonomic meeting places".
• OLAP computing will take place through simple notations with mostly implicit parallel capabilities.
• Inherently parallel structures in either of these scenarios will also take place through simple notations and a compiler's parallel planning capability analgous to a SQL compiler's query planning component.

So there will be much more parallel computing than we have today, but it will be simpler and mostly implicit. I think Paul Graham's OLTP/Workflow orientation is blinding him to the amount of analysis that takes place in the business world, and how much more can be automated in the next hundred years.

The Benefits of Computer Science for Non-CS Majors

This video taped lecture demonstrates the benefits of computer science for non-majors, just people who want to improve their ability to think.

But, as I have pointed out: Computer Science is not a science, and its ultimate significance has little to do with computers. The computer revolution is a revolution in the way we think and in the way we express what we think. I will defend this viewpoint with examples and demonstrations. -Gerald Sussman

A Computer Science Education for Thinking

If you are interested in computing and live near Salt Lake or Provo then Phil Windley will teach you how to think in this course that by all appearances is excellent.

I was fortunate to have a similar course with an emphasis on interpretation. Too many comparative courses are presented as a survey of disparate landscapes as opposed to a grounding in the fundamentals.

If you not in Utah, then look for similar courses near you. Indiana, MIT, Northeastern, Brown, and many others will have them.

If you are too far away then there are several on-line resources, e.g. on-line texts, Ars Digita University and some video taped courses like these from Berkeley that include Structure and Interpretation of Computer Programs.

Applications and Operating Systems as Message Pumps

KeithBa writes...

I also believe that OOP will be supplanted as the dominant programming model over the next 5-10 years as the XML and Web services model takes hold. I'm already building apps where I send hand-crafted SOAP messages within the application from one piece of code to another. Remember: the OS itself works off a message pump.

I agree, but we have examples already that are far simpler than SOAP/WS-xxx. Consider the open source Erlang system. Their messaging system is multi-language through the use of erl_interface and works with pure Java using jinterface.

What's the Simplest (distributed) Thing That Could Possibly Work?

Don Box writes...

My only point of contention is Sam's statement that when you control both ends of the wire, SOAP isn't the right answer.

I'm not convinced... For one, even in a homogeneous system, managing interface evolution in a CORBA/DCOM-like system is much harder than in a XML/SOAP-based system.

This might be a good argument if the choices were restricted to SOAP, CORBA, and DCOM. But remember, all three of these systems were intended for multi-language and multi-owner distributed interfaces.

If you truly own both end-points then you can provide very simple and transparent mechanisms such as those in Gemstone Smalltalk. Or far simpler, the open source Erlang.

And Don continues...

More importantly, as the supporting infrastructure for SOAP gets hardened into the (distributed and local) computing fabric, there is a lot to be gained from leveraging it. Today, this advantage isn't so obvious. Going forward, I think it will be.

No doubt there will be a lot to be leveraged as enterprises invest millions of dollars into a SOAP/WS-xxx infrastructure.

The question though is this the most efficient investment? Why?

The Web Services hype (and it is not much more than that today for most corporations) appears to me to be a "taking" in the making. If I controlled the purse strings I would sit back as long as possible to watch this stuff shake out. Let someone else spend the $$$ for now and then read the blogs that tell me what went wrong.

If I had to forge ahead with something I would do the simplest thing that could possibly work. I would look for the Web Services equivalent of Philip and Alex's Guide to Web Publishing

Abstraction, Interaction, Notation

Andy [Hertzfeld] says "the hard thing about being a programmer is keeping track of lots of abstract things." I disagree. The hard part of programming is creating static documents (whether tectual or graphical) that result in correct dynamic interactions. This static to dynamic barrier is something most people don't cross with ease.

I'm with Phil Windley on this. (The quote above is from his blog). We struggle to simplify dynamic interactions even conceptually. And we have not even begun to figure out truly useful notations.

Toss distance and asynchrony into the equation and we are still very much at a loss.

Roots of Web Services

From Infoworld's Web Services Report...

This declarative approach is the real essence of SOA, and of course it's not a new idea. I trace its lineage to MTS (Microsoft Transaction Server), which begat COM+ and J2EE. All these middleware technologies pull services (e.g., transaction management, connection pooling) out of components, and weave them into to the fabric in which the components are deployed.

We can (and should) go even 30 years back to discover some deeper roots of Web Services that run through but do not originate with MTS...

• CORBA 1.0 in 1991 was based on several earlier efforts from the 1980s.
• The modern transactional aspects have their roots deeper still at AT&T more than 20 years ago.
• Tuxedo is itself influenced by IMS in the 1970s.

Neo-Cons Have Long Had Damascus In Their Sights

As speculation about a U.S. invasion of Syria reaches a boil, it is helpful to look for hard clues. Many of the same neo-conservative hawks who campaigned for war against Iraq have signed reports in recent years that call for confrontation -- not mere engagement -- with Syria...

A 1996 report, titled "A Clean Break: A New Strategy for Securing the Realm," prepared for Israel's Likud Party leader (Binyamin Netanyahu) by Richard Perle, Feith, Wuermser and others urges focusing not only on removing Iraqi President Saddam Hussein from power, but doing so in the context of a larger strategy against Syria.

Whose Needs?

This article on xml.com states...

The discussion... will help readers recognize the need to coordinate a sequence of SOAP requests and responses, so that the total process looks like a business transaction.

(My underlining of the word "need".)

So whose needs are being represented here? Who actually needs this capability?

A bad habit we have is to avoid questioning the underlying assumptions that we build our concepts on. Distributed transaction protocols are not a "need". They are a means to an end. Why are we trying to recreate them in SOAP?

An alternative is compensating transactions which are also part of a proposed SOAP extension. This is a more practical alternative.

The question then becomes why do compensating transactions need to be part of a wire protocol? Whose design will fit my needs? How do you know?

Separation of concerns should be a key architectural principle. Can we keep the eager standards committees at bay long enough for clear evaluations?

Blackboard vs. Pipes and Filters

Abstracts from the book Pattern Oriented Software Architecture shed light on the difference between a "service oriented architecture" and a "space based architecture".

The Pipes and Filters pattern provides a structure for systems that process a stream of data. Each processing step is encapsulated in a filter component. Data is passed through pipes between adjacent filters. Recombining filters allows you to build families of related systems.

The Blackboard pattern is useful for problems for which no deterministic solution strategies are known. In Blackboard several specialized subsystems assemble their knowledge to build a possibly partial or approximate solution.

Blackboard operations are axiomatic in the sense that they can be used to compose a Pipes and Filters architecture and a wide variety of others. For example, how would you implement editing a set of shared documents using SOAP, including features like notifications when changes are made, a history of changes made, concurrent translation of the text components into multiple languages, and any arbitrary set of other operations?

I see two general choices for doing this in SOAP:

Create a new WSDL description and SOAP interface for each operation.

Create a generic WSDL description and SOAP interface once, whose arguments denote the operation and whose values expand as operations are added.

You also have to specify how coordination works to prevent processes from stomping all over each other's work. And of course you need to decide how to structure the shared documents to facilitate the desired mode of sharing.

However using a generic document space (a Blackboard) with an axiomatic set of operations for access and coordination, the problem is reduced to that last step in the pure SOAP approach, i.e. you simply decide how to structure the shared documents to facilitate the desired mode of sharing.

Lambda the Ultimate Label

Here's the PDF if you want it.

The Principle of Most Astonishment

What does the EJB specification have to do with the WS-xxx specifications?

Alan Knight offers several reasons against using Entity Beans, and anything but the simplest form of Session Beans, if you have to use EJB at all.

The root of the problem is the EJB standard was developed primarily in a paper specification (and I use that term loosely, having reviewed several drafts of the initial EJB spec). Had the standard been developed based on real experience with prototypes, I wonder if the result would have been as cumbersome?

How many prototypes are the WS-xxx specifications based on?

War, Rhetoric, Politics, Religion

Philip Greenspun:

If a president is in the middle of a war it is difficult to mount political opposition to that president without appearing disloyal and unpatriotic. Focusing media attention on a war prevents reporters from asking questions such as "How come William T. Esrey and Ronald LeMay, the two top executives at Sprint, deserved to get paid $311 million for their services to shareholders when the company's business and stock are in tatters? And then why is it fair that Joe Sixpack has to pay income tax but Esrey and LeMay didn't have to pay tax on their $311 million income? Would it have been fairer to divide the $311 million--equal to half of Sprint's 2002 profit--among the 13,000 workers that these guys laid off--$24,000 per worker--or possibly to the shareholders(!)?"...

Why have the Jews through the centuries made such good all-purpose targets for hatred? It is difficult to understand how Jew-hatred started so let's focus on the factors that have made it endure: (1) concentration in residence, (2) concentration in occupation, (3) smallness in number, (4) military weakness.

That only about half of Americans hold some of the same beliefs about Jews espoused by the Nazi party is comforting until one one reflects that Hitler was able to hold power in Germany with only 33 percent of the vote in 1932 and 44 percent in 1933.

...The United States Department of State estimates that "by the year 2010, America's Muslim population is expected to surpass the Jewish population, making Islam the country's second-largest faith after Christianity".

Aspect Oriented Programming

Jorg Kienzle and Rachid Guerraoui report on an attempt to build a transaction- processing system with AspectJ, where the key requirement is that transactions be executed completely or not at all (so that the system cannot debit one account without crediting another). They found it difficult to cleanly isolate this property as an aspect.

Aspect Oriented Programming is offered as a successor to Object Oriented Programming. The quote above illustrates that we have not found all the principles for when and how to apply AOP.

AOP is not addressing the right problem:

• The source notations, e.g. Java, even with some "aspects" of a program separated into distinct units, are still overly complex for most problems.
• The "computational units" like SOAP calls and SQL transactions, no matter what notation, are also overly complex and inappropriate for most problems.

Hypercard simplified the computational model for GUI applications with simple single-user databases.

Ward Cunningham's WikiWikiWeb simplified the computational model for Web applications with simple multi-user text databases.

What is the Hypercard or the WikiWiki equivalent for Web Services?

I have some ideas but I wonder what you think. Edit your thoughts into the page at WikiWikiWebServices.

OOP is *not* about classification

Look at this picture.

This is not what object-oriented programming is about. OOP is not about some kind of universal classification mechanism.

The Circle-Elipse discussion pops up on OOP lists more than once a year. The problem is that OOP is not supposed to resolve all knowledge representation issues. There is no OOP notation that can be used to represent all relationships I can imagine.

OOP is about computation, and so a Rectangle class in Smalltalk is not a "Rectangle" by the mathematical definition. The class Rectangle is a solution to a problem that is narrower than the mathematical definition.

OOP is a tool for solving computational problems, and so in any given system it may serve my purposes to have more than one kind of Rectangle class (in Smalltalk, I'd use namespaces to refer to specific classes). I would implement each class to solve a specific problem.

Whether or not I make some class named Circle be a subclass of some class named Elipse should be a decision made within the context of a specific computational problem.

So we should put this discussion to rest and deal with the real problem of OOP: even as a computational notation, modern OOPs are by and large too complex. They make us say more than is necessary to solve most problems they're applied to. Forget the Elipse-Circle discussion. There are real problems to deal with.

Jabber and Liberty

News on the Jabber front from Phil Windley.

A question: if you have Jabber, do you need SOAP?

Please add your comments on this JabberAndSoap Wiki page.

North Face Learning

Phil Windley refers to North Face Learning as a pioneer in accelerated computer science education.

The kind of acceleration described by Philip Greenspun is desirable.

My concern about what little information is available at the North Face Learning web site is whether the curriculum is computer science in general or simply training on today's popular hardware and software technologies. The industry needs more people who can deeply think and create to lift us out of this technological lull and into a real Cambrian Explosion that puts previous (and not really technological) Dot Bomb Era to shame.

If North Face Learning can pull that off I will be impressed.

Networks of Incremental Computing for On-going Results

In Sjoerd Visscher's blog response to The Hundred Year Language, he writes...

Historically programs are written to execute with existing data as input. But to be competitive on the internet, results need to be continuously updated while new input comes in. For example, you can sum live data by summing your input every time it changes. But this pseudo code would be much faster:

initial:      sum = 0
on add(v):    sum += v
on remove(v): sum -= v

I'm not even sure if i'd prefer to program this way, or if I'd just program summation for a collection, and let the compiler figure out the above code.

I can tie these thoughts in with some other sources that seem more or less related to me.

• Magnus Carlsson writes about Incremental Computations using Haskell. The simplest example of this being a spreadsheet.
• Ralph Johnson and Hiroaki Nakamura write about an adaptive framework for business systems.

These all represent to varying degrees the future of computing systems:

• assembled incrementally
• various participants
• somewhat ad hoc
• the whole emerging from the sum of its parts

And what they have most in common is there is no single "result". Instead the value of the system is the on-going re-evaluation of the previous results.

The Hundred Year Discussion on Language

The periodic static/dynamic discussion has reared its head in a more interesting way than usual on Lambda the Ultimate. My latest prgnostication goes as follows, but there are more interesting contributions than mine in this discussion.

First Alex Peake writes...

BTW, Smalltalk was conspicuously absent - a wonderfully productive dynamically typed, reflective,... development environment!

Nay, I say, the future is LISP (well Scheme maybe) and Smalltalk (if only someone could afford to Market either).

Python probably belongs in this category too, although it has way more cruft than Smalltalk or modern Lisps.

Very interesting thread here. One hundred years is a long time. I predict the next ten years will be more dynamic along these lines.

Perhaps after that better type checkers (but they'll just be called theorem provers), model checkers, etc. will provide more verification as systems begin to program themselves and each other with less manual input.

Subtle humor from a Lisp programmer

From Paul Graham's The Hundred Year Language...

If SETI@home works, for example, we'll need libraries for communicating with aliens. Unless of course they are sufficiently advanced that they already communicate in XML.

GNU Smalltalk 2.1

A major new releaseof GNU Smalltalk is available.

People Making a Difference: The Lives of Good Public School teachers

I made a couple of cynical comments in response to Philip Greenspun's attitude toward public schools and a friend of his who seemed to be shrinking away from minor hardships, while others are overcoming more significant ones every day. Another participant asked me to clarify my position. I did so constructively rather than cynically, and copy my response here.

What's the problem?

This story is unmotivating.

There are a lot of problems with public schools. The best people, including teachers, are trying to fix them, and are having success here and there.

My wife works in the public schools, and will be receiving her masters degree in education in a couple of weeks. She's worked in ESL (English as a Second Language) and still chooses to work in Title I schools (generally not well off, a lot of parents have many problems of their own).

Does she get frustrated? She doesn't show it much. The stories she tells are about the kids and the conditions under which they struggle to learn: the worst are dirty, undernourished, unaccepted by their peers, unsupported by their parents.

Most of the stories she tells are about how well *every* *single* *one* of them responds to the creative ways she and others reach out to them. Poetry lessons that get kids who can hardly write creating the most amazing pieces of work. (I've read many... you wouldn't believe them.) Science lessons with worms that spark their imaginations. School nurses who creative games with soap, deoderants, etc. to get kids to clean themselves up without embarrassing them or lecturing them or their parents. Teachers who supply parties for the birthdays of kids whose parents can't afford it... but they don't let on to anyone that the parents did not do so themselves. Teachers who also show up at the doors of these kids to bring them birthday presents (because they really know they could use a new shirt!)

I know I don't have the skills to do what my wife does, and other people like her. She should be making three times my salary.

You get the idea. Public schools don't need people like Philip's friend, and middle class students certainly don't need to "learn how to obey" so called privileged "leaders" like Philip Greenspun.

Do you think that there are people worse off than this writer and nobody should be allowed to complain if there are people worse off than them?

I hardly see anything unsurmountable in Philip's post. She has the right to complain, but there are far worse problems that better teachers are overcoming every day. Let's write about those people and learn from them.

Do you think that these are actually quite good conditions for someone to work under. So, once again, no right to complain?

Like I wrote above, I've heard far worse that were made better. It does no good to complain to Philip Greenspun. I call that whining. I'd rather hear how she made a difference, but I see no evidence she has the skills for that. Maybe she can teach math to bright kids. Who couldn't?

Do you hate the public school system so much that you rejoice in it's falling apart? And try to denigrate anyone who complains about it's state?

No. Instead I celebrate those who overcome obstacles and make *big* differences in the individual lives of less fortunate kids.

I am in awe of the people who can and do accomplish meaningful things in the public schools. I couldn't. But I can't indulge whining like what we've seen here.

Are you FIT?

Just rec'd the latest ONJava newsletter where the editor mentions FIT. Highly recommended for feature acceptance tests and as a replacement for JUnit-like unit test frameworks. FIT is implemented for several programming languages.

Welcome to the latest Java newsletter. Your editor attended a talk just last night on Ward Cunningham's FIT--if you've struggled with automated customer tests, you owe it to yourself to look at this framework. I hope we can find some articles on the subject soon.

Deprivation

Ben Brewer makes a point in his comment on Philip Greenspun's retreat from society. I agree from where I sit. Sometimes we just don't get it, being so privileged, we're blind and we don't even know it.

"Relative deprivation in the U.S. is sometimes not having the nicest SUV on the block."

(Also discovered his blog on fatherhood.)

Fall back men. There's people here who *yell*!

My wife works in the public school system. There're much worse stories than this, and sadly, they even involve kids, health, and well-being, which the following story does not really.

I am sorry. I cannot feel much sympathy for Philip Greenspun's tale of retreat to private schools. Better to have this kind of thinking isolated in the in-bred society anyway I guess. Make room for people who have the right attitude and can actually make a difference rather than just be a difference.

I too easily slip into cynism, and this is what I wrote on Philip's comment page...

Poor little thing. She even cried?

Must be rough. Retreat to the private school. Fall back, men.

Note to self: gotta close on the house in the gated community.

Hurry, men. There's people here who *yell*!

A Mighty Wind is Blowin'

Yes, if you liked Spinal Tap, try the Folksmen too.

Coming soon, I hope, to a theatre near, well... me. Meanwhile.

The Hundred Year Language

The latest from Paul Graham.

41 calls 43 with congratulations on the new Iraq

Listen in on a conversation between 41 and 43.

*Dynamically* Tolerating Failures

Luke Gorrie quotes early Erlang philosophy on the Lightweight Languages email list...

In programming large systems, many small programming errors _will_ be made - we view this as inevitable. Formal systems and exhaustive test procedures are currently not capable of ensuring fault free software for systems of the size and complexity of modern telecomms applications. Given that errors _will_ be made, we are interested in the problem of detecting and handling those errors in such a manner that the system as a whole exhibits satisfactory behaviour in the presence of errors.

CS Foundations: CSP now on-line

Hoare's CSP book is now available on-line.

Shouldn't this be required reading as part of an undergraduate computer science degree? Why not? This vocabulary should be on the tips of our tongues, like the periodic table.

Whither Smalltalk? A Classic Notation for Expressing Computation

Whither Smalltalk?

Instead I think the question should be, "Whether Smalltalk?" And the answer could very well be yes.

VisualWorks 7 is loaded with Internet capabilities: servlets, server pages, SOAP (Open talk). The VM is so much more mature than the CLR and JVM implementations.

The company appears to be solid, and has backed VisualWorks longer than anyone gave them credit for. Cincom rescued Smalltalk from history, after ParcPlace nearly bungled one of the truly great software systems of all time.

S# appears to be close to release for dotNET. Not only could this put some spark back into Smalltalk, but it could pave the way for widespread success of dynamic languages in general on the CLR, through the collaborration with Microsoft and the specific implementation techniques.

Not to mention that Dolphin and MT are still solid Smalltalk implementations for Win32 and on multiple platforms Squeak and GNU Smalltalk are too. The combination of VisualWorks and S# and these others make a cross-platform opportunity for Smalltalk. Systems built for S# or VW to a large degree should run on the other.

VW 7.1 is supposed to provide a more solid (non-beta) native Mac OS X implementation and VW is already solid on Linux and other Unix systems.

Smalltalk is a simple, classic notation for expressing computation. Code written 20 years ago still runs on modern implementations.

Why build important computational assets in notations that are overly complex (i.e. they make you say more than necessary) and are tied to implementation decisions that will soon be outdated (i.e. the "standard" notations are regularly updated with new features that compell adoption by marketing lust and lead to a chronological stratification similar to carbon dating)?

Java and C# books become outdated in a year. They no longer teach the notations the way developers want to use them. On the other hand, a developer can learn 90 percent of Smalltalk by picking up 1983's Smalltalk-80: The Language and Its Implementation!

Far better to build your important computational assets in a notation that is already long-lived, simple, flexible, and efficient, not to mention mostly unchanged for 20-30 years or more. There are a two of these notations that make sense to me: Lisp and Smalltalk.

I could go with either. Smalltalk is easier to explain to developers using other object-oriented languages.

Winter in Boston

From Dave in Cambridge...

On Friday I asked if it snows in Cambridge in May. Yesterday about 20 people told me it does. It's even snows here in June. Amazingly they're forecasting snow tomorrow and Tuesday. Two to four inches tomorrow. Hellp

Lived there for manyyears. First winter, experienced a Nor'easter. Think you're calling for help now? Just wait.

Space-based Programming is About Spaces, Not Transports

SOAP is a virtual transport. WS-xxx extensions are sometimes about transporting, sometimes about routing, sometimes about data (documents).

The appeal of space-based programming (e.g. Linda and JavaSpaces) is that it is all about spaces. Transports get commands and data (documents) from processes to spaces and from spaces to processes. Transports are inconsequential. There can be many. They can come and go and evolve.

The essence is the same: coordination through a set of simple space-based commands that can be combined in an infinite variety of ways.

REST and Linda

Does HTTP make a good coordination language?

So the next question is, how do the Linda coordination language and the HTTP coordination compare?

Linda has 'out', 'in', and 'rd'.

• Out - put something in your process out into the shared space.
• In - bring something from the shared space into your process (it is no longer in the shared space).
• Rd - copy something from the shared space into your process (the original remains in the shared space).

HTTP has Get, which is kind of like Rd. Except a Linda system is guaranteed to implement the simple definitions of Rd. A web server can implement anything at all for 'get', i.e. there can be untold side effects.

None of the other HTTP commands are specified to act like the remaining Linda commands. And every one can have untold side effects.

My quick assessment tells me that HTTP is not nearly as simple as Linda for the purposes of coordination. But a web server could choose to interpret some of the HTTP commands to behave more like the Linda commands.

My experience as a developer tells me Linda's commands make good coordination 'combinators', i.e. they can be combined into many variations of desirable sequences for coordinating multiple processes.

Wozniak and Stallman: Separated at Birth?

From Scripting News...

Steve Wozniak via email: "In my opinion the guiding principle, for all copyrighted material, should be that if any material is presented to your senses (sight, sound, etc.), free or paid for, then you should have the right to memorialize it, to record it and to get it later."

Presumably then this would apply to software. Which is nearly Stallman's position, right? (Stallman includes freely modifying, not sure if Wozniak's does. Stallman's includes freely sharing, not sure if Wozniak's does.)

A Perfectly Phrased Question re: Distributed Computing

Jon Udell puts it like this, referring to XML-RPC...

"Does distributed computing have to be any harder than this? I don't think so." In retrospect, that was a useful thing to ask at the time. Of course it conveniently swept under the rug a whole set of issues which we now gather under the rubric "service-oriented architecture." Today I might rather ask: "Does distributed computing have to appear to be any harder than this?"

Well said.

I attended another seminar last night on "Service Oriented Architecture". The result was another audience of essentially web and VB developers sitting in stunned silence as they attempt to relate the presentation to anything remote familiar in their own experience. Yet again the presentation included a parade of the various Web Services extensions:

• WS-Addressing
• WS-Coordination
• WS-Inspection
• WS-Policy
• WS-Referral
• WS-ReliableMessaging
• WS-Routing
• WS-Security
• WS-Transaction

Of course Microsoft and IBM will attempt to "peanut butter" over the inherent complexity in their respective IDEs. But is this really necessary? Most of the cases I think could be solved more simply and so with a simpler presentation for the typical developer.

The two missing aspects of every presentation I've attended so far are:

• Coherent use cases that demonstrate the advantages of when and how to use each of these features. A "Use Case" specifically would indicate who (which developer roles) would be interacting with which services during the processes of developing and deploying a "Service Oriented Architecture".
• Evaluations of alternative strategies. These standards-in-progress are not mature. They do not strike me as "the simplest thing that could possible work". And yet they are just assumed to be the new "distributed system language" of the future, that will make everything better. I'd like to have some transparency into the decision making process, and see what alternatives have been discussed. Unfortunately having participated in the development of standards a couple of times, I know that alternative evaluations are rarely given their due.

Adam Bosworth paints the big picture and the broad strokes look attractive

From Jon Udell, Adam Bosworth paints the big picture. The broad strokes look attractive, but I think there is a simpler picture that can live within the same basic frame.

Coarse-grained messaging. Get everything you need in a single round trip. Why? If not, your shut apps down.

I am interested in space-based messaging. It's cheap and scalable to place a "semi-persistent document space" in the middle of communicating processes.

Even now, the numbers are big: 3000 messages/second flowing through Web services message brokers.

"Document spaces" are a useful abstraction for managing the evolution of documents and processes over time, location, size, amount, and definition, five dimensions of loosely coupled systems.

Public contracts. You can change your operating system, you can change your object model, but don't change your public contract. We know this works. "The proof point is the web."

This works to a point. HTTP should evolve more slowly than a supply chain process, though. So we need flexible contracts and systems that can tolerate change. Contracts must include tolerance specifications. And so defining exceptional behavior is at least as important as defining the straight through behavior.

Asynchrony. "I've been surprised by how slow some companies have been to pick up on asynchrony." The recently-published WS-Addressing "contains the core plumbing for standards based asynchrony."

I have to say I have almost zero knowledge of WS-Addressing. My fear is that it is unnecessary. I am a big fan of black boxes. A process only requires a simple end-point definition. Put anything you want to behind that end-point.

Databases are great at asynchrony. Data comes and goes at unspecified intervals. Space-based data provides a simple database-like end-point for data exchange: putting a document in some space with this pattern implies an offer of or a request for services of some nature.

I am a big fan of black boxes. Another name for a black box is a "public contract". See above. Do what you want with that document, but obey the contract. My process interface only changes when the contract changes.

Message-driven model. We need a programming model for message-driven programming. How does a developer write code in that environment? "It's no problem for systems programmers, but for everybody else, it's a challenge to make it easy for them to write apps that wake up when a message comes in."

This is because the systems programmers run the show. Frankly, they tend to like complexity. I know, I've been one for most of my adult life, working on operating systems, compilers, digital logic simulators, manufacturing automation, video conferencing, financial systems, object-oriented databases, and probably something else.

There are a few examples of simplicity in the software industry. Latch on to those for your life, and question anyone who thinks you should let go.

By the way, to give you some bearings on my position: I think Java and C# are *horribly* too complex. The WS-xxx standards give me heartburn.

Declarative query. "We don't have a good query mechanism for XML... In practice, Web services messages have known schema. A query language that knows the schemas can optimize."

The only point to add here is that schemas and query languages must still allow for change. Queries should say just enough but not too much. Optimizers should not influence the language, simplicity should.

Scripting.

I simply interpret this as the need for simple languages and architectures that can be adapted at run-time. My implementation languages (Scheme, Smalltalk, JavaScript) are my scripting languages. Seamless.

Two complementary technologies will enable "dynamic runtime-modifiable systems, rather than static compile-time systems that make you shut down and restart your services." They are:

• XML Query, for declarative work, and
• ECMAScript for XML (E4X), for procedural work (see also this heads-up)

I'm not overly familiar with XML Query, but I appreciate the point.

XML repository. In Adam's vision of SOA (service-oriented architecture), a message broker sits in the middle of everything. It sees a lot of self-describing XML messages, and it has to do high-performance storage and retrieval. So it's a true XML database. There's nothing relational about this problem. But, being an old database guy, Adam sees an opportunity. In the message broker, the XML data store is not inherently multiuser. "It deals with messages in flight, and this makes it possible to do all sorts of optimization."

I hope to get some specifics on this. What does it mean for a data store to be "not inherently multiuser" and yet "sit in the middle of everything"?

Again the most flexible software device I have encountered is the document space. Gelernter's rules, now more than 20 years old, are the 80% sweet spot between messages, processes, databases, workflow, etc.

Great talk. It's always refreshing to watch Adam Bosworth paint the big picture!

Wish I was there.

U.S. Inactions Hindering Search for Al Queda and Taliban?

The chief of the US forces in Afghanistan, Lt-Gen Dan McNeill, said he was "frustrated" that the West had "not made a more bold step" to rebuild Afghanistan, adding that this could be an important lesson for Iraq. The US search for al-Qa'ida and the Taliban would have been easier if the aid had flowed faster, he said.

Why is all this a surprise again?

Maureen Dowd's op-ed she should have written before the conflict started...

Some of their soldiers are mere children. We know we have overwhelming, superior power, yet we can't use it all. We're stunned to discover that the local population treats our well-armed high-tech troops like invaders.

Why is all this a surprise again? I know our hawks avoided serving in Vietnam, but didn't they, like, read about it?