Risk Analysis and Decision Making

Ted Neward attended a talk on risk analysis and decision making. This is the gem for technology facing (and technology fascinated) developers. Heed...

Cost-benefit analysis is a skill, not an arcane art, and more technical leads and architects need to spend more time working on it. But we don't, because it's not NEARLY as cool as working with the latest O/R mapping layer or tool, despite the fact that using the latest O/R mapping layer or tool in of itself represents a risk that should be carefully examined using a cost/benefit analysis.

If only we got as excited about adding 25 points of business value to a system.

The Psychology of Query Programming

The intent is there, the accomplishment is there. The only bummer is the notation of the query language itself. Who wants to write code like that?

Taking a Stand for Our Public Airwaves

Whatever your political stripes, if you are an American who believes the airwaves are still a public resource, do all of us a favor and participate in this brief boycott of CBS.

Remember, the airwaves are licensed and renewed by the public to CBS. They are not privately owned.

Descriptive Temporary

Michael is examining some code in Smalltalk and C. Here it is...

" The original Smalltalk example. "
canvas displayLineFrom: (topPoint x - gibDistance @ topPoint y) to: (topPoint + 1).


/* The original C example. */
canvas.displayLineFrom_to( Point.asPoint(topPoint.x() - gibDistance, topPoint.y()), topPoint.MoveBy(1,1));

I would not hesitate to use the "Descriptive Temporary" pattern. (Is there one? Maybe there should be.) Taking this to the extreme, I might end up with something like the following in Smalltalk and C, respectively. Keeping to short methods, I would not have much more code than this in a single method or function. I think this style, while longer, is easier on my brain...

" The Smalltalk example with descriptive temporaries. "
x := topPoint x - gibDistance
y := topPoint y
lineBegin := x @ y
lineEnd := topPoint + 1
canvas displayLineFrom: lineBegin to: lineEnd.


/* The C example with descriptive temporaries. */
int x = topPoint.x() - gibDistance;
int y = topPoint.y();
Point lineBegin = new Point(x, y);
Point lineEnd = topPoint.MoveBy(1, 1);
canvas.displayLineFrom_to(lineBegin, lineEnd);

Now, this also has the effect of making Smalltalk more readable than C since there is less line noise. But I learned Smalltalk before I learned C.

Addressing DLL Hell with 40 year old technology

Ted Neward anticipates solutions to runtime downloads, DLL hell, and other nightmares, realizing Java is in a similar boat...

And lest anybody start to think otherwise, the JVM presents the same problem as the .NET Runtime, and when we solve the .NET Runtime versioning issues, we'll have solved the JVM ones, using the same solution.

The linking problem has been solved for Java... there is the GNU Compiler for Java, as well as commercial native Java compilers. JNLP is intended to solve versioning problems. I have used it, but never in the presence of versioning problems.

The bigger answer is time. Systems will thrash less as they mature. This is for better or worse, since the ability to change even undesirable features is inversely proportional to the number of dependencies applications have on them.

Now is a good time to bring up a theme of mine: the software world has a few examples of highly dynamic systems; the rest are evolving to become more like them, truly. Now is a good time to point out that highly dynamic systems like Lisp and Smalltalk have a long, proven history of backwards compatibility:

1. Because they are mature. They've been around for decades.
2. But also because they allow themselves to be amended and redefined down to their core.

If your application requires a feature that comes out of the box in another dialect or from a previous version, chances are good you can take that feature with you to a new dialect or version. If you wish the root class Object had a feature the vendor did not think to include, you can add it yourself just in the applications that you designate.

The Right Stuff

this stuff is being integrated at the platform and framework levels on both the MS and non-MS side so that developers can no only take advantage of it without having to understand all the nitty-gritty details, but also gain maximal benefits with minimal code.

There be dragons.

I'm getting punchy. I better go.

Messages and Objects, Again

Good points on all sides of the ongoing debate on messages and objects and distributed systems.

I will just add one observation...

Dropping a feature is not the same thing as fixing a feature.

More on Inheritance Being Evil

Much lamenting of the perils of using inheritance in an OO language. But these perils are part of a well worn path. Take heart and remember to learn from the mistakes of others,

White Box and Black Box frameworks... it's a matter of evolution.

Who's the Loser?

I can only whine a bit in response to this desire to create new flow of control sytax.

Yesterday I posted a new way to define functions and their tests contiguously using a new syntax in Scheme. It took about 10 minutes and less than 10 lines of code.

Likewise in Lisp as well as Smalltalk defining new control structures is childs play, mainly because the control structures in those languages are defined using the very same mechanisms that are available to you, the dear programmer.

Yes, I know, I am a loser to even bring this up.

Is COM interop more stable than dotnet's?

I'm recalling something for the the dotnet reality check...

Because dotnet is a more "complete" object model than COM (inheritance and shadowing of public and private aspects) and because different languages (VB.NET and C# to name two) have somewhat different rules for inheritance, it may be that dotnet has more interop problems than COM. I couldn't say because I don't know COM.

A problem with dotnet inheritance was documented some time ago. I don't believe it has been fixed.

Minimum Requirements for One Runtime, Multiple Languages? Less is More

Dare Obasanjo follows up on Jon Udell's investigation into all things dotnet with a piece on language neutrality.

Two points:

• The dotnet framework to date is "language neutral" within a very narrow definition of neutrality.
• Should we care?

What is really needed in language neutrality? A few thoughts:

• Shared nothing, scalable threads. Let our languages run in the same address space, but force them to say at an arm's length from each other.
• Short-circuited message passing. Let our language collaborate the same way whether they are together or apart. Just make it more efficient if they are together.
• I/O and other system services. Let them use a common set of system services, virtualize the limitations of the host OS.

There is no reason why one could not build a Longhorn on this "less is more" model, and the result would have more potential (flexibility, scalability) than the one that appears on the horizon, as far as I can see.

A thread of conversation on the agile-testing yahoo group is speculating on the ability to define a function and its tests "contiguously" to reduce the effort to make the contextual switch between "coding" and "testing".

Here is a simple Scheme macro to approximate the idea...

; The following is a macro that allows you to supply test input and
; test results when you define a function. After the function is
; defined, the tool automatically runs the function on the test input
; and checks to see if it matches the expected result. There are a
; number of enhancements (list of test inputs, define test function,
; etc.) to consider. 

(defmacro define-eg (name-result parameter-input . body)
  (let ((name (car name-result))
 	(eg-result (cadr name-result))
 	(parameters (map (lambda (p) (car p)) parameter-input))
 	(eg-input (map (lambda (p) (cadr p)) parameter-input)))
    `(begin (define (,name ,@parameters)
 	      ,@body)
 	    (equal? ,eg-result (,name ,@eg-input)))))
 
(define-eg (product-of-sums 108) ((a 5) (b 7) (c 9))
  (* (+ a b) c))
 
> (load "define-eg.scm")
> (define-eg (product-of-sums 108) ((a 5) (b 7) (c 9)) (* (+ a b) c))
#t
> (product-of-sums 5 7 9)
108
> (define-eg (bad-product-of-sums 108) ((a 5) (b 7) (c 9)) (+ (* a b) c))
#f
> (bad-product-of-sums 5 7 9)
44

PERT and Agility

Robert Martin (home, blog) in Software Development magazine provides a simply useful introduction to PERT charts, Critical Paths, and a couple of simple tools to replace them for software project management.

Catfish in the Memepool

Catfish in the Memepool... I have been enjoying Brian Foote's writing for almost 15 years, I guess, and now he has a blog (with a feed).

A Method for Design By Contract

At the BCS OOPS Resource page, the slides from Richard Mitchell's December session on a "Method for Design By Contract" (zipped ppt).

This presentation is worth reading on two levels:

1. This is one of the best presentations of Design By Contract, what it is and how to do it.
2. This is just a good example of an expository presentation in general.

BTW the ideas in Design By Contract are useful for defining services in general.

Programmer Tests vs. V&V Tests

...when up to half of the output of a full-blown TDD-style project can be test code, we’re going to want to find ways to automate and streamline the effort.

We have to make a distinction about what kinds of tests we're writing and what kinds of tests we want to automate.

This goes back to an exchange I had with Jarno Virtanen earlier this month. In that exchange I wrote...

The TDD tests should be just enough to get to a satisfactory design. Depending on the system being constructed, you should still consider acceptance tests, including performance tests and more complete suites for functional coverage than were needed just for the design process.

We don't want to "over automate" the TDD tests since they are programmer tests, i.e. a programmer's tool for thinking.

We do want to automate the validation and verification tests because we want to eliminate boredom and error, allowing the V&V team to spend their time on activities like creative exploratory tests.

Idiomatic Lisp and Idiomatic Scheme

I guess I agree with Brian Marick's code reading style more than Richard Gabriel's. Rather than using an optional private parameter in a public function, though, I prefer to use the "named" let, with an accumulator there, for tail recursive functions.

Is it idiomatic? It's fairly common in Scheme. I am not sure why it has not been adopted more widely into Common Lisp.

;; Richard Gabriel's Idiomatic Common Lisp
(defun fact (n)
  (labels ((f (n acc)
	      (if (<= n 1) acc (f (- n 1) (* n acc)))))
    (f n 1)))

;; My (Idiomatic?) Scheme
(define (fact n)
  (let loop ((n n) (acc 1))
       (if (<= n 1)
	   acc
	 (loop (- n 1) (* n acc)))))

;; "Old School" Common Lisp
(defun fact(n &optional (acc 1))
  (if (<= n 1)
      acc
    (fact (- n 1) (* n acc)))

Turn a Handheld into a Desktop

As PDA hardware gains more GHz and MB, attaching large monitors, keyboards, and mice directly to the cradle makes a great deal of sense.

I would think this makes much more sense than a table PC for a large number of users. Well, me.

Bluetooth, anyone?

Composing (Avalon) and Partial Classes (XAML)

Why does Avalon at once promote composition (slides (PPT)) *and* simultaneously base XAML on partial classes, a new language construct for weaving a single class from multiple sources?

Why is composition not good enough? Or if not, why is inheritance not the next best thing? I have yet to see a rationale for introducing another mechanism for organizing object-oriented code.

Answers can go here.

Business Week: "Waking Up From the American Dream"

Paul Krugman, on Business Week's article titled "Waking Up From the American Dream"...

Suppose that you actually liked a caste society, and you were seeking ways to use your control of the government to further entrench the advantages of the haves against the have-nots. What would you do?

"Managed Runtime"

From the things that make me go, "Hmmm", department...
Years from now will programmers wonder what the heck the "managed" in "managed runtime" means?

Inversion of Control

Folks are trying to push back the history of the notion "Inversion of Control".

I can't recall the phrase itself, but the notion goes back at least into the late 1980s when the object-oriented community was talking about "libraries" (you call it) vs. "frameworks" (it calls you), e.g. Designing Reusable Classes, Johnson and Foote, 1988.

I bet the essence of IoC was also expressed by Peter Deutsch in 1983's "Reusability in the Smalltalk-80 Programming System". I can't remember if this paper actually used the term "framework". But that was a main point, and draws on the Smalltalk work they did at PARC in the 1970s, including the first "famous" inversion of control, Model View Controller, with Trygve Reenskaug

Catch 22: Atom is available, but it is all or nothing

Update: This from Blogger in response... There is not a want to publish both RSS/Atom. If you do not want to yet switch to Atom, you can use your RSS feed until you choose to switch to Atom.

So if you want to publish both RSS and Atom from Blogger, let them know.

I can publish Atom if I change a setting in my blogger configuration. But I won't, at least not yet.

I've sent an email to Blogger/Google to explain this. Apparently, (from what I can see in the UI), I have to choose between RSS 1.0 and Atom. I cannot have both, unless I write it myself, and I have no interest in that.

How can the Blogger base switch to Atom this way until all the universe supports it first? How will the universe be encouraged to support Atom unless the Blogger base publilshes it? Catch 22.

Leave one test failing

One thing that Kent Beck mentioned in Test Driven Development was that if he's not finished test-driving a chunk of code, but has to leave it for a while, he'll leave one test failing. When he gets back to pick up where he left off, he runs the tests, sees the failing one, and by making that test pass, gets back into the mindset needed for that design problem.

I do this all the time. This is especially handy when you are programming at home and are susceptible to frequent family interruptions. It's also handy when you are done for the day. When you come back in the morning, the test is there ready to tell you exactly where you left off.

What you need to know about the next piece...

...is contained in the last piece...

The ceramics teacher announced on opening day that he was dividing the class into two groups. All those on the left side of the studio, he said, would be graded solely on the quantity of work they produced, all those on the right solely on its quality. His procedure was simple: on the final day of class he would bring in his bathroom scales and weigh the work of the quantity group: fifty pound of pots rated an A, forty pounds a B, and so on. Those being graded on quality, however, needed to produce only one pot -albeit a perfect one - to get an A.

Well, came grading time and a curious fact emerged: the works of highest quality were all produced by the group being graded for quantity. It seems that while the quantity group was busily churning out piles of work - and learning from their mistakes - the quality group had sat theorizing about perfection, and in the end had little more to show for their efforts than grandiose theories and a pile of dead clay.

Art is human; error is human; ergo, art is error. Inevitably, your work (like, uh, the preceding syllogism) will be flawed.

What you need to know about the next piece is contained in the last piece.

I have nothing to add. Buy the book.

dotnet events and external language bridges

Update: Note that the Python.NET described here is *not* a CLR implementation of Python. This is "regular" C Python. You drop the bridge DLLs into a Python folder, then import CLR, and at that point have access to all the dotnet classes, events, etc. There's been some confusion of this with the more experimental IronPython, which is an all-new CLR implementation of Python and not ready for production use.

The VisualWorks Smalltalk dotnet bridge does not yet handle dotnet events...

But registering for events in .NET was not part of the version 1 plan. It is extremely difficult and there is no solution whatsoever at the moment (AFAIK for any *-.NET bridge).

The problem is that you need to give .NET a delegate object to call (type safe callback) and that has to be part of the managed world. We are thinking about synthesizing such objects (their classes) and having an extra callback mechanism with dedicated marshaling

For the record, the very nice production quality Python.NET does support dotnet events in the manner described above. I have not looked at the code for how, but it is open source.

Here is an example of registering for a dotnet event in CPython...

    def add_menu(self):
        self.Menu = MainMenu()
        self.tool_menu_item = MenuItem()
        ....
        self.ellipse_tool_menu_item = MenuItem()
        self.ellipse_tool_menu_item.Text = "Add &Ellipse"
        self.tool_menu_item.MenuItems.Add(self.ellipse_tool_menu_item)
        self.ellipse_tool_menu_item.Click += EventHandler(self.pick_ellipse)
        ....
    def pick_ellipse(self, sender, args):
        self.ellipse_tool_menu_item.Checked = True
        self.canvas.current_tool(self.canvas.ellipse_tool)
        ....

Writing Run-time Optimizers in High Level Languages

Miguel de Icaza is crazy about this...

One of the fascinating things about Jikes's JIT engine is that it was completely written in Java. Absolutely mind-blowing. The upside is that many things can now be done in a higher-level language than C like JITing and garbage collection, the downside is that these systems are fairly heavy during compilation time and are best suited for server use and not desktop use.

And I am crazy about the following, and wonder how they compare...

This talk summarises two decades of work on Smalltalk and Self compilation and virtual machine technology and describes a novel attempt at an adaptive optimizer for Smalltalk that is written in Smalltalk and to a meaningful extent, portable across implementations... the explicit representation of code and execution state make it convenient to implement a hybrid architecture where an adaptive optimizer written entirely in Smalltalk, guided by type information obtained from the VM, produces optimized bytecoded methods that are executed by a more-or-less conventional VM.

Smalltalk about the enterprise

Roy Osherove is asking about Smalltalk in the enterprise. Gemstone Smalltalk was the first object-oriented application server to my knowledge. The core architecture has been in development for about 20 years. The systems developed include manufacturing floor, billing, banking, insurance, shipping, and logistics applications.

There are a surprising number of new web applications running Gemstone Smalltalk too. I would probably choose Cincom's VisualWorks over Gemstone today since it now has a lot of server capabilities, better web tools and objects, as well as better product support.

AS/400 and Smalltalk

Richard Demers connects the IBM AS/400 (which has an object/capability based OS) and Smalltalk.

The AS/400 is an interesting animal. Actually, seven years ago Dave Thomas (founder of OTI, bought by IBM) gave a keynote talk to Gemstone's company meeting. There he suggested that Gemstone Smalltalk be ported to the AS/400.

I am not sure the suggestion was seriously discussed at the executive level. The company shortly after decided to take on IBM, Sun, and Oracle in the Java/J2EE market. (This was before Weblogic made a name for itself.) And the rest is history... Smalltalk is still their cash cow.

Why Smalltalk?

What makes Smalltalk that much more productive?

Jim hits all the right notes. I would add a couple other notions...

There is no main. Rather than writing programs, you construct objects. Rather than running programs, you test objects. A Smalltalk environment will have several browsers up at a time. Some for browsing, others for editing. Another typical scenario is to have several *workspaces* up at the same time. A little code here, a few tests there.

This ties into my other notion, that Smalltalk is essentially *testable*. Before the onslaught of XP, Smalltalk culture was even then based on writing a little and testing a little.

Plus common practice (trivially achieved in the Smalltalk environment) is to keep this "image" of multiple windows and multiple, incremental tasks flowing between sessions. Smalltalk has trivial persistence that supports the development environment greatly.

Adventures in 64 bits

Michael Lucas-Smith expands his adventure with registers to the 64-bit world.

The 64-bit world is interesting for another reason: data representation. Not only does the IA-64 have a large number of registers, and not only can you "rename" them so that they don't always refer to the same physical register, but since they are 64-bits wide you can fit a lot of data into them as "immediates" in a dynamic language like Smalltalk.

How would you represent a 64-bit floating point number in Smalltalk?

In the 32-bit world, you would have a 32-bit object reference where some number of bits would indicate the "tag" or the data type and some number would indicate the memory location where the data actually resides. The tag may be zero bits if all data is stored in memory and never immediately in the reference itself. Another option is to use a 2-bit tag, say the lowest two bits, to indicate the data type and then align the allocated memory to account for the "wasting" of the lowest two bits. (They indicate tags instead of pointing to byte aligned allocations.)

These two bits are then 00 or masked off to get to the allocated memory. Otherwise some other pattern (11, 10, 01) indicates the data is immediately in the reference itself. This may be a 30-bit integer (again, the tag is wasted), or a character (some other byte or 2-byte integer in the upper 16-bit word) or some other design. (Google should turn up some creative designs, this is off the top of my tired head.)

But for 64 bits the world of encoding gets *very* interesting. Back to the 64-bit floating point number...

In the 64-bit world, just represent a 64-bit FP number as... hey, a 64-bit FP number. Use all the bits!

But then how do you represent everything else if all the bits are used for FP? Well, there are a large number of 64-bit arrangements that are not actually FP numbers, rather they are "Not a Numbers" or NaNs. But not all 64 bits are used to indicate some arrangement is a NaN. Actually a small number of bits are used to indicate NaN (something like 10 or 11).

A very large number of integers can be represented as 64-bit FPs. Only the *really* *really* large need to be memory allocated. A lot of other interesting data can be represented immediately in the 64-bits (and so passed around in registers, stored immediately in vectors and matrices, etc.) These data might be characters (easily up to 32-bits of character data, more bits could represent common fonts, styles, etc.) 32-bit color, the same.

Dynamic languages like Smalltalk become significantly more efficient for number crunching (engineering, finance) because they're no longer using memory allocated numbers for floats and large integers.

So much for science...

Back to politics: the Bush administration wishes to replace this time-tested, widely respected process of evaluating scientific research and proposals for funding of research, with a process that places control in the hands, not of experts in the field in which the research is pursued, but of the White House and the OMB...

The administration says that policy should be based on scientific truths so well-established as to be beyond question. But there is no scientific truth beyond question; that is the very nature of science. This is not about basing policy on better science; this is about interfering with the decision of what constitutes good science... to the financial advantage of various industries that contribute to Bush's campaign, and to the political advantage of the Bush administration as they throw a bone to their religious "conservative" (read: radical fundamentalist) supporters.

The war in... is but a symptom of a far deeper malady within the American spirit

...communism is a judgment against our failure to make democracy real and follow through on the revolutions that we initiated

Ian Bicking has a commemoration of MLK, Jr. and he chose to quote from one of his last speeches.

What a beautiful choice. The mainstream media tends to choose quotes and clips from many years earlier when his focus was still primarily civil rights for African Americans.

King was beginning to shine the Light on society's even greater ills. These are neither as obvious nor as easy to resolve as were civil rights. And so they're still with us and he was assassinated not long after this speech.

"Why are you speaking about the war, Dr. King? Why are you joining the voices of dissent?" "Peace and civil rights don't mix," they say. "Aren't you hurting the cause of your people?" they ask. And when I hear them, though I often understand the source of their concern, I am nevertheless greatly saddened, for such questions mean that the inquirers have not really known me, my commitment, or my calling. Indeed, their questions suggest that they do not know the world in which they live. In the light of such tragic misunderstanding, I deem it of signal importance to try to state clearly, and I trust concisely, why I believe that the path from Dexter Avenue Baptist Church -- the church in Montgomery, Alabama, where I began my pastorate -- leads clearly to this sanctuary tonight.

OO Contracts

This financial contracts specification would make an interesting Smalltalk library. The original claim was that a functional language should be used because combinators, unlike objects, can be easily composed.

But combinators are as easy to build with objects as they are with functions.

Riding Giants

"Dogtown" was such a good documentary, I'm a definite for "Riding Giants" when it arrives in a theater near me.

I was skateboarding in the same era as the Z-Boys. Unfortunately I was in Ohio. Our biggest stunt was to make it down a long hill on a sidewalk that ended in small gravel, or grass if you could steer well enough to get to the grass.

More than a few times we'd recover from rides by plucking out very small rocks deeply embedded in the calluses of the palms of our hands. These were the bonding moments for young adolescents, and so I guess that's why they're the things I remember.

On the Health of "Offshoring"

Google is opening a Bangalore engineering office...

The office will operate in an identical manner to our other engineering groups, with the same scope of work, hiring standards and unique Google culture.

I wonder if they'll offer the same health and insurance benefits.

GarageBand and the Garage Band Method

GarageBand from Apple gets a references from Jon Udell on turning "consumers" into producers.... this might be a good time to plug "The Garage Band Method". Unfortunately the book appears to be unavailable from Amazon as well as Powells.

I'll have to cherish my copy even more. The Garage Band Method is a no nonsense approach to making *interesting* music quickly while learning piano, guitar, or sax. (Moreover, all three are covered in the one book.) The method is not "learn on your own". You're expected to have a teacher, but the book tells you what to look for in a teacher and how to work with one. The other key point is to get with other people who want to play and may also be beginners in their instrument.

Toysight, Eye Toy

The Toysight for iSight for the Mac seems kind of like the PS2 Eye Toy, but I don't know the details for either. My 11 year old got the Eye Toy for Christmas.

It's a lot of fun and even more potential than realized so far.

An Object Lession: XPathDocument2, Is the 2 Noise or Information?

Objects are for communication. A key to good design is to choose good names. This Longhorn class should be renamed now: System.Xml.XPathDocument2 to set an example.

What does the name XPathDocument2 tell me, the reader? Well, it basically tells me there is an XPathDocument (or maybe an XPathDocument1 and perhaps an XPathDocument3).

It also suggest that the 2 class was developed some time *after* the first one. It might tell me not to use the first one. I don't know. The 2 is noise that wants to be information.

A better name would tell me that XPathDocument2 can be disconnected from the source, not tied to a DOM, and used for read/write, and provides change notifications. The numeral 2 cannot tell me that. Neither does it tell me that the original DOM-bound XPathDocument still may be a good choice in some scenarios.

I don't know exactly what the better name is, but it is out there in the conversations that led to its creation, and (unfortunately) in the conversations that will take place among its consumers. Maybe ReadWriteXPathDocument is a good choice, but only real conversations (as opposed to the ones going on in my head right now) will tell.

This post has nothing to do with XML and XPath or Longhorn. It has everything to do with why objects are important and will remain useful for their original purpose: code organization. Good names are a key to good organization.

Well, this post *does* have something to do with Longhorn... the developers at Microsoft are creating now the language many developers will use for probably a decade. Please take the time to provide meaningful names. Two years before the release date is *not* the time to affix numerals to the end of classes for relatives that provide alternate functionality!

Blitz is an open source JavaSpaces implementation

Freshmeat...

Blitz is an open source JavaSpaces implementation designed to ease development and deployment of JavaSpaces technology. It is Jini 2.0 enabled, and uses established VM principles. It also implements smart indexing, tuneable persistence, and active/passive lease cleanup. It is designed with experimentation and expansion in mind.

Python and groupby

For Python's list comprehensions comes groupby...

Guido inspired SQL-like GROUPBY class that also encapsulates the logic in a Unix-like "sort | uniq".

class groupby(dict):
    def __init__(self, seq, key=lambda x:x):
        for value in seq:
            k = key(value)
            self.setdefault(k, []).append(value)
    __iter__ = dict.iteritems

# -------------------------- Examples -----------------------------------

>>> letters = 'abracadabra'
>>> [g for k, g in groupby(letters)]                # grouped
[['a', 'a', 'a', 'a', 'a'], ['r', 'r'], ['b', 'b'], ['c'], ['d']]
>>> [k for k, g in groupby(letters)]                # uniq
['a', 'r', 'b', 'c', 'd']
>>> [(k, len(g)) for k, g in groupby(letters)]      # uniq -c
[('a', 5), ('r', 2), ('b', 2), ('c', 1), ('d', 1)]
>>> [k for k, g in groupby(letters) if len(g) > 1]  # uniq -d
['a', 'r', 'b']

RDBMS: Inherently Loosely Coupled?

Carlos Perez writes...

In general, a relational database provides a fixed, queried, self-describing, lazy evaluated system that is inherently loosely coupled. It's surprising that its perceived to be more tightly coupled than a component based system.

I agree 100 percent with this statement. I think the current popular SQL databases are archaic for a number of reasons, yet they are a mechanism for loose coupling. What's more, they are widely understood by developers. An unfortunate aspect of relational databases is there is typically so much ceremony involved in putting one into production. There is a time and place for ceremony, and likewise for agility. We should be practicing agility with databases, and vendors should be listening for ways to improve them.

AI Never Really Worked At All?

AI never really worked at all, says Tim Bray in his latest installment in an interesting technology winners and losers assessment. I'm enjoying this series so far, but the AI line item struck me funny.

First of all, looking at this list, some of these technologies are very specific, such as Unix/C and SQL/RDBMS. But "AI" is a broad, 40-year-and-continuing, label for several families of hard computational problems. How can these be compared in terms of "hard to build"?

Digging into various AI categories, clearly some have been very successful, such as expert systems, and various forms of search and planning. Other categories of AI research have been "successful" in terms of learning exactly what the problems are in that category. Not all successful research results in working products.

Test Driven Development

Jarno Virtanen writes...

My personal gripe about test-driven development is the complete lack of the so-called white box testing methods. In white box testing, you use the source code in order to create test cases that cover the whole functionality of the testable unit. I would go so far as to claim that the trickiest bugs are in the edge- and corner-cases of the code which you can reveal with white box testing. But, of course, tests in advance of coding are better than no tests at all. (I am too often guilty of writing no tests at all.)

I think it is important to emphasize that test-driven design does not imply that the tests you write toward a design are not necessarily the only tests you should write to get to a production quality release. The TDD tests should be just enough to get to a satisfactory design. Depending on the system being constructed, you should still consider acceptance tests, including performance tests and more complete suites for functional coverage than were needed just for the design process.

Can you learn YAML in five minutes?

Can you learn YAML in five minutes? I did.

Then I read through a lot of the YAML articles, web, and spec this weekend. I was kind of aware of YAML some time ago, but this was my first in-depth look.

YAML appears to be just popular enough to have a future, but of course XML is a juggernaut.

I like that YAML maps to simple data structures found in essentially all modern languages. There is no separate "DOM". YAML also appears to have a good set of base data types with extensibility. On top of all this, the syntax is significantly simpler and more human readable, which also means it can be hand edited more readily.

YAML might make a good syntax for a build system, as opposed to ANT. It would also be a good candidate for "preference" files and other text documents that people read and write by hand, but also process with applications.

There is a simple mapping between YAML and a subset of XML. YAML is a good deal more compact than XML, although I wonder of that would significantly reduce the benefits of HTTP compression on large files. Depends on the markup to content ratio I guess.

Register Allocation

Michael Lucas-Smith calculates the methods that can use register allocation effectively...

I put in italics the methods that would require a spill object. Only 0.37% of methods in the system will spill!. Clearly this indicates that the technique will work even on a system with 8 registers like an x86.

I think that is to some degree optimistic since no registers will be left on the x86 for intermediates.

Data+Algorithms

Keith Ray responds to my recent item on coordination via databases...

As long as databases only store data, and don't store whole objects (data+algorithms), they will always risk disassociating meaning and values.

I understand there is this risk. But the scenario we're comparing to is the distributed messaging paradigm where messages have no algorithms associated and the various end points use different languages. People have decided they don't want the "whole objects" scenario.

I think there is a tremendous benfit to whole objects in either scenario, if good choices are made about the language. The majority has not gone along with that. Ever. Back in the days when CORBA and OODBs were hot topics at OOPSLA, there was a clear message being presented from an aggregated look at the experience reports. Namely, distributed Smalltalk and the Gemstone Smalltalk OODB were relatively easy wins, while distributed CORBA and other OODBs would work OK but were a pain in the ass.

Coordination via Databases

Jay Han responds to my item on using databases for coordination...

We know that databases provide concurrency control and transaction management. These features let applications share data -- you can call this coordination at low level. But what about coordination at high level? How can they exchange semantics of data? e.g. "9999-12-31 in date field means now? never? forever in the future?" or "20 point means 20 basis point." (Schemas and constraints can check syntax of data but not the "validity" of data.) Because I don't see how databases provide meaningful (hence ad-hoc) coordination, I don't quite understand Patrick's last sentence above especially the second half.

Databases have no advantage nor disadvantage in this respect (the "meaning" of data) to any other coordination mechanism.

Edd Dumbill on XML and Databases

Edd Dumbill makes an interesting observation about XML and databases based on submissions to XML Europe 2004...

Databases. Though there's a reasonable amount of interest in the W3C XML Query language, there's not much to say about XML and databases. It doesn't seem to me that the integration of XML with relational databases has taken off in the way we once thought it might. Is this just a temporary lull in the convergence activity? Probably.

BigCo Dynamics

Jon delivers a great sermon on dynamic languages and wraps it up with an observation...

Somebody asked me yesterday why platform vendors like Microsoft and Sun are never at the forefront of dynamic-language innovation. I don't know why that's so, but it does seem to be true.

Sun has been at the forefront in the labs, e.g. the Self language was partially a Sun effort. The Java HotSpot VM is essentially a productization of many dynamic language implementation innovations.

I think an answer to the question above has more to do with the psychology and sociology of programming than it does with anything else. The big companies are almost by definition the ones whose value chain ends with giving the majority what they want. What they want is not always what works good in laboratories, nor is it necessarily what is "best for them".

Looking at language evolution in the long run, especially as plotted against a graph of Moore's Law, clearly the trend is to become more dynamic. A good indicator is a major industry journalist writing about such things.

I don't know what the name will be for the ultimate dynamic language, but it is almost certain to have {curly braces} in the syntax and come from Microsoft. Well, sociology may determine the syntax to have <angle brackets>, but it almost certainly will come from Microsoft. 8^)

We *Have* Magic

Tim Bray on the tripod of objects, messages, and tables...

...for me, this essay brought into focus the fact that anyone who isn’t comfortable with object-oriented design, and with relational data modeling, and with wrangling XML messages; anyone not comfortable with all three, I say, just isn’t fully trained.

Well said, for developing today. But what about tomorrow? When he writes...

Lots of architects have learned, painfully, that you usually can’t magick relational rows away behind object/class abstractions. The right way to think about a database is as a set of normalized tables that are designed to be addressed with SQL strings.

...I can only suggest that we need to upgrade our languages to provide more magic.

Lisp, Smalltalk, and Python come to mind. These are the magic we have for now, but there's more to come if we stay the course. Our databases are still based on early 1980's implementation ideas. Let's keep the good concepts and upgrade the rest. Our messages are struggling with nascent infrastructure and representation issues.

Our languages (i.e. "objects"), our databases, and our messaging are all struggling to become more dynamic. As they do, they will also become more unified, or at least "unify-able".

Will it be Cow Paths or Paved Roads?

Jon Udell... For me every document is a database, and every database is an assembly of documents.

Fair enough, I like the convergence too. Hopefully our tools will provide some examples and guidance toward good structures for documents for various scenarios. There are best practices in the decision support arena that are not always employed even by us professional nerds. New query tools and calculators would do well to steer the evolution toward the paved roads or we may be processing XML on the cow paths for longer than necessary.

Keith Ray writes about multiple return values in Python and Smalltalk...

# Python example...
beforeNotAfter, afterNotBefore = foo.findNonOverlappedElements( before, after )
# do stuff using beforeNotAfter
# do stuff using afterNotBefore

"Smalltalk example..."
foo findNonOverlappedElementsOf: before and: after 
    doing: [ :beforeNotAfter :afterNotBefore |
                "do stuff to beforeNotAfter".
                "do stuff to afterNotBefore" ]

The other aspect of the Smalltalk solution here is "tell, don't ask". We tell 'foo' to find the non-overlapped elements, and we tell foo what to do with the resulting lists of elements (execute our block). This forces a certain cohesion to our code. The Python example is violating the spirit of the Law of Demeter, if not the actual Law (1) because we ask and then work the results of what we asked.

But I don't see how the Law of Demeter is involved in this. Neither the Python nor the Smalltalk examples divulge anything about the implementation of foo. There are no undue dependencies.

In fact this is one use of a Block in Smalltalk that obfuscates the code. Python's simple syntax for returning tuples into a multiple assignment is the simplest mechanism for this scenario.

If only you believed in miracles, we'd get by

Americans increasingly [tell] pollsters that they believe in prayer and miracles, while only 28 percent say they believe in evolution.

Isn't evolution a miracle?

Snow, Snow, and More... Freezing Rain???

New Year's Day we had 5" of snow and great sledding. The only good sledding snow in the last four years down in the valley.

Last night the snow was even better. But I had an all day meeting in the office. Checking in with my family throughout the day, my boys spent all day outside. The sledding was great. For them. 8^(

Before I even left the office, the landscape had been receiving a coat of freezing rain. The fun would be over. But wait...

Up ahead in the driveway there is a snow ramp and kids are falling down hard in the road. The ice coating is almost perfect and the roads are empty.

Several of us make the full trek up the road. Again and again. There was plenty of sledding left this night.

Tomorrow the muscles will ache and I have Part 2 of the two day meeting. 8 am.

More Songs about Objects and Data (and Pattern Matching)

On DevHawk the discussion continues about objects and data. Herewith are some thoughts...

Objects are first and foremost for code organization. Computers could do their work just fine, thank you, without objects. And so if the "data" or "messages" or "documents" are not going to map to objects, then will we be better off?

I think the salient, yet somewhat conflicting, points include these:

• Objects are for code organization
• Commands, messages, and documents can be organized as objects
• Different languages have widely different implementations of "objects"
• Interoperability should not assume anything about a specific object model
• Still, mapping data exchanges into the objects of your language are desirable
• Therefore, attempt to model interoperable data exchanges as objects to the extent it will help you with your code organization

In addition to objects, the other code organization mechanism that will lessen your burden is pattern matching. The better pattern matching is integrated into your language, the less code you will have to write when passing semi-structured data around.

Erlang is a good example of a language that does not have "objects" per se, but Erlanf does have a very convenient pattern patching syntax for interprocess messages. Languages that have both objects and pattern matching are even better off.

Microsoft Business Framework

Update: What happened to IBM's San Francisco framework for Java? Was it too ambitious? Will MBF be more practical? After a half decade of EJB and J2EE, is there anything comparable? In any language? See MicrosoftBusinessFramework.

From the PDC, maybe the best presentation I have view thus far... the Microsoft Business Framework.

Continuations: Python, Haystack

From the Chandler email list, some references to the use of continuations in the Haystack system and the Python language...

In September, Hamish Harvey described some of the neat things about Haystack, including UI continuations. (Harvey's message was posted Fri, 26 Sep 2003; there's an article describing Haystack's continuations at http://haystack.lcs.mit.edu/papers/uist2003-uicont.pdf.)

I just wanted to add a reference to a tutorial article on continuations using Python: "Continuations Made Simple and Illustrated" by Denys Duchier at http://www.ps.uni-sb.de/~duchier/python/continuations.html.

I also want to second Harvey's enthusiasm for the idea. When the design work on CPIA gets into high gear, I strongly recommend reviewing this technique.

Regards,
Don Dwiggins

Databases for coordination

Jim writes...

The plain truth is, not that many projects really need complex distribution mechanisms. Most apps look a lot like this:

• Read data from a store (db, files, etc)
• Modify data
• Dump data back to the store

I think we have gotten too far away from this wisdom. The typical database has distribution and coordination built in. That means for the most part you get to write your application in isolation. The database also to serves to coordinate the application's own actions.

We need to get back to the future and look at how to simplify coordination and not just message passing. Databases are a coordination mechanism for one or more applications, and one mechanism is better than two.

Small pieces loosely joined, but how?

Just some thoughts on my current favorite hammer and nail, the XML-based nested tuple space and some coordination systems I'd like to know more about...

It seems to [Tim O'Reilly] that the original Unix/Linux architecture, and the architecture of the internet, are based on a model of "small pieces loosely joined" (to quote David Weinberger). Web services can also operate on this model. However, there are alternate visions, including .Net and J2EE, in which there is a quest for "one ring to bind them all."

Tim also wishes that Nat Friedman (of Novell/Ximian) would finish up Dashboard for Linux

I'd like to learn more about Dashboard in 2004. In fact there are three "connecting" systems I'm wondering about...

• Dashboard
• Central
• Groove

Central and Groove do many things, but to boil them down to their essence, at least the parts that interest me most right now, I would say the following. Central provides awareness of selected information among a set of Internet-enabled applications all under my control. Groove provides awareness of distributed actions among a set of shared Internet-enabled applications under a small community's control.

Dashboard on the other hand peeks into the more or less internal information of less deliverately cooperative applications.

Kind of the downside of each of these, from my cursory understanding, is that these underlying connecting mechanisms are each tied to larger frameworks. Apps in Central almost have to be Flash from top to bottom as far as I can tell. Apps in Groove have to be Windows based or at least use SOAP to get to Groove on Windows.

Apps in Dashboard... there are no apps "in" Dashboard from what I can tell. But Dashboard has to be able to peek into the apps your interested in, and they seem to have to run on Linux.

Would each of these capabilities benefit from more loosely coupled "connective tissue"?

Central-like connections could be made by having any application's selections be published to a local blackboard (aka nested tuple space).

Groove-like connections could be made by having the distributed applications communicate by implementing persistent shared spaces and/or queues (aka nested tuple space).

And Dashboard... perhaps Dashboard-like connections would be enabled to work with any app that uses a local searchable tree as its working memory (aka nested tuple space).

No, strike that, the big deal is it's DATABASED

From Rands in Repose on hammers, nails, and...

The obvious and simple fact is that, yes, there is quite a bit of functional overlap between spreadsheets and databases. They both, basically, are representations of tables of data and most folks want to perform interesting operations against those tables. Databases are more structured, spreadsheets appear more flexible and easier to use...

Task mind meld aside, what is more relevant about the application is that it's web-based. No, strike that, the big deal is it's DATABASED. Ahhhhhhhh. Sure.

New Year's Eve at OMSI

I spent the day yesterday at the Oregon Museum of Science and Industry with my wife, youngest son (11), and his friend across the street.

The whole place is "hands on", but the physics and chemistry labs are the best.

Now with Posi-Blog Action

2004 will be the year of the Posi-Blog for me. Every post will be about something I think is good, or I won't post. It's too easy for me to slip into disconnected curmudgeon mode.

A have several drafts that have been sitting around for a while so my first excercise will be enlightening for me, to find out which ones are Posi-Action and which ones can be edited to be so. And which ones can't --- you'll never know.

Happy New Year!

One last post for 2003... Happy New Year!

Signing off for 2003. One last post...

Dan writes about good news for organic beef farmers. "They're not allowed to feed animal remains to their cows."

Here in Portland, Oregon we get Painted Hills beef.

Thinking Good Thoughts

Another break in the holiday festivities to note that Don Box is asking us to think good thoughts...

Think HyperCard. Think VB 1.0. Think classic ASP.

I think it's going to matter big time going forward as the industry wakes up from its C++/Java-induced haze and starts thinking about making computers programmable again.

A break in holiday festivities for a technicalogy wish worth wishing...

Via Wired, Rael Dornfest, author of Google Hacks and the mobilewhack weblog, with a wish worth wishing for 2004...

"I'd like to see consumer mobile devices -- palmtops, hiptops and handsets --scriptable. It was scripting that drove the Web, taking it from a static online catalog of content to an operating system. Gaining simpler programmatic access to the contacts, calendars and other assorted user data; Bluetooth; messaging; image capture and manipulation on the phone will open up the mobile to the people prototyping the next generation of applications."

Thoughts heading into 2004

Whatever else Jesus was, he was almost certainly a radical.

He challenged the mainstream religious authority.
He challenged the mainstream governing authority.

He challenged the mainstream attitudes toward those who are not in the mainstream.

My wish for us all is peace and true prosperity.
May we all take one radical step in that direction in 2004 and we'll be more than a billion steps closer.

REST and Linda for Distributed Coordination: Elaboration vs. Layering

Mark Baker makes an interesting distinction in my wRESTling with tuple spaces...

Patrick seems stuck with how to reconcile his position that generic abstractions are a good thing, but that systems should be built independent of the protocol. Note to Patrick; this is all well and good for transport protocols, but application protocols define the abstraction; for them, protocol independence requires that you disregard that abstraction.

This distinction of transport protocols vs. application protocols is exactly what I am wondering about REST. As I read the definition of REST, the architectural style being described is for a transport protocol rather than an application protocol. Not much is said really about the behaviors of the client or the server. Even when you bring HTTP per se into the definition of REST, Fielding makes a somewhat confusing statement about transport protocols...

HTTP is not designed to be a transport protocol. It is a transfer protocol in which the messages reflect the semantics of the Web architecture by performing actions on resources through the transfer and manipulation of representations of those resources.

Is it a transport protocol or not? Let's ignore that and pursue the part about "performing actions on resources" because that *does* seem to be about an application protocol. Fielding continues...

It is possible to achieve a wide range of functionality using this very simple interface, but following the interface is required in order for HTTP semantics to remain visible to intermediaries.

And so this is where I begin to have problems with REST, as I read it, as an application protocol for distributed system coordination. The problem is not that it is inappropriate, but rather that it is too vague.

I don't mean "vague" in a derogatory manner. What I mean is exactly what Fielding writes, i.e. it is possible to implement a wide range of functionality using this very simple interface.

How is this different from the tuple space interface? I have written, and many others better than I have written, that it is possible to implement a wide range of features using the very simple tuple space interface.

The difference is this: the HTTP interface is vague and the Linda interface is specific. Linda has precise, simple semantics. The possible range of behaviors exhibited in Linda-based systems benefit from being layered on *top* of those precise, simple semantics.

HTTP, on the other hand, has to be *elaborated* into something more specific in order to have a useful meaning as an application protocol. WebDAV is an example of such an elaboration.

Every web site that implements custom behavior using forms with GET or POST is an example of the open ended nature of HTTP per se. The architectural style of REST supports the HTTP transport protocol underlying these forms moving across the web, but the application protocol, that is, the behavior of the forms on the client and especially on the server is defined (at least in code) by each specific instance.

Distributed systems wishing to use HTTP, or more generally REST, to perform coordinated work will therefore require some more specifically defined application interface than that provided by REST, or that provided by HTTP.

WebDAV is one option as stated already, and it is proven to be viable in some specific cases. I don't believe the full range of systems that can be usefully built with WebDAV has been exhausted. By the same token, neither do I see a lot of evidence of that range being nearly as broad as that of Linda tuple spaces.

Vanessa Williams provides an elaboration of HTTP for a tuple space application protocol. As I understand REST this should therefore provide the application protocol of a tuple space on the architectural style of REST using the HTTP transport/application protocol mix. In this case the advantage of using REST and HTTP is supposed to be found in the hardware and software that would already be in place between the client and the server.

I think and hope this is fairly accurate. I am eager to be clued in further by Mark and others. I am still unsure that this advantage is significant over a less pure elaboration of HTTP, as in XML-RPC or the arguably more RESTian SOAP. I think there is a lot to be said for something else altogether as a transport for tuple spaces, in particular Jabber or perhaps Spread. The bottom line is the usage models of distributed systems coordination would benefit from a well defined, simple, axiomatic application protocol, but the best transport protocols *will* have to evolve just because the usage models themselves will have to evolve. For all but a handful of services (e.g. Google), they just may act nothing like today's web.

What Kinds of Queries?

Queries: reportedly, Adam Bosworth said at the XML conference...

"I don't know how we're ever going to truly optimize these queries."

Before we worry about that, though, we should consider how the average user is going to form those queries. The last twenty years have made great strides in forming SQL queries for numerical data analysis. The models underlying those queries fortunately support both the relatively non-technical users who wish to form them as well as the highly technical adminstrators who wish to optimize them.

What kinds of queries do these users wish to form? What are we trying to optimize? What relationships will these have with current typical business analysis?

Will we wait twenty years for the evolution to settle into a widely used winner?

These Changeable Things Avalon Has

Update: I almost forgot Doug Lea's thoughtful design and implementation of a collection package, which predates the java.util collections. That specific link will take you directly to his wisdom on designing type checked immutability. (Which makes me wonder if Microsoft has hired him yet too?!)

Greg Schecter writes about the Changeable class. Not knowing much about the implementation this seems like a reasonable concept. (EmeddedChangeableReader sounds like it might be a bit over the top, but I don't know.)

Compare this to Object.freeze in Ruby. The Ruby implementation appears to be simpler, but the overall lesson I think is this: what we have in the Changeable class is more evidence of Java-like languages struggling with their underlying dynamic tendencies.

Inside every statically type checked language there is a dynamic language struggling to get out.

The simple statically checked approach is to create a class without mutation, e.g. Brush, and then add mutation in the subclass, e.g. MutableBrush. Another approach is to simply throw an exception in a method that would otherwise cause a mutation without any system-wide designation in the code. See the Java Collections API for example. In particular the unmodifiableCollection static method and its siblings for List, Set, etc.

Then again there is the option of just allowing mutation for all instances even when it's not expected. One of the first actions I took in my first Smalltalk system (the Tek 4404) was to change Black to White. Oops. Fortunately although Smalltalk has a persistent image, it's not difficult to back out of the change.

Wanted: Comparison of XML-based Query and Table-based SQL

Jon Udell writes about XML for the Rest of Us wherein Adam Bosworth writes...

"The relational database is designed to serve up rows and columns," said BEA's Adam Bosworth in his keynote talk. "But our model of the world is documents. It's 'Tell me everything I want to know about this person or this clinical trial.' And those things are not flat, they're complex.

I agree with the idea of semi-structured searching and manipulations. But I don't expect anyone would deny we still need traditional (e.g. business) calculations and those will be well served by more structure than less. I'd like to see a more direct XML Query and SQL comparison. As it stands, I'm being led to believe I'll should use XML Query (i.e. something with XPath-like stuff in it) for doing non-numerical property-tree searches over data that has been or could be expressed in an XML text; but I should use SQL for doing relationally flat table-column calculations over data that has been entered into a relational database.

There is a merger out there somewhere that I'm not seeing, or maybe just an example of what such a merger might look like.

Useful InfoPath Session at PDC

Watching this PDC session on InfoPath is worth the time. Better than previous demonstrations and white papers I've read, especially if you looking for somewhat technical information for InfoPath in the context of a modern Microsoft-based IT shop.

My First Computer: the IBM 5100

A collection of first computer stories relayed by Dan Gillmor.

My first computer was an IBM 5100. Flip the switch up, you're running BASIC. Flip the switch down, you're running APL.

I'd hardly call it a portable computer though. The box was not even as mobile as those "luggable" computers like the Kay Pro from a few years later.

What's On Public Radio

http://www.publicradiofan.com has a listing of, well, audio links to what's on public radio right now.

Teddy Bear's Picnic on DVD in February

Great news from Harry Shearer...

Now, some real Xmas cheer: many of you have been asking when you can buy my film "Teddy Bears' Picnic" on DVD. The answer, apparently, is this February. Be warned, you may have some trouble finding it, since, as the result of a monthlong battle lost this week, the cover art on the box will more closely resemble Porky's 4.

This is merely proof that, while it's now easier to make a movie outside the Hollywood mainstream, to get it seen one still has to run the usual gauntlet of Visigoths. These particular barbarians, in a nice twist, are Canadian, and they appear to believe that the target audience for this movie is teenage boys. I'd share the art with you, but I promised a low-impact blast, so take my word for it--the bikini-clad blonde who dominates the cover art appears nowhere in the film.

If you'd like to send a greeting to the primate who insists on this approach, his address is: rmanis@thinkfilmcompany.com. Yes, thinkfilm. When your business is irony, even that comes back to haunt you.

More on tuple spaces

Phil Windley expresses an interest in tuple spaces and points to an item that spells out in some detail why I am still unsure of what ReST really means at a deep semantic, implementation, and performance level. I wrote about this in a couple of (admittedly superficial and this is more of the same) items in April 2003.

Whether using "pure" HTTP, or SOAP, XML-RPC, or Jabber, (or SOAP over Jabber, XML-RPC over SMTP, SOAP over BEEP over HTTP, or... even RSS, RSS-Data, and polling or cloud APIs as part of the transport) the key is to distinguish transports from actions.

A tuple space (or XML space, where the tuples are represented as XML text) for distributed, asynchronous computing can be (and should be) implemented on top of multiple transports. The important aspect for applications are the actions (tuple space semantics). The implementation and performance of the transports can (and should) evolve independent of the simple semantics of tuple space actions.

Above the simple actions, independent of the transport implementations, the simple tuple space actions can be combined into various kinds of databases, queues, exchanges and marketplaces that we really want to focus on evolving in the first place.

Where's IBM's Linux Desktop?

From consultingtimes...

Have you noticed that the most likely source of technology expertise, IBM has simply refused to provide a Linux Desktop? With all of their Lotus applications neatly running on their own UNIX products, they won't let you have them on Linux. Instead, they suggest you purchase Windows XP Professional.

The internal strife existing at IBM over producing a Linux desktop has the potential to hurt IBM's business model. A powerful internal software organization wants to grab server market share from Microsoft without disturbing Microsoft's desktop. Anyone inside IBM that mentions a Linux desktop has the potential for losing their job. While few people at IBM know what exists outside the company, the powerful software group may have top executives walking the same plank as the rest of us if Microsoft remains the only Intel desktop platform.

While the software people at IBM have their heels dug in, they may find out that their Web Services strategy based on Java has no place to go. Sun may not have put IBM in "Check," as Scott McNealy has put it, but Sun's Java Desktop System definitely places IBM in a Microsoft dilemma. IBM will have to decide if they'll continue to provide Java Web Services and find a desktop to accommodate it, or watch their Java developers transfer their code to Microsoft's Java Language Conversion Assistant.

I used Don Box

It's true, and in the morning I felt kind of bad. But it was for the good. I've done it in the past, but this time it was coldly calculated. Please read on...

Actually when it comes right down to it, I don't care what happens to Visual Studio. I also don't care what happens to Emacs. VS will continue to improve, but moreover it will continue to be hugely popular no matter what. Emacs has the audience it does, and will probably not improve beyond its current state, because it is in itself an axiom. Probably it's appeal and usage characterisitics will remain about where they've been for the last twenty years.

So why did I use Don Box?

More people than I could ever hope to draw on my own have had a chance at least to read the story about Emacs and the secretaries in the 1970s. Was this story really intended to promote Emacs and to benefit VS?

Not really. I saw the opening and ran for it. Here's the message: the Longhorn preview takes over five gigabytes to install. How much of that is for the typical user?

Very good arguments could be made that all of it will eventually trickle down to the non-technical user. There is no way I could or would argue against that.

But in the 1970s a few typical secretaries had a simple tool for helping themselves, the same tool most programmers have intimidated each other from using even as an influence. In the 1980s typical non-technical users were building multimedia applications using Hypercard. Emacs and Hypercard together take a miniscule fraction of the installation space and still a small fraction of the intellectual power required for computing with XML, DOMs, XAML, and WS-xxx. Are the secretaries going to be doing this in Info Path?

In all of these five plus gigabytes of impending computations, what are we doing for the typical user or even the non-technical MBA? Maybe this was an inappropriate way to use blogspace.

On just one difference between Emacs and Visual Studio Dot Net (besides the length of the name)

Don Box wishes he could be disagreeing with James Robertson's observations on Visual Studio Dot Net, but apparently can't. Here's what makes the difference.

I would consider any advice Don brings from Emacs an improvement for VSDN. But the fundamental difference is also the fundamental failing of not just VSDN but practically every IDE I have seen including the vaunted IDE for Java using Eclipse.

The irony is the term "Visual" because VSDN is a visual nightmare. The beauty of *most* uses of Emacs (remember Emacs is a flexible tool-building platform like Eclipse, except simpler and more expressive) is the visual simplicity and just-in-time functionality. Where VSDN gives you panels, panels, everywhere panels of things to do and be concerned about, Emacs gives you an editing buffer. Everything else is a keystroke or menu click away. All the power of VSDN and more is waiting for your call to action, but visually you are "just editing".

Let's recall this story from the 1970s about secretaries (as they were called then) using Emacs, essentially the same Emacs you're using today. (You *are* using Emacs, aren't you? For shame!)

...programming new editing commands was so convenient that even the secretaries in his office started learning how to use it. They used a manual someone had written which showed how to extend Emacs, but didn't say it was a programming. So the secretaries, who believed they couldn't do programming, weren't scared off. They read the manual, discovered they could do useful things and they learned to program.

Would we ever read a similar story about VSDN? For want! Not by the 2070s.

One ring to bind them all. Emacs Semper Virens.

Game Programming with Python and PyUI

I just picked up Game Programming with Python by Sean Riley. Sean is also the author of the PyUI user interface framework. He explains and uses PyUI in the book as well.

Just thumbing through the book, I would give it a thumbs up. It looks good and I hope it pans out. No pun intended.

Are we moral relativists?

Before we get too caught up in the capture of a dictator we (the U.S.) supported for decades, let's take a tally of the others still in our (the U.S.) favor.

In two fine speeches recently, President Bush made it clear that autocratic regimes in the Middle East, including U.S. allies Egypt and Saudi Arabia, need internal reforms to stop churning out terrorists. Somehow, though, he forgot to mention Azerbaijan and Uzbekistan.

If the president's ratings go up based on the recent capture of a former ally now out of favor, should that be considered a mandate to terminate relations with these others? Or does the administration itself suffer from "moral relativism"?

Peer-to-Peer Sockets

From Brad Neuberg, OnJava, an insightful abstraction of sockets on p2p on sockets...

P2P Sockets effectively hides JXTA by creating a thin illusion that the peer-to-peer network is actually a standard TCP/IP network. If peers wish to become servers, they simply create a P2P server socket with the domain name they want, and the port other peers should use to contact them. P2P clients open socket connections to hosts that are running services on given ports. Hosts can be resolved either by domain name, such as www.nike.laborpolicy, or by IP address, such as 44.22.33.22. Behind the scenes, these resolve to JXTA primitives, rather than being resolved through DNS or TCP/IP....

The P2P Sockets project already includes a large amount of software ported to use the peer-to-peer network, including a web server (Jetty) that can receive requests and serve content over the peer-to-peer network; a servlet and JSP engine (Jetty and Jasper) that allows existing servlets and JSPs to serve P2P clients; an XML-RPC client and server (Apache XML-RPC) for accessing and exposing P2P XML-RPC endpoints; an HTTP/1.1 client (Apache Commons HTTP-Client) that can access P2P web servers; a gateway (Smart Cache) to make it possible for existing browsers to access P2P web sites; and a WikiWiki (JSPWiki) that can be used to host WikiWikis on your local machine that other peers can access and edit through the P2P network. Even better, all of this software works and looks exactly as it did before being ported. The P2P Sockets abstraction is so strong that porting each of these pieces of software took as little as 30 minutes...

The SOA Antidote

Russell Levine writes in the Business Integration Journal about the Myth of the Disappearing Interfaces. If you work in IT, have been involved in some EAI projects, and are a relatively critical thinker, then there probably is not a lot of new information for you. However the piece serves nicely as an antidote to the run-of-the-mill "Service Oriented Architecture", well, pablum.

More good information can be found at Doug Barry's site. Almost too much at once without a trail guide. Better than run-of-the-mill, without a doubt.

From Russell:

• n^2 vs. n comparisons should be considered harmful.
• A clue that there might be a problem with this argument is that these pictures often have applications with names such as A, B, and C.
• Applications along a value chain often have many different connections.
• Ultimately you need to understand data flows to assess the complexity of the integration challenge.
• Data mapping requires intimate knowledge of the data and how it's used.
• You must understand every data relationship. That hard work is unavoidable.
• Any benefit must be balanced against the effort of creating an intermediate, or "canonical", model.
• Portfolios with the critical mass to justify such efforts don't emerge overnight.
• Focus on business benefits.
• Estimate costs with and without the integration technology.
• Be conservative!

There you go.

So what's all this got to do with XML?

Jon Udell...

So what's all this got to do with XML? If you buy the notion that we are projecting ourselves into networked information systems, then we can't only focus on how processes and data interact in these increasingly XML-based systems. The quality and transparency of our direct interaction with XML processes and data -- and with one another as mediated by those processes and data -- has to be a central concern too.

When I think of XML, two things come to mind. First, I think of the movie Brazil, because XML is still this grab bag of stuff that happens to share one thing in common, angle brackets.

Second, I think of David Letterman's bit he calls "Is this anything?"* --- We expect XML to be something, anything, more than a grab bag of stuff that shares something in common beyond angle brackets.

A third thing comes to mind: the black knight in the Holy Grail, after all his limbs have been cut off. XML is utterly helpless in and of itself. It's everything *around* XML that has value, most of which are hindered by XML per se, not aided.

---

*(David Letterman's latest zany recurring bit is something he calls "Is this anything?" It consists of a setup of the bit followed by the pulling open of a curtain where a performer or, sometimes, a nonperformer, is doing something that may or may not be worth seeing or even worth "anything." David and his band-leader cohort, Paul Schaeffer, then discuss what they've just seen and decide whether it amounts to "anything," They don't always agree, but if the action behind the curtain exhibits creativity and talent it's usually declared "something" and if it's showy but pointless it will garner a "not anything." When Letterman and Schaeffer disagree, it's because they have different perceptions of what constitutes "anything.")

OpenAugment

One of the more interesting projects I have come across in a while, OpenAugment...

The OpenAugment Consortium is a not-for-profit open source corporation dedicated to the preservation of the Augment legacy. Founded in 2002, the consortium is comprised of a small dedicated staff and a number of research partners and associates.

Created in the 1960's by Dr. Douglas Engelbart and his imaginative team at Stanford Research Labs (SRI), Augment is one of the most groundbreaking and important historical artifacts of the software industry. Many of today's desktop and network computing innovations can be traced back to the original Augment system.

Today, the OpenAugment Consortium is taking steps to ensure that future generations will have access to the Augment legacy through this open source initiative. Please explore the rest of this site to find out more about Augment, the OpenAugment Consortium and how you can play a part in preserving this vital piece of computing history.

Answer me this

Today I am listening to one of the local "classic rock" stations. The DJ announces it's "Two for Tuesday". He also announces "We're in the middle of a 25 song classic song salute."

So does that mean for the 13th artist "two-fer" they're gonna play a classic and then a flop?

I'm just thinkin'.

Why Unix?

Better Living's take on software the doesn't stink is fine, but this is a controversial paragraph that caught my attention...

For what its's worth, I think that open-source is no panacea, and in fact is one of the biggest black-holes sucking away human talent needlessly these days. How many man-hours have been spent building a clone of the 30 year-old Unix operating system? There are many better areas for us to be applying talent. And I don't mean to diminish the professionalism of Microsoft developers. The product teams here are some of the most well-tuned machines I have ever seen, but "best" is not the same as "perfect" or even "as good as possible".

(BTW --- What's the difference between "perfect" and "as good as possible"? Nevermind.)

I agree open source is no panacea, but I think it's a better Cambrian Explosion than the Procrustean Bed that is the Microsoft platform.

And why should one develop an open source Unix clone? For one thing, because one can! The ideas are well known and successful, which makes for the best patterns and lowest risk. That's what's known and now accepted as pattern-oriented software development, and so most software development in general should be like this. See Eric Raymond's book, The Art of Unix Programming.

There are many better areas for us to be applying talent.

But going all the way back to Stallman's instantiation of GNU, clearly (in hindsight!), there is a need for a platform for innovation. The platform should be well known and successful, but also unencumbered by proprietariness or the unspoken requirement to abide the vendor's cash cows.

Why try to innovate on a platform vendor whose not-so-implicit intention is ultimately to own any and every idea that succeeds? Enough said there, even so, "Why Unix?" should be *obvious* to a software developer for many reasons. None of which are, or need to be, "Because it rocks!"

Why PythonNet?

Gordon says Python is complete, so why PythonNet? I agree that Python 2.3 has a lot to offer out of the box. But I'm using PythonNet for the same reason I'm using Jython, as Gordon suspects, to get to the libraries in dotnet and the JVM, respectively.

In particular, in this moment, I want to get to SWT, Java2D, and GDI+ for drawing. I could just use the GDI+ DLL without dotnet, but the dotnet API is better. I also want to use other C# code from Python and vice versa.

By the way, I am also starting to use Cocoa via PyObjC for the Macintosh. Other than a thin layer of low level GUI and graphics, I have a growing capability to do "fully native" write once, run anywhere, from Python. Of course with Jython in the mix you have to be careful, since it is not up to the 2.3 definition of C Python. That's OK for now.

Why not just use WxPython? Because I want to use the native APIs and integrate with other native code and I don't want a layer as big as WxWindows in between. It's really not that hard. When you structure your system like this...

1. Domain Model
2. UI and Drawing Model
3. Native UI and Drawing

...you can make the drawing model fairly rich because all the brushes, pens, clipping regions, and graphics contexts are at roughly the same level of detail and capability. But for the UI model I'm finding it's better to keep the abstract UI model very simple, essentially just a set of Commands, Tools, and basic Layouts that map to many more detailed UI objects at the lowest native layer.

That is, don't try to create a WxPython. That's too much work with little payoff. Who wants to program at that level of detail even if it is cross-platform? Let the high level UI objects *generate* all the low level GUI objects in a platform-specific way. Details to come, when I have more of it figured out!

Not on the Up and Up: Oil in the Caspian region

Also from Fresh Air on the 12th, listen to journalist Lutz Kleveman talk about his new book, The New Great Game. Pay attention to which side we're (the USA) on in the "game" for Caspian oil, then tell me we're in Iraq to liberate the people from an evil dictator.

Worse is to come: disgusted with the US's cynical alliances with their corrupt and despotic rulers, the region's impoverished populaces increasingly embrace virulent anti-Americanism and militant Islam. As in Iraq, America's brazen energy imperialism in Central Asia jeopardizes the few successes in the war on terror because the resentment it causes makes it ever easier for terrorist groups to recruit angry young men. It is all very well to pursue oil interests, but is it worth mortgaging our security to do so?

On the Up and Up: More evidence against the Semantic Web

Listen to the segment on this page with the linguist Geoff Nunberg. He addresses misunderstanding in conversations (the human-human kind). This is interesting unto itself, but throw in a machine and see what happpens.

640k: This memory needs error correction

Update: Jon Udell addresses this today from another angle, i.e. the query language instead of the data model. Here's his conclusion: It's about smooth interop between the next-gen Windows filesystem and the larger ecosystem it will play in. If Microsoft will be getting into the role of "schema provider" they'll have to do better than their recent Office XML provisions. The rest of us want multiple platforms and an unencumbered standard.

Ray Ozzie writes glowingly about the officially years-away WinFS file system. ...

Microsoft will obviously drive the initial schemas required by the core system - such as Contact - but where will it go from there?

Nothing two years away in the computer industry is obvious to me. One thing that is not obvious at all to me is why wait two years?

For one thing, Macromedia has a pretty good story with Central, which is based on Flash, is in beta already, and already includes a growing list of initial schemas, such as Contact. A second example is Chandler, which will also have a repository of arbitrary types of Items, including an initial set for the functionality that will come out of the box. Both Central and Chandler are attempts at a multi-application framework for "rich clients" bases on semi-structured information.

I could guess how such a set of schemas could be aided by WinFS, but is Microsoft making this point themselves? How do we know they'll get it right? Is there a convincing reason to wait for WinFS to begin?

Consider instead Google, Google Sets, and Syncato. These are all ways of structuring, searching, and organizing information in various bits and pieces. None of them require a new kind of file system.

I think we have what we need for the back end and front end of a new ecosystem of semi-structured information. I don't see how waiting two years or so for Microsoft is going to help. The key will be evolving these multiple attempts to be more aware of each other, not to wait for Microsoft to eventually get around to duplicating these ideas in some singular vision.

From James Robertson's Smalltalk blog...

Java mostly lives on the server - it's been a roaring success there, but it's failed on the client for the same reason that our product, VisualWorks didn't get that much traction on the desktop - end users really, really want apps to look and feel the same. We are addressing this by moving towards Pollock

The irony in this is that with Longhorn, the GUI is becoming more variable in appearance and layout, as well as lighter weight. The Longhorn GUI is going to be *more* like the emulated (i.e. "drawn") widgets of Visual Works, but with a more sophisticated drawing model.

By and large people have been successful with the variety offered by DHTML user interfaces and game interfaces. One of the most appealing user interfaces that I am aware of (Hypercard) is also notorious for breaking the user interface guidelines established by the same vendor.

Still, Pollock is a good thing for Smalltalk, hopefully providing the flexibility to access all of Avalon when or if Longhorn finally arrives.

My day just got a little brighter: Write Once, Run Anywhere!

The rain has let up and I can see the sun is out over the coastal range, but that's not the real source of sunshine in my day. I had been working on translating a little bit of Java into C#. I gave the automatic translator a quick try, but a strange error was not motivating enough to push through. Hand translation soon gave way last weekend to another project.

I had started to doubt the value of my time vs. the effort of getting up to speed at all. I installed NUnit (not nearly as much documentation as with jUnit) and NAnt (ditto vs. Ant). I managed to piece together enough of a .build file to compile a partially translated .DLL and run a few tests. (Using the unobvious NUnit2 task as opposed to the NUnit task!)

Then the light came from above. What's the state of C Python for .Net scripting? Production ready. Great news! What does it take to run it? Download and click on python.exe. Greater news! I can program using the simple, interactive, (and familiar to me) Python environment.

In just a couple of minutes my whole day, and project, turned from dread to desire. Rather than translating from Java to C# and maintaining two source paths, I'm translating from Java to Python and maintianing one source path that has a little glue into the Java library via Jython and a little glue into the dotnet library via CPython.Net.

Every indication is the CLR will eventually support Python and other dynamic languages much better as first class members of the CLR the way Jython works in the JVM. Meanwhile CPython.Net will do fine, consider this example of handling events...

          def handler(source, args):
              print 'my_handler called!'

          # register event handler
          object.SomeEvent += handler

          # unregister event handler
          object.SomeEvent -= handler

          # fire the event
          result = object.SomeEvent(...)

C Python is the best way to program in dotnet, and Jython is (one of) the best ways to program the JVM. (The JVM still rules on my personal list of interesting languages.)