Cement And Such

From the P.D....

It’s fascinating to note that we spend a massive amount of time focusing on making software malleable at compile/build time (Spring anyone?) but considerably less effort on ensuring similar flexibility post deployment making for brittleness in face of failure, upgrade, configuration changes, scaling etc.

Free Data

You have to be real careful when you invest in some vendor's technology. Will they help or hinder getting you where you want to go.

Microsoft I gather is *real* scared of google, and so they try emulating a bunch of googlesque capabilities. As usual I have not read a lot about them, but this one about "live data" and "data wants to be free" caught my attention.

See, calendar information seems to me to be the data that *most* wants to be free, but isn't yet as free as it should be. Is calendar information now free from Microsoft?

Or is there motto: "Data wants to be free, and will be, unless we can lock it up in a proprietary server, obfuscate it in proprietary formats, and extort you to pay for a lot of client licenses to get to it using tools of our choice"?

Maybe things have changed -- please correct me if I am wrong, or explain why you don't need fully functional calendar information to be free.

Take Five: Processes

Guido van Rossum (via Joe Gregario) (via Sean McGrath) on "threads"...

The difference is, for an OS kernel, there really isn't any other way to benefit from multiple CPUs. But for Python, there is -- run multiple processes instead of threads!

Yeah, I don't like threads one bit.

How much support does Python have (in the distribution or from other contributions) for starting and stopping processes on the same or other machines? And for communicating with them?

I am all for this approach, but little attention is being paid that I know of to programming this way, regarding "all the little things" that are needed to develop, test, and deploy concurrent, distributed systems. Not to mention "post modern" systems using various languages.

The "common language runtime folks" (clr/dlr and jvm, etc.) are salivating over using each others classes. We should be more interested in using each others processes.

Just because Java was once aimed at a set-top box OS that didn't support multiple address spaces, and just because process creation in Windows used to be slow as a dog, doesn't mean that multiple processes (with judicious use of IPC) aren't a much better approach to writing apps for multi-CPU boxes than threads.

Even in the same address space it's better to pretend you're not. Java applets, etc. were aimed at the everyday developers, so why did the language designers give them such things as threads and locks? More out of habit and lack of questioning their hidden assumptions than anything else, I'd bet.

Yeah, it will be a good thing when we can do all our cross-platform, distributed process management stuff in IronicPython and/or Jython! (This might have been more interesting even five years ago. Now we know we could have been, and should be running in the *opposite* direction. Instead of running at full speed, we're still docked, with the steamliner facing the *wrong* direction.)

"Common Language Runtimes" -- phooey. Think broader. Patrick Mueller points to his investigations with Java in a comment to this post. Is Microsoft's CLR/DLR going to do a better job at cross-platform, distributed process management???

Hyperflux

Hugh Winkler says... (via Sean McGrath)

But if you're not pushing a bunch of hypertext down to my browser, you're not helping me explore the space.

I agree with this. Nothing about the server should be specific to the client. Adobe promotes FDS (Flex Data Services) and there may be a time and place to use it (or not), but that's not "on the web".

I don't think developers should slide into the mode of piece-wise assembly of a bunch of widgets and wiring them up to the automated updaters keeping objects-in-sync across networks.

But taking a more web-oriented approach doesn't preclude using client stuff that's a bit more expressive than current popular browsers. There're good ajax apps and there're bad ajax apps. Same with other "RIA" technologies.

RIAs built on something like Flex and possibly Apollo, but using the web in the right way is appealing to me. Finding more simple ways to build the RIA part *and* get the most out of the web is an interesting challenge.

Jini Panic

Fuzzy relays the state of the Jini world... great Java technology. Community is awful. Not easy to invest in that direction, when the signs of community since River started have been no better, or worse, than prior to that event.

Sad, really.

On Thursday an email went out to arrange a get together on Monday at JavaOne. I will be down in SF on Thursday and Friday, but too late to change plans for Monday.

Sigh.

chromatic on silverspoon

chromatic wonders about duplicating silverspoon on linux...

I’m not sure that making the life of the marketing department of a convicted monopolist which just loves to embrace, extend, and extinguish competition is the best way to spread freedom through software.

Here's my deal: the MSFT community seems to hang on every word of every potential innovation from Redmond. The same is true to some extent in the Apple community, but to a far lesser extent. In part this could be so because MacOSX is based on Unix, and so the culture and innovations can cross over much more easily.

By and large Linux, Java, and the other various language-centered communities have the tools to innovate and the culture to innovate and *do* innovate more (in my preception - not formally measured) than the MSFT community.

And so would you rather your world be based on the governed pace of innovation from one large bureaucracy? Or would you rather have more freedom to move in directions and at a pace best suited to your market?

I can count all kinds of devices running Linux in my house, from several vendors. Not so for Windows. Getting silverspoon running compatible, not getting incompatible, etc. is not a good choice when given the choice to be more innovative, whether based on Adobe Flex/Apollo or anything else.

"Anything but MSFT." seems to be the most logical choice unless your market is already so caught up in MSFT's and your interests are so closely aligned with theirs.

They will *not* give you the tools you need unless that meets their needs. You choose.

Better to ignore them or force them to play your game, than try to play theirs.

Making Us Stuck

Intel is concerned about their multicore roadmap. Chips could potentially have 32 cores within three years, but Intel may have to find other things to do with all their transistors until software can catch up and take better advantage of so many cores. This is not the first time software has failed to keep pace with hardware.

Next questions: What effects might the Sapir-Whorf hypothesis have on software evolution? (As interpreted by Ken Iverson re: programming languages. (pdf))

What contraints or propels software evolution vs. hardware evolution?

What Goes Around

Is the actor model on the list of foundational topics in CS programs today?

Is the actor model about "objects"? Why or why not?

Is the actor model relevant today? Why or why not?

Are today's systems becoming more or becoming less like actor systems? Why or why not?

Why Unix?

Why ask?

Bungle

Something screwy happened with blogger. I cannot tell, but I may have lost several comments from today. If you don't see yours and you are so motivated, please try again. Sorry.

Flair

Sun gets in the game of replacing java, at least on the desktop. I would *never* make the mistake of underestimating Dan Ingalls, now a Sun Distinguished Engineer and previously the primary implementor of Smalltalk at Xerox PARC, Apple, HP, and elsewhere up through and including Squeak.

"AJAX deals with all of the old way of doing things. It makes it simpler, which is great, but underneath it’s still all this junky HTML, Document Object Model, cross site scripting, all that stuff, where 30 years ago, we knew how to do that stuff cleanly with a dynamic programming language and a simple graphics model," Ingalls said.

I Love This

Adobe and MSFT going head to head, competing for the most appealing, most open, next-generation web client technology. All of a sudden the web is a lot more interesting than just tracking what ajax toolkits run in which versions of what browsers in order to get us more than green screens of zzzzzzzzzzz.

Browsers and HTML were great because they got the UI out of the widget builder era. All of a sudden there were no rules for how pleasingly creative a UI could be. But that was the 1990s.

Now the new tools are putting back good things from the structured graphics era, including those widgets, but they retain the creative flair of the HTML era.

Let the atom, json, structured graphics era begin!

Apollo May Become The Best Ajax Platform

James Robertson writes...

Microsoft is trying to create the kind of walled garden they stumbled into on the desktop out on the net. The problem is, a lot of us would rather develop/deploy on Linux (because it's easier to manage a Linux server remotely). At present, Silverlight is completely uninteresting if that's where you are, and they aren't likely to change that...

Adobe's Apollo, on the other hand - it's going to end up inside and outside. It's the game to watch, IMHO.

James on silverspoon vs. ajax...

Ajax doesn't limit you in the same ways.

And the cool thing about apollo is its support for flex *and* ajax. Apollo may become the *best* ajax platform. One, because it has all the other apollo features. Two, because it will be the *same* ajax environment on all platforms.

I think the ball is solidly in adobe's hands. It is their ball to drop. They can leverage their proprietary components and their open source directions into quite a sweet spot.

Pilgrimage to Someplace Better

Mark Pilgrim rants about something to do with Apollo, Flex, Flash, etc.

Not that Apollo is the final destination, but it is a helluva better step towards something useful than the place browsers seem to be going anytime soon.

These things *are* becoming more open and they are "on the web". And they are a helluva lot more programmable than browsers.

How long as SVG been around? And it can do what, again? With how much complexity?

Have fun. Fuzzy explains it.

HTTP Is Pushy

There's a lot of rest-related stuff about push vs. pull, and how pull is just fine. I really like Sean McGrath's recent explanation of rest. I am wondering about this one thing, though...

Is there a dramatically simpler solution to the push-centric, transactional, reliable once-and-only-once one that human language has a way of veering us towards?

Just a thought: rest emphasizes that "push" does not have to go hand-in-hand with "enterprisey". Looking at the Atom Publishing Protocol, isn't the big point of APP that it *is* about "push"?

We pretty much all *get* GET, hmm? It's the other stuff we don't really *get* yet, like "A and B pushing changes into the URI space" as Sean puts it. That's good stuff.

The Do What I Say Dept.

(Via Steve Dekorte), comes this...

George W. Bush, 4/9/99, Houston Chronicle:

"Victory means exit strategy, and it's important for the president to explain to us what the exit strategy is."

George W. Bush, 6/5/99, Scripps Howard/Seattle Post-Intelligencer:

"I think it's also important for the president to lay out a timetable as to how long they will be involved and when they will be withdrawn."

Cross Platform Development -- Is Java The Loser?

From an article on silverspoon...

The outstanding question is whether Microsoft plans to offer Silverlight support for Linux. Although support for Flash for Linux lags behind Windows and Mac, Warriner noted that his company can still count on Flash Web applications running on Linux.

Yeah, but here's the other thing about cross platform internets...

I want to *develop* on other platforms than Windows. Not just deliver. (Moot point for silverspoon -- it neither delivers nor develops on Linux, apparently.)

The difference between using Windows and Linux for software development tools, etc. is *immense*. (Well, maybe *you* like futzing with cygwin to achieve a fairly lame ersatz unix.)

Although a Flex Builder tool for Eclipse on Linux would be worth trying, I sure don't miss it now. I use the SDK and a command line very well inside Emacs.

The one thing I noticed today though, is a co-worker is using Flex Builder's "suggest" capabilities and found a method on DisplayObject I was looking for. Good command line tools would help, but such is the way of the IDE these days -- use it or lose it. In lieu of really good search tools, using "suggest" only as a search tool is an option - my co-worker had his second screen set up for this.

Apollo is *clearly* the internet platform to beat at this point. Given that Adobe has also donated Flex and the Actionscript VM to open source, esp. Mozilla/Firefox, I wonder how long before Firefox becomes an open source, cross platform browser with built-in Flash/Flex and oh by the way has Apollo-ish desktop capabilities right there, USB access, OS, User Profile access, and so on. And apparently SQL in the not too distant future.

Cross platform. Virtual machine. Development tool. The works. -- Peter Fisk has interpreters running in Flash, but the Flash VM will give people access to the byte codes so they can run all kinds of languages directly ultimately.

Microsoft is not about to take their WPF-lite into the cross platform desktop domain. Competition is good - Adobe and MSFT can push each other on into the future -- fine with me.

With all this MSFT Silerspoon vs. Adobe Flex/Apollo the real loser on the desktop is *java*. Adobe is heavily into Java on the server, but Flex/Apollo works at least as well with all kinds of servers. (Flex Data Services notwithstanding -- that's not a huge enticement to me even for server-side Java.)

No Thanks Netflix

I use Linux. I've been thinking about Netflix, but also considering other services such as the one from Tivo and Amazon, since I already have Tivo. But I also have iTunes and a video-capable iPod, which I love dearly, so I am kind of watching where Apple TV is going. I would have to buy a Mac *and* Apple TV. I got rid of my last old Mac a while back. But that's a net positive. I almost have Windows out of the house entirely and do not want to build any more dependencies on the one my wife still uses.

But now that Netflix has declared my business to be unimportant...

Netflix plans to adopt Silverlight as the foundation for its instant-viewing feature; a demo showed off high-quality streaming video overlaid with DVD-like menus and controls.

My decisions just got one step easier. Maybe next time, Netflix, when you decide not to rule me out of your internets.

The Simple Things You See Are All Complicated

Note: IronicPython actually *adds* to the CLR to achieve a DLR.

Step forward? Whatever.

Maybe Knot

From http://astoria.mslivelabs.com/...

The goal of Microsoft Codename Astoria is to enable applications to expose data as a data service that can be consumed by web clients within a corporate network and across the internet. The data service is reachable over HTTP, and URIs are used to identify the various pieces of information available through the service.

But, em, er, to find out more about Microsoft's "data service... reachable over HTTP" you are expected to download several Microsoft Word documents.

Bzzzt. Next?

On The Other Hand: Don't Fidget With Widgets

Peter Fisk puts the net in its place. Well, he makes reasonable parallels anyway.

As for "newer" things like flex, xaml (silverspoon or something?), hey, the same ideas have been around quite a while too. Sending GUIs and graphics over a network was not unheard of in dynamic languages in the late 1980s.

If only someone had made NeWS a bit more available.

Don't fidget with widgets, draw!

When run on workstations with 10 MIPS or faster processors, interactive response is excellent, indicating that ezd’s simple structured graphics drawing model can be widely applied.

Short Stack

Don Box on books...

I’ll read the shortest book on the topic.

Let me repeat that.

I’ll read the shortest book on the topic.

Amen. I just saw a pre-announcement for a book on Continuous Integration. The book is supposed to run at 320 pages. That seems kinda big for a book on CI. Either the book is incorporating a lot of related topics, which I'd rather not have included, or there is a lot more to read on CI than I'd imagined.

And I'm doubting I want to *know* that much about CI, let alone *read* that much on it.

Reality Based

Tim Ewald...

Building something real has a way of focusing your decisions about technology.

Adobe to Open Source Flex

That's kind of cool, in a way.

:-)

(Not) On Top Of The Web?

Mark Baker says Adobe Apollo would be...

so much better had they simply innovated on top of the Web

I don't understand how Apollo is not "innovating on top of the web". Sure, it is not *solely* on top of the web. But the browser is not *solely* on top of the web either, is it? I mean the browser accesses the desktop. Apollo apps can do the same, only as a developer I can use Apollo's desktop abilities to go beyond the browser's desktop abilities.

Is one (the browser) a good "web" use of the desktop, but the other (Apollo) not a good "web" use of the desktop? Why or why not?

Apollo simply points out how badly the browser generally sucks and essentially stagnated. Working groups or whatever, there is some catching up to do, on top of the web or not.

Over time I expect to be able to browse in Apollo as well as run all kinds of other, more expressive applications that are "on the web" or even off. At some point down the road turning off Firefox could be an option.

Actionscript Loses Eval

Like Javascript, previous versions of Actionscript had eval(). The latest, Actionscript 3.0, does not.

Peter Fisk's Smalltalk and Lisp in Flex is a treasure in its own right. But in light of Adobe taking away eval(), the desire to have some kind of language interpreter at runtime goes exponential.

I could do with evaluating Actionscript at runtime. There are legitimate reasons to have a fairly-Javascript-like language for customizing interactive applications, although Lisp and Smalltalk appeal to me more, personally.

But Adobe appears intent to make Actionscript as much a static language as possible, and one that has a fairly strict boundary between development-time and run-time. Sadly.

I don't think this can or should be chalked up to security. There are better solutions to security than to throw out run-time evaluation altogether.

Oh well. As Peter has demonstrated, reflection still works fine, so you just build your own eval() for whatever language you desire.

Dependencies

Tim Bray writes about IT...

But the real take-away is this, and it’s something that worries me more and more. I’m convinced that, increasingly, the proportion of enterprise software development based on dynamic languages and AMP technologies and Rails and rapid-iteration continuous-beta “Web 2.0” thinking will increase for the foreseeable future. But that doesn’t mean that Java or .NET or even COBOL are going away, and the brutal truth is that we don’t really don’t have anything like industry consensus on the best practices for integrating Rails and PHP and Java EE and .NET/SQLServer and Cobol/IMS and Ada and all the other weird old shit that’s out there doing boring vital business functions.

It's worse than that.

I've worked on all kinds of systems for design, manufacturing, collaboration, and more. These systems were written in Lisp, Smalltalk, Modula-like languages, C++, C, Java, and more.

The systems that caused problems were those that had unnecessary dependencies among the components. Some of these couplings were all in the same language, designed roughly together. Others of these were in different languages, designed at different times by different people.

Most of the Lisp and Smalltalk systems had as many problems as the others. The nature of the languages made them somewhat easier to deal with. Somewhat.

IT systems are parallized for several reasons. First and foremost is that past systems have so many unnecessary dependencies that they are nearly impossible to disentangle. Significant, ongoing, incremental improvements are few and far between.

Yesterday's best practices did not circumvent this, except in rare circumstances. Tomorrow's likely will not either, except in rare circumstances.

I would almost always choose a dynamic language over a static one, even over those that have implicit and/or optional type checking. (Oh, no. Don't comment on that here.) I would also lean towards rest, atom, "web 2.0", etc. Those are the ways to go.

But incredible diligence will be required to turn those into long-term, evolvable, systems that continue to meet the needs of the business with reasonable investment. (I'm not sure how to measure "reasonable investment", but the cost of change should be somehow proportional to the difference between the business functions already built and the business functions to be built.)

And Now An Example That Works

Alright then, updating to Actionscript 3 the example linked in the previous post which was Actionscript 2, we get something that works, like this...

<?xml version="1.0" encoding="utf-8"?>
<mx:Application xmlns:mx="http://www.adobe.com/2006/mxml" creationComplete="initApp(event)">
    <mx:Script>
    <![CDATA[
             import flash.events.Event;
             import flash.events.MouseEvent;
             import mx.containers.Panel;
             import mx.controls.Alert;
             import mx.controls.Button;
             import mx.controls.Label;
             import mx.controls.TextInput;

             public var submit_button:Button = null;
             public var first_text:TextInput = null;

             public function initApp(event:Event):void {
                 var panel:Panel = new Panel();
                 panel.title = "Hello World";
                 addChild(panel);

                 var label:Label = new Label();
                 label.text = "First Name";
                 panel.addChild(label);

                 first_text = new TextInput();
                 first_text.text = "Patrick";
                 panel.addChild(first_text);

                 submit_button = new Button();
                 submit_button.label = "Submit";
                 panel.addChild(submit_button);

                 submit_button.addEventListener(MouseEvent.MOUSE_UP, onMouseUp);
             }

             public function onMouseUp(event:MouseEvent):void {
                 Alert.show("Hello " + first_text.text + "!", "Message");
             }
       ]]>
    </mx:Script>
</mx:Application> 

If you have the free Flex 2 SDK you can compile and run it like this...

$ mxmlc something_else.mxml
Loading configuration file /home/patrick/flex2/frameworks/flex-config.xml
/home/patrick/dev/patrick.logan/flashstuff/formstuff/something_else.swf (143036 bytes)
$ flashplayer something_else.swf 

Another Flex Tip

I got another promising tip from Garrett Hart here inside Liberty Mutual...

He sent along this Adobe documentation on "Dynamically Creating User Interface Components". Here we go...

Em, hold off on this one. The code as it is in the article seems to be for ActionScript 2 rather than ActionScript 3. There were substantial changes in the display object class hierarchy.

Back to the drawing board, but the Vista Smalltalk swf example is still a good indicator.

Update

The url Garrett refers to in the comments is this one. Blogger kind of mangled it.

Vista Smalltalk for Flash

A hat tip to Patrick Mueller in the comments of my most recent post on starting up with Flex.

I'd not been paying much attention to Vista Smalltalk. Very nice. And Peter Fisk seems to be using the controls from mx.controls that I had problems with. Now to dig into his code to figure out what if any magic lies behind his Lisp or if I was simply missing something.

Either way, good news. The blogs come through.

My RIA Is Dynamic -pause- Naught

Update: It's been a while but there are some corrections to this post.

• The Flex API *does* support drawing on pixmaps so a MacPaint-like capability using this would be efficient.
• All the controls can be created using the Flex 2 API fr ActionScript 3. There is just a minimal mxml wrapper required around the AS code. I can dig it out here if anyone wants to see it, but I'm too lazy at the moment.

End

I've been playing with Flex2 (I'd go all out with Apollo except it does not run on Linux yet and I don't have steady access to Windows XP or Mac OSX). I have some questions that I've posted to some lists, but maybe this blog can attract some feedback as well.

No Pixmaps?

First some general impressions: Flex could be so good, but I am not sure if it is. The basic display architecture is reasonably what I'd expect. I am kind of surprised there isn't a pixmap level drawing capability. (Or is there and I missed it?)

I created a simple paint program that frequently cycles through colors, transparency, and line width. After a minute of "painting" the retained structured graphics system bogs down and mouse events are handled fewer and farther between.

No Intelligent Redisplay?

Which leads to my next question: even if all graphics have to be retained as heavyweight objects, does the repaint really have to slow down mouse handling so dramatically, so soon? If my program is adding a new line just a few pixels long, shouldn't the damage and repaint occur in a small rectangle around that line? I cannot tell what's going on in the slowdown to be sure.

Since I have not found in the API any smart damage capability (e.g. "invalidate just this region for repainting") and only a display-wide "invalidate" then I might assume the entire display list is being repainted unnecessarily. Since the API does have a reasonable hit detection capability then I would expect that to be used to determine what to repaint, if the system had some understanding of where the damage is.

Controls Must Be Precompiled?

I would really like to generate some UI controls (e.g. labels and input controls) based on some data received from somewhere in the course of the application. I wrote a little app to try this out. Then I ran into an error at runtime when trying to add a label to a vbox... "TypeError: Error #1006: getInstance is not a function."

Apparently all the controls in mx.controls can *only* be created through XML and the Flex mxmlc compiler. Actually this is not strictly true since the -keep switch will keep around the generated ActionScript files. Em, not the most beautiful or concise code considering the app consists of a window, a vbox to stack controls, and one static text control.

All kinds of craps are generated that have *nothing* to do with the app. This can also be seen in the runtime behavior of the XML version of this simple thing. While the few small Flex apps I have written to date come up and are ready very quickly, the XML generated app comes up slowly. Looking at the code it is apparently due to all kinds of initialization of things the app will never use.

Ugh - factories initializing craps - a very heavyweight stretch of code. Generated from XML input run through a compiler. (Apparently in spite of ActionScript having reasonable support for XML *in* the language, mxml code can only be generated from a compiler written in Java that runs at build-time. (Or on a "server" if you want to call out to one that can run the compiler for you!)

The result is "rich" but not very "interactive" -- especially given the heritage of ActionScript as a dynamic language. ActionScript, Flex, and such appear to be based in the developers Java heritage. The result is a Java-ization of what used to be a dynamic browser environment into a fairly static Java webstart/applet-like environment.

So given that mxml generates AS code and I can duplicate that, then to some degree this code could be hand-written and be data-driven at runtime (i.e. be done via a "gui interpreter" of data just like a browser interprets HTML). I am going to look into this some more. The code is readable if not concise. It only needs to be done once.

I'd love to find out Flex provides an easy-to-program, dynamic GUI/graphics capability. Almost the opposite appears to be case so far. Several companies are apparently building apps like word processors for Apollo. They have to be doing some fairly low-level and efficient stuff with the text layout and redisplay. I wonder what level of the API they are using and how much code they've had to write themselves.

I could be missing wide swaths of the API that just isn't covered in the typical documentation aimed at building static GUIs in an IDE via XML.

Gambit / Termite / Erlang

Dominique Boucher compares Erlang with Gambit Scheme, and in particular Termite, which is an Erlang-like system the runs on Gambit. His conclusion...

I would choose Erlang over Gambit-C+Termite for the development of a new scalable and robust distributed application. That being said, Gambit-C is a nice piece of software (in fact my preferred Scheme system), it has a great debugger, and the Scheme Now! project will certainly lead to a number of good portable libraries for Scheme. But from the perspective of an industrial developer, Erlang wins easily.

I would have to go along with this conclusion for almost any system that might have to be "production-ready" in the next couple two, three years. On the other hand I think Gambit Scheme is the better investment for the long haul. Erlang is a great language, and OTP is a solid framework for distributed systems. But Scheme is the better "foundation" language and Gambit has all the qualities of Erlang to become a solid foundation for all kinds of distributed systems, even an Erlang system, as exhibited by ETOS (Erlang-to-Scheme).

And In The Darkness Bind Them

secretgeek is a bit more tongue in cheek about this than the Church would recommend...

Lisp is so simple that you can implement it in any language in just a few pages of code. This might never happen though, because once you've learnt lisp you'd never want to write anything in any language other than lisp, so you wouldn't bother implementing lisp in any language other than lisp.

Lisp can be fully implemented in lisp in just a handful of lines. I just implemented lisp in lisp, fully, while i was hopping onto a bus and paying for my bus ticket all at the same time...

Only lispers have a true definition of fun.

Dichotomies

Jim Waldo comments on a resurrected Jini and OSGi thing...

In fact, OSGi and Jini are service architectures built for completely different contexts. OSGi is a service architecture for services that are in the same address space. It allows you to build programs out of cooperating services. And for that sort of thing, it is pretty good.

Exactly what makes Erlang so nice as a "service-oriented" programming language:

• Invoking services in another process is easy.
• There is no distinction between a service in the same address space and one in another address space.
• There are no guaranteed service behaviors, i.e. Erlang faces head-on the problems inherent in distributed systems.

Practice Begets Simplicity

Smalltalk is a much simpler language than Ruby. The ideal implementation is correspondingly simple. This has several positive implications, as Avi Bryant points out...

Dan Ingalls did a lovely binary compatible re-implementation of the Squeak VM in Java as an exercise to learn the language, in a tiny fraction of the time that JRuby has taken, because everything important, down to the parser, compiler, process scheduler, windowing system, and IDE, were implemented in Smalltalk anyway and so could be reused. That’s the kind of trick I’d like to see Ruby able to pull off.

There are a number of idiosyncracies in Ruby, in spite of its relative simplicity. Part of the problem is Ruby has been around well over a decade with just one implementation by essentially one developer.

Consider that in Smalltalk's first decade it changed fairly drastically and did not settle until toward the end of that decade. Also consider during that time there were a handful of implementors, many users, and the intent was to change the language for the better.

Not For Long?

Yes, Seaside is mighty nice. But for two reasons:

• Browsers still suck as an application platform after so many years.
• Programming languages still suck at distributed processing after so many years.

Tweny years ago had someone suggested that in 2007 something like Seaside would be the leading edge of developing distributed systems for end-user interaction, I would have blown my coffee out through my nostrils.

Yes, Seaside is mighty nice. Sadly.

Iraq - That's Right

MoveOn is holding a virtual town hall on the Iraq catastrophe.

You can attend a local house party and view/participate from there if you'd like.

Here is a banner and link you can put on your blog if you'd like.

They are asking which candidate has the best position on Iraq in your opinion. Here is my opinion, which I don't believe is so very far off from many opinions, if not expressed in the same way...

It is a *fucking* mess and getting out of it won't be easy, but I hope several people eventually go to jail because of it. I won't vote for anyone who does not admit it was a mistake, should never have occurred, and vows to apologize to the world as the next president of the United States. Enough bullshit.

That's right. I probably will not vote for president in 2008 unless I change my mind of this.

Innovation Happened Elsewhere

Paul Snively on language innovation...

Ironically, I think it's exactly Sun's hanging onto the Java language spec for so long that's led to the level of experimentation in other languages targeting the JVM that we're seeing. I find it fascinating that the JVM hosts a Scheme as good as SISC and a statically-typed OO/functional language as good as Scala.

Stem Cell Therapy May Combat Type 1 Diabetes

In the any news is good news dept....

A pilot study of people newly diagnosed with type 1 diabetes found that stem cell therapy eliminated the need for insulin therapy for varying periods of time.

This is the first trial to look at stem cell therapy in humans with this form of the disease. But experts stressed that the research is preliminary and urged caution when interpreting the results, which are published in the April 11 issue of theJournal of the American Medical Association...

Type 1 diabetes develops when the body's immune system attacks the pancreatic beta cells, which produce insulin -- the hormone that transports sugar from the blood to cells for energy.

"In type 1 diabetes, the immune system is out of balance," Skyler explained. "Ordinarily, all of us have some cells with the potential to destroy the pancreas, but the regulatory immune system prevents those cells from becoming sufficiently active. In type 1 (diabetes), there's a greater proportion of activity of the destroying cells and lesser activity of the regulatory cells. The goal is to try to bring that back into balance."

Dynamic Language Symposium 2007

Montreal? October? Dynamic languages? I am *for* it.

How To Tell Stuff To A Computer

Some interesting things happening on knowledge representation, by way of Bill Clementson.

Paul's Cliff Notes

Paul Graham continues...

When I wrote that Microsoft was dead, I didn't mean it literally. I couldn't have. Companies aren't alive, so they can't die.

In fact "Microsoft is Dead" was what we in the trade call a metaphor. I meant something else. Over the last couple days there has been some disagreement about what I meant....

So maybe I'd better explain exactly what I did mean...

Technology companies are projectiles. And because of that you can call them dead long before any problems show up on the balance sheet. Relevance may lead revenues by five or even ten years.

Rock

This Rock thing from Sun looks interesting. From the blog of Jonathan Schwartz...

Rock is 16 cores - we haven't said how many threads per core. Nor have we said why this chip heralds the golden age of effortless parallel programming, or how it brings fault tolerance to the masses. But stay tuned, I think we're planning on talking up both in the next few weeks.

Yeah, I think there'd be some cool software to write for these.

Rediscovering Smalltalk Kind Of

Rubyists are rediscovering Smalltalk, or at least its idioms, often one at a time...

>> class A
>>   def a_whole_new_me
>>     self.class.new
>>   end
>> end
>> a = A.new
>> a.class
=> A
>> a.a_whole_new_me.class
=> A
>> class B < A; end
>> b = B.new
>> b.class
=> B
>> b.a_whole_new_me.class
=> B

Attention all Rubyists, buy these books...

• Smalltalk Best Practice Patterns
• Smalltalk With Style

April Portland Smalltalk Users Group

James Foster and Dale Henrichs of GemStone will present a beta version of the talk on running Seaside in GemStone that they'll be giving at IT360 / Smalltalk Solutions in three weeks...

What: Portland Smalltalk Users' Group
When: Tuesday, April 10, 7:00 - 9:00PM
Where: GemStone Systems

See Eric Winger's blog for more information. From the blurb...

The Seaside framework provides a layered set of abstractions over HTTP and HTML that can be used for developing sophisticated web applications in Smalltalk. Seaside was developed in Squeak and ports are available for VisualWorks and for Dolphin. While the Seaside framework elegantly addresses HTML generation and application flow-of-control issues, it still leaves a few challenges for the developer-including persistence and multi-user coordination. In this seminar we will demonstrate a port of Seaside to a new dialect: GemStone/S. As a multi-user, persistent Smalltalk implementation that has no native user interface, GemStone/S provides an excellent environment for serving HTML and keeping domain objects persistent.

Subscribe yourself to the email list for the Portland Smalltalk User Group for main-lining the fun directly into your inbox.

Gambit Wiki

Gambit Scheme now has a public wiki.

No Comment

Bill de hÓra writes...

I found myself strangely unmoved, unsurprised, unshocked, unconcerned. I saw that a firestorm has not been lit across weblogs, as would have been the case not even a year ago. It seems that no-one cares anymore

Success By Failure

James Robertson notes...

Failure is way, way safer than trying something that isn't "mainstream".

I have seen many people succeed personally essentially by enabling the failure of their organizations. I guess I shouldn't knock it.

Latent Technical Complexity

James Robertson's notes from SPA 2007, speaker Dave Thomas (the longtime Smalltalker Dave Thomas)...

The problem: we're in an incredible complexity mess due to the badness of the current languages. This has generated a plethora of tools to try and compensate for that, but it only makes for a bigger pile. We have what Dave calls "Latent Technical Complexity".

Graham's Latest

Speaking of Paul Graham, his latest essay is as entertaining as any...

Microsoft saw the danger of Javascript and tried to keep it broken for as long as they could. But eventually the open source world won, by producing Javascript libraries that grew over the brokenness of Explorer the way a tree grows over barbed wire...

The surprising fact is, brilliant hackers—dangerously brilliant hackers—can be had very cheaply, by the standards of a company as rich as Microsoft. So if they wanted to be a contender again, this is how they could do it:

1. Buy all the good "Web 2.0" startups. They could get substantially all of them for less than they'd have to pay for Facebook.
2. Put them all in a building in Silicon Valley, surrounded by lead shielding to protect them from any contact with Redmond.

I feel safe suggesting this, because they'd never do it. Microsoft's biggest weakness is that they still don't realize how much they suck. They still think they can write software in house. Maybe they can, by the standards of the desktop world. But that world ended a few years ago.

I already know what the reaction to this essay will be...

Lisp: Beating the Averages

I just installed the latest Gambit Scheme, continuing to be the best Lisp on the block. I first used Gambit around 1990 or so, when it was brand spanking new. The thing can really kick some serious ass, if you're looking to beat some averages.

Coming back around to thinking in Lisp, to some of Paul Graham's stuff...

The designers of Lisp didn't put all those parentheses in the language just to be different. To the Blub programmer, Lisp code looks weird. But those parentheses are there for a reason. They are the outward evidence of a fundamental difference between Lisp and other languages.

Lisp code is made out of Lisp data objects. And not in the trivial sense that the source files contain characters, and strings are one of the data types supported by the language. Lisp code, after it's read by the parser, is made of data structures that you can traverse.

If you understand how compilers work, what's really going on is not so much that Lisp has a strange syntax as that Lisp has no syntax. You write programs in the parse trees that get generated within the compiler when other languages are parsed. But these parse trees are fully accessible to your programs. You can write programs that manipulate them. In Lisp, these programs are called macros. They are programs that write programs.

Programs that write programs? When would you ever want to do that? Not very often, if you think in Cobol. All the time, if you think in Lisp.

Way Out

From Javaworld...

Engineers at Sun are eyeing May for the release of a 1.0 version of JRuby , which provides a Java implementation of the Ruby language... The JIT (just in time) compiler will be enabled by default this month. April plans call for deciding on the final features for the 1.0 release as well as a major bug-hunting push...

Future directions for JRuby include Ruby 2.0 bytecode support and leveraging the HotSpot JVM to speed execution.

Running the Numbers

My sister-in-law sent this link to me, using art to convey significant cultural statistics.

Crazy Fuggers

(Via Blaine Buxton) Edward Povazan exclaims...

Initial exposure to Smalltalkers can be rather confusing. At first you think you understand them, after all they are speaking of an object oriented language. Then utter confusion, they are not talking about a language so much as a whole world...

I am starting to understand things and want to be in this lively Smalltalk world.

Systems

Would you rather your body attempt to act as one "logical" cell, or are you happy to survive as a system of cooperating cells with billions dying off and being replaced each day?

Steve Loughran reminding us of a Note on Distributed Computing. From the note...

We look at a number of distributed systems that have attempted to paper over the distinction between local and remote objects, and show that such systems fail to support basic requirements of robustness and reliability. These failures have been masked in the past by the small size of the distributed systems that have been built. In the enterprise-wide distributed systems foreseen in the near future, however, such a masking will be impossible.

A favorite quote from elsewhere by Jim Waldo, one of the authors...

I've been known to claim that there are two kinds of reliable message systems. The first kind are those that come with an asterisk; following the asterisk leads you to the small print, where you find out when the messaging system can fail and so it is not, therefore, reliable. The second kind are those systems that simply lie – they are no more reliable, but they don't tell you that there are circumstances where they can fail.

Scary Cursors

If this story about vulnerable cursors does not tip us over the line, what will? Whoddathunk that a screen cursor could be the gateway for bad things on your PC?

What to do?

Capability-based systems go back to the 1960s. Someday we'll have them on the Internets, a series of tubes.

How many billions spent on broken security mechanisms? And we still have screen cursors with the power over the entire machine.

Bye Agassi

Bill hits the nail on the head writing about the resignation of SAP's Shai Agassi...

And so it goes. The slow death of SOA.

Netweaver was Agassi's baby and was expected to be more revolutionary for the company than it is turning out to be. Several SAP architects I'd spoken to did not even realize that Netweaver included an implementation of JMS, nor did they really understand what JMS is and when it might be used instead of their XI orchestration mechanism (which is built on R3's relatively cumbersome ABAP workflow engine, not on Java/Netweaver).

The most agile mechanism for working with SAP that I have seen remains an open source Perl API demo'd at OSCON several years ago.

I do hope Agassi is successful applying his abilities and resources to the energy industry though. That could be a better move than remaining with SAP.

Flash, Flex, Apollo -- Nice Bits

Via Blaine Buxton, the Omaha Dynamic Language Group will be discussing Flex and ActionScript 3.0.

Other programmers here have been diving into that the last few weeks. I've been reading in my down time from other things, but would like to write some code in the next few days. Flash 9 has a lot of general improvements as an environment for structured graphics applications, and Flex builds on that for GUI front end things that are not so oriented toward animation and graphics as the base Flash api.

Apollo just looks to be a leap ahead of other "Web 2.0" (if you will) front end environments. Busting out of the crappy browser that in spite of AJAX has been mostly languishing for years and years. We're not using Apollo yet here, but for some of the front ends we may need it could be. It's only in an initial alpha release currently. I want to play with "the bits" as they say at an Adobe rival that should be really concerned with the polish Flash, Flex, and Apollo are showing.

FIT Wiki Rules Spaces

Fuzzy writes about the work we're doing...

We beat our rules module into submission.

We're writing FIT tests in our Confluence wiki. A custom FIT runner grabs the tests using some specified label and the Confluence XML-RPC api.

We also have Bamboo up and running, and soon Jira. The Atlassian folks have some nice tools.

As Mike writes, the Javaspaces and JBoss Rules came together well. When I had looked at Drools some time ago I was turned off by its XML nature. Jess is not free, but not expensive, and has good Java integration. Jess also has a good scripting language in its own right (Lisp-based). Back then it would have been a no-brainer.

Since JBoss took on Drools they now have all but done away with XML. In fact none is required from what we've found so far. The Java integration and the rules language itself is not as good as Jess, but it is pretty good and it's free and open.

We've not tried the decision table mechanism. If we're lucky that takes care of most of the Java integration ugliness and if we're *really* lucky our business analyst can use it in whole or in part with little or no programmer intervention. She's very smart, but non-technical. That's the aim of the decision table, I believe. The Jboss site also has something on FIT and JBoss Rules, but we've not tried that yet either. Our FIT tests go through our own fixtures.

On the spaces side we dynamically update and make available all kinds of reference data and object prototypes via a space. When it came time for rules, we did the same thing. Rules are compiled and tested in the build scripts, and stashed away as release artifacts. When the environment is initialized the compiled bytes are sucked out of the files, wrapped in a RulesEntry and written to a space. The rules compiler has all kinds of dependencies but the compiled rules at runtime just depend on one core jar.

We have a mini-grid of sorts that run any kind of TaskEntry.execute() from a space. If such a task happens to need a rules engine, it makes one, takes *reads* (rules are shared, and so, read, not taken) the appropriate RulesEntry from the space, and squirts the bytes into the rule base. If the rules need to be updated then the new bytes in a new RulesEntry simply replace the old bytes in the old RulesEntry. The next time a task tries to read the entry, it gets the new rules. The TaskEntry objects and code can be updated the same way, no worrying about what is or isn't on some arbitrary JVM's classpath.

The "facts" needed by the task also come from a space. The task takes them from a space and assertObject(entry)'s them into the rule engine's working memory.

The "many, flat" nature of entry objects flowing through spaces and the ease of writing rules around "several, flat" facts makes these two mechanisms the good fit Mike describes.

I will repeat: in my experience, Jini and Javaspaces make Java applications as close to a concurrent, distributed, dynamic system experience as can be done. If you would like to use Erlang, but have to use Java, then try using Jini and Javaspaces.

We're just getting started with this experiment, but everyone's seeing the advantages over even the relatively flexible JMS products.

Meet the New Boss

Nati Shalom, not an unbiased, but still an accurate observer...

Oracle's acquisition of Tangosol is one more acquisition in a series that indicates Oracle has finally come to the conclusion that the relational database is no longer a sufficient infrastructure platform for a large class of applications, such as Extreme Transaction Processing (XTP) and real-time analytics.

Let's see how much gas Tangosol has, though. Prior to the acquisition, the head of Tangosol had speculated about the combined advantages of their data cache and the jini, or at least javaspaces, distributed architecture. Now are they back to square one with J2EE?

Massive

Massive.

Massive.

Massive.

Massive.

Delegation and Inheritance

In 1986 Henry Lieberman presented at OOPSLA a really simple object system based on delegation to any other object rather than on inheritance through some fixed tree of definitions.

Confused looks led to friendly arguments, led to the Treaty of Orlando a year later. And then Self, etc. expanded on this, Newton Script, and Javascript. (But the wonder of it has been lost along the way. Javascript is getting "serious" by adding classes and type checking and so on. Bah. Forgive them for they know not...)

As long as we still care about objects then, here comes Ian Piumarta with his new one(s), er, two, no, one: "Pepsi" and "Coke". In a recent addition to Pepsi, Ian has delicately spliced the two worlds together...

Compared to inheritance, delegation is the more flexible and general of the two techniques. However, they both have their place within an object model: inheritance for sharing of implementation state (and the methods that act upon it) for a single prototype (within a hierarchy of related prototype families), and delegation for sideways composition of (independent and previously unrelated) prototypes into a single logical composite object. This is the position adopted (and implemented) for sideways composition of Pepsi objects.

Silver Anniversary

Sometime in the last year or two I passed my Silver Anniversary of using Emacs. That pre-dates even the existence of GNU Emacs. On jwz's Emacs family tree, I have used...

• The original EMACS on a DEC-20 / Twenex
• ZMACS on Symbolics and TI Lisp Machines
• Gosling Emacs -- eh, ok.
• Unipress Emacs -- on a Data General Eclipse, the only Emacs that ever sucked
• MicroEmacs -- from Craig Finseth's family tree.
• GNU Emacs
• Epoch
• Lucid Emacs
• XEmacs

Calling EMACS an editor is like calling Earth a hunk of dirt.

— Chris DiBona

I've had Emacs running for weeks on-end with buffer lists in the dozens. I love Emacs and I am ready to be more than just friends.

How Emacs works.

Game Flow

The interactive "game flow" chart ESPN uses is a great visual overview of a high-scoring sport like basketball. See the game flow chart on this page for the Buckeyes vs. Memphis game this afternoon. (Especially because the Buckeyes won.)

Be sure to roll the cursor over the game flow.

Process Improvement

There are a few things I have been wanting to tie together for the last week or so. Now I guess I am getting around to it. Otherwise another few weeks could fly by.

In no particular order they are:

• Gilad Bracha's ideas on "Service Objects" and their lifecycles.
• Ian Pumierta's (et al.) work on bootstrapping an ideally minimimal, evolvable object system. (video from a Stanford lecture)
• Dan Creswell's thoughts on objects and dependency injection, and concurrent, distributed systems evolution.

Maybe you can see what this is about just from that list. Now I am going to tie in a couple other things.

• Gilad's talk mentions Erlang.
• Everything about objects gets back to Alan Kay sooner or later. One of my favorite quotes of his...

Smalltalk is object-oriented, but it should have been message oriented.

What this means to me is that a system (in the home, in the data center, around the world, whereever) should be able to send messages to each other far more easily than they currently are able. And systems should be far less concerned with what is "on the inside", or that their insides necessarily have anything in common with each other.

Erlang is currently the premier tool for building concurrent systems. And Joe Armstrong is its prophet. Really interesting web servers (yaws and erlyweb), IM servers (ejabberd), mail servers (experimental and commercial), voice servers, data bases, and enterprise message servers exist in Erlang. And a ton of other things. Aside: recently Erlang started taking advantage of multicore and SMP hardware in addition to distributed hardware.

I appreciate the important ideas and work that Gilad, Ian, and others are doing. At the end of the day though that work strikes me as improvements on the current mediocre state of the intraprocess runtimes. The Smalltalk folks have already shown the runtime can be far simpler, stable, and evolvable than the ones most of us use most of the time. (PDF on the Silt server in Smalltalk.) Gilad and Ian's work may ultimately improve on that.

Does your app leave little "pid" files around to indicate something is running with some OS process id? Or can your app actually start up and detect that some other process in the system is running? And begin communicating with it?

Can those various processes come and go, and yet the system continues to run, perhaps after healing itself?

That is *service* oriented and that seems to be more about messages and not so much about objects. We should have a "programmer's holiday" where we all pause our current work and play with Erlang for a week. Not that we should all be using Erlang for everything, but we should all be *influenced* by Erlang for everything.

The Computer Is (on the?) The Network

Dan Creswell writes about the coming onslaught of multicores and multinodes and the prevailing mindset of byte code virtual machines that still closely resemble the UCSD Pascal system from 1980.

For all the attention that multicores are getting from AMD, Intel, Stanford, Berkeley, and elsewhere it is shockingly apparent that on the software side the more things change the more they stay the same. Rather than the other way around.

Apache Meetup

The Apache SOA folks would do themselves a favor to meetup with the Apache River folks.

And vice versa.

Not that I care two hoots and a holler about SOA. I'm just saying.

Dynamic DST

A fairly good indication of software brittleness reared up today in the US, and anywhere else time is computed for the US. Most software has some fairly hard-coded time zone mechanism that never anticipated the federal government's regulation moving Daylight Savings Time this year. Most systems that were upgradable required more manual labor than they should have.

How many other assumptions are baked in that would benefit from a more dynamically loaded aproach?

We want our pieces to be small and loosely joined, but by and large they are far from it. Moving in that direction is not easy. There is so much we have not even considered yet.

State

Dan Creswell writing again, about "state" this time...

Maintenance of state is a shared responsibility for a system. We should seek to place that responsibility in appropriate places at appropriate times and be much more aware of responsibility boundaries and when it’s appropriate, share that responsibility amongst components.

Generally we consider TCP to be responsible for ensuring that state makes it to the other end of the connection. One hands some data to the TCP layer and we expect that it will ensure the data reaches the recipient. But is this true? What happens if we suffer a power outage before TCP transmits the data? When the machine restarts, is TCP going to restart and resend all that unsent data? Clearly not, whoever delegated responsibility to TCP for this data will now need to take steps to recover the situation.

Minimalism

Dan Creswell writes about the various directions Javaspaces could be taken...

I’m a minimalist, I like my JavaSpaces nice and simple and I like being able to construct cleanly layered frameworks on top of JavaSpaces. This allows me to have nicely separated responsibilities at each layer leading to (IMHO) better, more understandable, more maintainable design in my systems. I also like to avoid building such layers on top of JavaSpaces if there’s something out there already that can do the job better in a specific scenario.

How Far The Browser

I've not done anything with Flex yet. Some co-workers have. It seems kind of nice relative to the current state of Ajax or the Java/C#-based "rich clients".

Web browsers really suck when you look back at their history. They've been around too long to be so bad.

More Erlang

Over on Lambda the Ultimate a growing thread on concurrency, Erlang, Lisp, and such. The thread centers around another successul use of Erlang in the real world, the online game "Vendetta Online"...

The new erlang based system is now in production. For those who haven't been following, we ran into problems with our existing Lisp-based system (named "Deliverator") which handles high-level AI behaviour.. large groups of NPCs, large battles and the like. Over the last couple of months, we've been in the process of migrating to a much more scalable architecture (named "Kourier") based on Erlang, an elegant distributed-programming platform.

Concurrently, a new book by Joe Armstrong is in the works (available now as a beta PDF) published by Pragmatic Programmers. The table of contents looks good. I bought the PDF but who knows when I'll have time to dig into it. There is an older book online that's pretty good, but somewhat out of date, "Erlang in Real Time".

The original book, "Concurrent Programming in Erlang", is a smaller effort, fine for beginning, but the two mentioned above get into more details required for building complete systems.

call-with-current-continuation and java

Fuzzy sent a link to this piece on adding continuations to Java, more or less. The article mentions a potential JSR, but from reading through it, I'd expect a good bit of experience should be racked up before that would be practical. Apparently the implementation works now but plays some tricks to do so.

Continuations have been incorporated into several Web application frameworks, including RIFE and WebWork. In this interview with Artima, RIFE project founder Geert Bevin discusses how continuations can simplify complex workflows, and how they are implemented in RIFE.

A BIG Snow -- Scheme Now!

Snow (short, for Scheme Now!) is something the Scheme programming language community has needed for a long time. Snow is a packaging specification and repository that has been ported across multiple Scheme implementations. Yow. Marc Feeley, long-time implementer of Gambit Scheme, is one of the primary forces.

Snow is a general framework for developing and distributing portable Scheme packages. Snow comes with a set of core packages that provide portable APIs for practical programming features such as networking, cryptography, data compression, file system access, etc. Snow packages can export procedures, macros and records.

While Snow depends on non-standard features of the host Scheme system, the APIs Snow provides can be used in most R4RS Scheme systems. The Snow framework is a specification of a package structure and a package distribution protocol. The framework is not biased toward an existing Scheme module system, but can be mapped fairly directly to many existing module systems allowing Snow packages to be used from code that is Scheme system specific as well as from other Snow packages.

Such specialized Snow framework implementations have already been written for some of the popular Scheme systems and efforts are underway to implement more. We have also written a generic Snow framework implementation that works on most Scheme systems but that does not achieve the same level of integration with the host Scheme system. It provides features which are helpful for testing portability of newly written packages and it is useful when a specialized implementation of Snow is not yet available for the host Scheme system. The generic Snow framework implementation currently supports a dozen host Scheme systems.

Non-Symbolic Languages

Gilad Bracha writes...

Java was actually designed to have tuples from the start, but they never quite got in. At one point, I tried to add a tuple-like construct as part of JSR-65. We wanted to extend array initializers to full blown expressions. Eventually, that effort got quashed; I didn’t really resist, since the extension was painful, as such things always are in Java. We should have just done straight tuples as a separate construct - but at the time, that was frowned upon too.

This is because Java's syntax and the better part of its semantics are derived from the C family. Ironically Java is essentially a "symbolic language" underneath (i.e. the implementation is derived from the Lisp/Smalltalk family.)

Arguably both of these choices increased Java's popularity. Unfortunately the choice of syntax and semantics has stunted Java's expressiveness and will never overcome that beyond incremental improvements.

Update

I am asked in a comment what I mean by "symbolic language". I could have written "dynamic language". Essentially I mean the languages that have evolved from or were greatly influenced by the original Lisp implementations from the 1960s. The "symbolic" part goes back to Lisp's fundamental data types: symbols and lists (or symbols). Of course it has numbers and all kinds of data types now, but originally Lisp was intended for manipulation of symbols more than numbers, and so was called a "symbolic language".

End

Rotten Apple

Katie Dean wonders "if Apple lost Jobs"...

Apple has said that Jobs knew of backdated option grants but "was unaware of the accounting implications," and an internal investigation cleared him of misconduct.

Hypothetical satire follows...

Steve Jobs: "Gee, yeah, I knew we were backdating stock options to ideal dates where we could capitalize on changes in stock price that already occurred in order to enrich ourselves, but... you mean to say there is a problem with that?"

AlgoKit

Like Process One, Joel Reymont is also going at it with Erlang (and OCaml)...

AlgoKit is the high-frequency automated trading platform that I’m developing in Erlang.

My focus is futures and I’m not planning to use technical analysis. I want to capture and replay ticks, build a “Depth of Market screen” in software and make it simple to write, test and execute strategies that rely on volume, market delta and depth of market. Once the trading bot is ready I want to be able to co-locate it near the exchange, run it 24/7 and monitor it remotely via web or instant messaging. I’m after low to ultra-low latency so performance is definitely something I keep in mind...

I can receive a market quote, make a decision, execute a trade and receive a comfirmation within 5-10ms if I co-locate near the exchange. I would need to monitor the health of my algorithms, though, and would want to receive periodic updates. Erlang has both a web server and a Jabber/XMPP server so I can modify parameters of my algorithms using a web browser and check up on things via instant messaging...

The ideal solution is to marry the best of Erlang with the best of OCaml and write the algorithms themselves in OCaml while receiving market data from Erlang and feeding trades back to it for execution. AMQP and RabbitMQ (written in Erlang!) may just be the connection.

Outer Limits

Mickaël Rémond writes in the Process One blog how they are taking ejabberd to the outer limits...

At Process-one, we have always been performance freaks. ejabberd is without a doubt one of the most scalable XMPP server around and we have a strong reputation among our customers...

Angie is our internal codename for our program to move ejabberd to gigantic scale and make it able to support millions of users in a single domain...

We have reached 600,000 simultaneously connected users in benchmark, but we now want to increase the performance to be able to pass the 1 million users mark on the same hardware (We can probably go further, but we would need more machine).

Memorablies

Among the several artifacts from my 24 hour rant I will always treasure are the following quotes...

• "Refactoring for factuality, I think he means something more like..."
• "I've seen synchronized markers employed like pixie dust until the problem seems to go away."
• "I have a hard time telling what's really going on on that blog page."
• "Blimey, that’s gonna cause a bit of a ruckus."
• "It's an apples and rhinos comparison."
• "Lots of dodgy assertions and straw-man--bashing, but there are some lucid moments."
• "You're looking at premium car-wax, retina burning shine'y"
• "Right now the post is so heavily edited that it looks worse than a wiki page on thread mode during a flame war."

Yes, and I edited that page by myself. No help. 8-)

Classics. Thank you for the quotes and the arguments one way or the other.

On Complexity

The question is asked over on LtU about my transactional memory rant...

why is the under-the-covers complexity of STM bad, when the under-the-covers complexity of garbage collection is good?

There is a glaringly obvious answer to this...

A garbage collector eliminates tons of complexity from the application developer's burden, allowing the app developer to focus more on the true problem domain.

Transactional memory does no such thing. Application developers have to think about shared memory, potential conflicts, how to express them as transactions, and as mental points out: ultimately how to *recover* from a transaction failure.

On comments from Mental...

I don't know that you're really forced to deal with conflict resolution -- most of the popular STM implementations deal with conflict for you by terminating the conflicted transactions and automatically retrying (perhaps with backoff). Conflict resolution is taken care of (or, if you want to be pessimistic, taken out of your hands).

Common optimistic concurrency control techniques make matters a little sticky in that regard: you can easily end up in a situation where a long-running transaction is starved by a steady stream of shorter transactions whose successful commits conflict with it.

It's a slippery slope. New kinds of conflict resolution will be thought up and implemented. I've seen this with transactional systems like Gemstone for Smalltalk and Java. The "reduced conflict" classes take various strategies during the commit process to turn bitwise conflicts into logical successes. There is an arbitrary number of retries before the transaction service just gives up and fails the transaction. Then it is back in the app developers hands to figure out what to do.

This is not unlike the "fallacies of distributed programming" where some system attempts to hide failures that can arise through distribution. No, that illusion can always be broken somewhere and cannot be fully ignored.

Without the ability to reserve objects to the longer-running transaction (i.e. locking), it's hard to resolve such a one-sided conflict.

And that's another problem. Once this thing is implemented in software (and hardware), as Nat Pryce pointed out earlier, there is still the whole coordination thing.

All that said, STM still has composition properties that other shared-state techniques do not, and there are always going to be situations where shared state is most appropriate. I think it can be a useful technique where shared state is required, and transaction sizes would be modest and relatively consistent.

And at the end of the day I am not arguing against the mechanism per se so much as I am arguing against its widespread use. Although I do believe the number of appropriate uses is so small that 90 some percent of us should not even have to pay attention to it. My fear is this will: (1) draw attention away from getting the majority of us running on simple shared-nothing message passing systems, and (2) end up in the average programmer's toolkit as a shiny fob to get out and play with all the time.

Something that may help limit STM to those situations are its practical restrictions on non-transactional effects -- even if you're not in a language where you're forced into an STM monad, any non-transactional effects need to be idempotent because the transaction may be retried arbitrarily many times. (On the other hand, cue novice programmers wondering why the output from their giant transaction occasionally gets duplicated several times...)

Yeah. It's a slippery slope that is just way to shiny. We all get fascinated by it, end up on that slope, and before you know it, smelly messes everywhere. The alternatives are just so much more promising for the majority of cases.

It's Fiddly

"Mental" writes in a comment (i've started a new post) to my rant on transactional memory...

I was having fun implementing STM until I realized that I was able to implement the Actor model correctly in a few hours, versus several weeks for getting the fiddly aspects of STM down.

Good tag line... "Transactional Memory: It's Fiddly!!!"

Here's the entire comment because I like it...

Having written several implementations of both STM and various message-passing-based concurrency models for Ruby lately, I'm a lot less sunny on STM than I used to be even a few weeks ago.

I was having fun implementing STM until I realized that I was able to implement the Actor model correctly in a few hours, versus several weeks for getting the fiddly aspects of STM down.

The biggest immediate problem for STM is starvation -- a large transaction can just keep retrying, and I'm not sure there is a way to address that without breaking composability. ...and composability is the whole raison d'etre of STM in the first place.

Misguided: The Road Not To Be Travelled

Update:This is turning into a lot of work. Why did I start this?

Oh, I remember. Because this is a really bad feature that could screw up a lot of software for years to come.

Quote of the Day

...from Phil Dawes...

Blimey, that’s gonna cause a bit of a ruckus.

Wrong Programming Model

Not much love over at LtU. So LtU readers -- here's the gist of my diatribe -- the programming model is simply not needed and will lead to shared memory code at least as complex as the stuff we're writing today. Should everyone switch to Erlang? Well that is the general direction general purpose languages should go. Switching from today's monitors to transactional memory is not going to be a baby step either! It's a huge step. In the wrong direction. So, yeah, as long as we have to step this way or that way to get into a multiprocessor, multinode, concurrent world -- let's do step in a direction that is a good bit simpler than the one we're in today *and* far simpler than the one proposed by the transactional memory folks.

Meanwhile back at the comments...

Guillaume Germain, the author of Termite, a really nice Scheme-like, pretty much shared-nothing, concurrent programming system, comments...

I think STM has some very attractive aspects, most notably efficiency and composability. I see it as a better replacement for other low-level concurrency constructs, but not for large scale systems.

The efficiency of a mechanism may not translate into efficient *use* of the mechanism. As for composability, I think that could very well turn out to be an academic attribute. Is this something most programmers should be composing concurrent threads with? No. Absolutely not. Most programmers should not be using shared memory threads at all.

Then Guillaume comes to his senses, 8^)...

I have a few concerns about it...

The first concern is that I'm not sure how much the composability of STM will scale. If layers upon layers of transactions are built, I fear some dependencies between layers might start surfacing at upper levels, causing surprising conflicts. It could become hard to get it right. I can see misguided programmers starting to sprinkle their code with 'atomic' statements ("just in case"), a bit like one would do with 'yield' statements in a non-preemptive concurrent system. Also, 'atomic' statements could be forgotten, causing nasty bugs...

But my main concern with it is that after all, STM is still shared-state concurrency. It mixes together the control flows of programs in ways that can be hard to visualize and comprehend. Erlang doesn't have that problem, because every "connection point" of a process with the exterior is obvious and well-defined...

in the end, it seems to only solve a small part of the problem, and it doesn't really help with the actual design of concurrent systems.

rektide then comments...

the one thing i really like about STM is that good implementations are exactly like ZFS, copy on update

And hey, if you want to build a persistent file system, by all means the mechanisms in ZFS are neat. But does this automatically transfer over to main memory shared everything concurrent programming as a desired programming model for most applications? No way. I can't make that leap so easily. At all really.

And he writes...

currently, any place where data over time is relevant requires user implementation to store history, which usually means debugging tools and printf(). to me, history is just a series of transactional states, and being able to lookup old states seems like second nature. i would really like to see more Saga like transactions available in the mainstream.

And I am all for making the history of state available and first-class. Yet there are many ways to do this within a sequential process, not requiring main memory shared everything concurrent programming.

if you've got magic bullets for [distributed shared state], the world is always buying.

No. Avoid distributed shared state. That doesn't mean all concurrent, distributed problems go away. It just means they are that much more manageable. This shared everything transaction thingy makes the problem much worse than it has to be. I think the starry-eyed admirers see how hard shared memory monitor-based programming is. But that programming model should *never* have been admitted into the Java language to begin with. There are already better solutions than this proposed garble.

End

From the ACM Queue, "Multicore Programming With Transactional Memory"...

Transactional memory brings to mainstream... programming proven concurrency-control concepts used for decades by the database community... Under the hood, a combination of software and hardware must guarantee that concurrent transactions from multiple threads execute atomically and in isolation. The key mechanisms for a transactional memory system are data versioning and conflict detection.

By the way there are a few folks at Gemstone Systems who've done this far better than anyone else. They've been doing it for over two decades and it's in production in heavy use at financial institutions, factories, shipping, and so on. They've got it working in distributed shared, multi-user transactions with efficient distributed garbage collection. I pointed some researches that way. Not the ones in this paper.

That said, this would be the most tragic turn imaginable for programming in the 21st century. There is no way I would want to do this in the small on one machine or in the large in a Gemstone-like system. That is the wrong way to program concurrency for most systems.

Very wrong. And it is scaring me how shiny this thingy looks in so many people's eyes right now. I don't think it will be so long before this shows up in Java and C#.

Wrong, Wrong, Misguided, and So Wrong

We need to be headed primarily toward shared *nothing*. Sharing at the level prescribed in this paper, whether with locks or transactions, is simply uncalled for 99% of the time. Sequential processes with shared-nothing message passing should be the direction.

Get better isolation mechanisms for conceptually multiple JVMs and dotnet runtimes in a single OS process. Better yet, turn these systems into legacies and just move onto something better for future work.

You can hold me to this: transactional memory if turned out into the wild will turn out to be a *mess*.

Damien Katz writes in the comments (and since he agrees with me so well, I promote it here)...

I *completely* agree. Shared state threading is a hack built on to existing languages. A useful and fairly efficient one, but a hack nonetheless. And like raw pointers and unprotected memory, it's hard to justify it's use for most programming tasks. Compared to languages like Erlang, it's nearly impossible to justify it on efficiency or performance concerns.

Transaction memory is another hack bolted on to all the previous concurrency hacks, and somehow it's going to make all the other concurrency hacks to work reliably. Yeah right.

In another comment worth displaying up front, Dan Creswell expresses, well, let's have Dan's quote. It even deserves its own subtitles...

It's Deadly Shine'y

You're looking at premium car-wax, retina burning shine'y

I read the same article and it just frightened me. All that magic hidden under the covers, bleuch.

It's deadly shine'y because at the surface it appears like it just makes all the concurrency stuff such as locks "go away" in line with many a programmers desire to ignore such details. Couple that with the familiarity most have with transactions and you're looking at premium car-wax, retina burning shine'y.

I don't even want to imagine debugging one of these systems. You're going to be confronted with some strange behaviors due to transaction conflict or whatever and because it's all supposed to be done by magic under the covers wading in there to understand what's broken will be a nightmare. It'll make debugging from Java thread-dumps look like a holiday.

8^)

More comments. This is great.

Here's part of one from Nat Pryce...

Does software transactional memory support coordination? Or must that be supported by other primitives, such as semaphores or monitor condition variables?

From what I can tell you still have to build up from the basic transaction mechanism and the shared variables. But it is worse than that.

Most systems today should be ignorant of the "inside my OS process", "on my same node", and "on some other node". The benefits of that can be seen in Erlang, which Damien pointed out. This mechanism is still an "inside my OS process" mechanism.

Well, at the lowest level of runtime implementation of an OS process, you need some mechanisms like this. I would argue all the machinery, hard and soft, for transactions, is way overkill for that level of systems programming.

But to continue to have application programmers deal with this mess for the next umpteen years is nothing but ludicrous. As Damien also wrote, it's like continuing to have today's programmers deal with raw points and memory. No way -- very few programmers should be dealing with that level of complexity.

Nat also writes...

The interaction between distributed (and therefore concurrent) activities involves two things: data transfer and coordination. Synchronisation to avoid race conditions is just one form of coordination. Systems also need to coordinate activities that don't share data.

Yes, absolutely. Let's focus on the real problem for software development. Transactional memory is *not* it.

And now sigfpe (Dan Piponi?) writes...

it's nice to know that there are other people in the world having issues with STM.

Yes, now's the time to raise a ruckus.

And Cale Gibbard comes to the defense of the dang thing...

The major advantage of STM as a system is that it gives you certain guarantees about composability of already working systems. It's not magical, it doesn't always ensure that things will play nicely together, but it does give you far more guarantees about the correctness of compositions than previous systems have.

But Cale, there's not that much Haskell code out there yet anyway. Don't lets have the Haskell people start in with transactional memory just because there still trying to demonstrate they can do imperative programming better than the rest of us.

The world is getting ready for shared nothing, semi-functional programming. Get out of the backwater and catch up to your audience.

And if some group is going to retrofit transactional memory into some significant Java or C# system, well, they would be far better off investing that time into a simpler, shared-nothing rewrite into a language like Erlang, a simpler language like Smalltalk, or even a better shared-nothing coordination mechanism like Javaspaces.

All that low-level monitor-based garble? Just leave it will it is now. Walk away from it slowly. Turn your back, and run.

Cale again...

That being said, if you understand the additional compositionality guarantees, what exactly is it that you find lacking?

Em, simplicity? Em, elimination of totally unnecessary mechanisms too far removed from the domain problem?

And on...

Limiting shared state is obviously good -- there's nothing in this system which prevents that. However, it's a system for cleanly sharing any state which should be shared.

Yeah, right. People would abuse that mechanism all to bloody hell. If a relatively small group of programmers can develop the various soft real time Erlang systems we've seen over the last several years, there is no indication in my mind the effort required to implement and teach this transactional memory thingy would benefit anywhere near 90 some percent of programmers on this earth. No evidence whatsoever.

Other forms of communication, including those in Erlang still have this problem to deal with.

Yes, complex problems remain with implementing concurrent systems. But mechanisms like Erlang's raises them to a higher level, closer to the problem domain. This transactional memory is a false hope covering a bottomless pit that could have easily been walked around in the first place. Stay on the path.

Suppose you have a bank server with clients happily communicating deposits and withdrawals to it, and everything is working. How do clients implement a transfer between accounts safely?

First let's not turn every little data structure into a bank account transfer problem. That's just not the case. Second, these cases that really do exist should be well isolated and the mechanisms not put in every programmers' hands. Third, account transfers have been occurring quite a bit over the years without this new transactional main memory thingy. Why complicate everything for everybody, even if this were the best way to transfer funds???

Sorry, that is just a *really* unconvincing example.

Anyway, this is getting long and I probably can't do as good of a job of it as the paper can, so everyone please check it out before writing more gibberish about transactional memory.

Yep, read 'em. This is not the first time I've commented on it. Just now it seems like it is gaining momentum. The people writing the gibberish are those inventing these things without comprehending the damage it will do.