Simplest (Overlooked) Things

Core Data looks like a simple, useful data extension to the Next Step, er, Cocoa, framework. (via Bill Bumgarner)

Objective-C and Cocoa seems to be an ongoing counter-example to Jon Udell's take on Microsoft's apparent dynamic language strategy. Jon wrote...

Jim Hugunin, who created first Jython and then IronPython, may be the world's foremost expert on this topic. When I met with him recently, I asked if he thought we'd see official .NET Framework classes written in IronPython. He said that, although dynamic languages will accelerate the development of the framework, extensions written in Python will likely be rewritten in statically-compiled languages for production use. To some dynamic-language advocates that may sound old-fashioned, but to me it sounds pragmatic.

The counter-example is Objective-C itself and the way Next (now Apple) has developed its frameworks. Objective-C makes C available where developers determine to use it, but the key to its capabilities is the dynamic object language cleanly integrated into C.

Objects in the language are fully dynamic and first-class. They enable C and other static languages to participate easily in the object-oriented frameworks without extensions or modifications to those languages. But they also enable dynamic languages to participate easily as well.

Witness the really neat integrations of Python, Smalltalk, and Lisp with the Next Step frameworks. No contortions or compromises, just simple, straightforward integration of reusable frameworks into your language of choice, static or dynamic.

Objective-C and Next/Apple's use of it is underrated and overlooked in its technical simplicity relative to the struggles of dynamic languages in the C++, C#, and Java environments. Notably, this approach dates back to Next's original effort around 1985. This is a twenty year old approach, which satisfies those of us who live in the past. 8^)

Spreadsheets and Databases

Over on the new Smallthought blog there is a conversation about spreadsheets, databases, and end user programming. I added this in a comment, but want to capture it for my own purposes and your enjoyment...

Databases — I have a friend who made a living for a good number of years on a dbase application he wrote for small businesses. Over the years he added more capabilities. He’d go into a new customer’s site, install it, tweak it, maybe implement some new things, and then take the new stuff back to previous customers that may want it. He had no formal software development education or even formal business education. He was a tinkerer, built his own airplane, his own house, etc.

Another story — a friend, MBA in finance, capable of doing decent things with Excel, tried to implement an Access application to track a small golf tournament he was organizing. Nothing ambititous at all. Access presented him with a number of “formal” database concepts that just did not matter to him. He go some things running but mentally associating various tables and forms was not immediately obvious to him, or to me when I looked at it. Maybe it was just us, but even I did not enjoy trying to get Access to do relatively uncomplicated things.

There is a spectrum of choices between a traditional spreadsheet and a traditional database. Perhaps a more usable system in this space would allow someone to begin working more like a spreadsheet and allow shapes to emerge and be supported more like a database, but not expose that support in traditional database terms. The support for those shapes should be more identifiable by the end user programmer without a 20th century software education.

New Lisp

Ted Neward wonders...

Is Ruby the next Lisp? And if so, does that make Ruby a "second chance" for widespread acceptance for Lisp?

I would not say Ruby is the next Lisp. Rather, Ruby is one of the most recent Lisp-like languages. Python is a Lisp-like language going back a decade or whatever longer.

Smalltalk is even a Lisp-like language, as has been noted recently, and Kay states Smalltalk was directly influenced by Lisp. (pdf)

Can I learn to be a better Ruby programmer by learning Lisp, or vice versa?

Yes. For that matter, one can learn to be a better C programmer by learning Lisp. Or Ruby.

If one of Lisp's strengths is writing programs to write programs, does that make Lisp a natural candidate for building DSLs?

Absolutely. And has been in practice for 40 years or more. Literally.

And, again, if the Ruby == modern(Lisp) equation holds true, does that make Ruby a candidate for building DSLs?

Ruby would not be *bad* for building DSL's. The difference with Lisp is the syntax and syntactical tools are not as good for doing this in Ruby as they are in Lisp.

DSL's are best built in Lisp and Smalltalk, somewhat less so in Ruby and Python. But Ruby and Python programmers could argue that pretty well with me. I think I know both of them pretty well but not so well as to say I know I know both of them pretty well.

Python Moving Into the Enterprise

From Slashdot...

"Seems that Python is moving into the enterprise. At the recent PyCon it has become apparent that it's not just Google, GIS, Nokia or even Microsoft anymore. The article points out that Python is increasingly becoming a perfectly viable and even preferred choice for the enterprise. More and more companies are looking at Python as a good alternative to past favorites like Java. Will we finally be able to code for living in a language that's not painful? Exciting times!"

Evolutionary

"There is no such thing as a non-self organizing system." -Esther Derby(?)

Yep

"Poor management can increase software costs more rapidly than any other factor." -Barry Boehm

Buddies, ACL's, and Capabilities

Tim wrote some time ago about making everything network aware, including...

• If you assume ad-hoc networking, you have to automatically define levels of access. I've always thought that the old Unix UGO (User, Group, Other) three-level permission system was simple and elegant, and if you replace the somewhat arbitrary "group" with "on my buddy list," you get something quite powerful. Which leads me to...
• Buddy lists ought to be supported as a standard feature of all apps, and in a consistent way. What's more, our address books really ought to make it easy to indicate who is in a "buddy list" and support numerous overlapping lists for different purposes.

Rather than ACLs, I'd like netware to provide me a list of actions it's capabable of doing on the network, along with an easy way of composing new actions. Then I'd like to make several groups of those actions from all my netware. One grouping for myself, others with decreasing capabilities for family, friends, associates, and the world at large.

My buddy list would then include for each member the groups of capabilities I have granted them. As relationships change I would then be able to increase or decrease their reach into my world.

I've not followed the e-rights world for a while. I wonder where this is going.

Triples, Quads, Quints?

Bill de hÓra writes about RDF, triples, and quads. I think the point is RDF representa triples (X-Y-Z) but he wants to introduce another party into the relationship, e.g. he wants to distinguish (A-(X-Y-Z)) from (B-(X-Y-Z)). Apparently RDF cannot do this concisely, i.e. you need more than A, B, X, Y, and Z in order to denote that A and B relate to the same triple (X-Y-Z) in each in their own way.

Triples, quads, etc. I have to say I've not used RDF at all explicitly.

I guess "quads" are intended to support the (A-(X-Y-Z)) and (B-(X-Y-Z)) relationships concisely.

But, the engineer in me ignorantly wonders, won't we soon run into a desire for a fifth party in this relationship? Why stop at quads?

Quints, hexes, septs. How many is enough?

I wonder if we need a concise way of representing many occurrences of Many-Many relationships. Something like a Star Schema would simplify bulk Many-Many relationships. Bonus, it implies certain expectations about the contents of the relatonships, i.e. you can speak of measurements, occurrences, hierarchies, allocations, headers / details.

Star schemas can be seen as a shorthand notation for many implied individual RDF triples. Bonus, they are proven to be incredibly scalable and support high performance with fairly simple database implementations. The column-based storage approach goes back at least to the 1970's. Another example is Kdb, although "array based" and columnar, it's not explicitly star schema based.

I'm just thinking RDF triples are probably too primitive, and thinking out loud about what might be more expressive. Mostly I am confused about a triple to quad to (?) progression trying to represent increasingly complex relationships. Star schemas encourage a discipline, style, and simplicity into complex relationships that can scale and evolve.

I'll have to learn more about the triple / quad issue and think about this before I can become more coherent on comparing these modeling approaches.

JINI and JavaSpaces

JINI (and therefore JavaSpaces) in action, not inaction, via Tim Bray...

Jini has the fingerprints of Bill Joy and Jim Waldo all over its architecture, and these are obviously two of the Real Smart Guys. Rob Gingell, another one of them, said “Those who do not use Jini are doomed to re-invent it.”

Jini is not just a theoretical triumph; it’s being deployed in large mission-critical applications at places including Orbitz, Orange (the mobile-phone company), Raytheon, and the U.S. Army.

Must (Try) to Understand This?

Sean McGrath writes about "must understand" rules in loosely coupled message passing...

Whatever about the pros and cons of REST versus SOAP, I think it is abundantly clear that the mustUnderstand model [1] is a key concept in developing loosely coupled systems that can evolve independently.

I need April 2 asap.

One thought this does provoke though, for real, is that a sender probably has no business telling a receiver what it must understand or to what degree. I'm all for ways for receivers to describe, more or less formally, what they will understand and how they will understand it. Senders just send messages and set their expectations as best as possible. Hopefully both senders and receivers have robust fallback capabilities.

They Are Principles

Kent Beck writes...

If we abandon our principles at the first sign of trouble, our principles must not be worth much.

I think integrity and accountability are particularly important in a climate of fear.

Democracy Inaction

John Robb wonders...

What we may end up with as part of this push towards "democracy" in the Middle East is civil war. Lebanon, Iraq, and Palestine are all on the brink of it now. Are we better off with this?

Yeah, I think the next chapter is still being written while people rush to chalk up an "ends must justify the means" scenario for democracy breaking out in the mideast.

There are so many directions this could go. Think of this as a complex system that is not very well understood with more than a few elements of chaos.

This is not a "guzinta-guzoutta" simple equation. And no, Bill Maher. I agree Bush stumbled into something, but a fresh pile of democracy in the mideast is not yet clearly the pile he stepped into.

That's the Way of the World

"That's the Way of the World" has been one of my favorite songs for about 30 years. Not one of EW&F's funkier tunes though. "Shining Star" too. A little bit funkier.

(Blaine Buxton likes funk.)

Hubris

Unfortunately, Truly says, the prevailing wisdom at the Pentagon is that "fuel efficiency is for sissies."

(via John Robb)

Hubris. Marines don't eat granola, they just kick the shit out of haj's. Scary when the stereotypes show up in reality.

What Goes Around

(via John Robb)

The MS 13 sprang up in the late 1980s, created by the children of Salvadoran immigrants who fled to California during a bloody civil war.

Can we not think in terms of systems? Everything is compartmentalized and treated as independent, discrete actions. Not only does every action have a reaction, but the systems we are creators of, and participants in, are far more complex than we ever think about in the moment.

Has science taught us nothing about society?

Know(n)Now as AJAX

From FoRK on the AJAX controversy...

"Yeah, I liked that idea the first time around.
When it was called 'KnowNow'."

See also mod_pubsub.

We Build Our Own

The misunderstanding about XP and "doing the simplest thing" is more often than I'd like taken to absurd extremes. For example I encountered a developer recently repeating a claim that an XP team would not use a framework that had been built to simplify a certain style of software development.

Certainly the XP team would want to understand their experience with the framework, the maturity of the framework, and the nature of the problem's suitability for the framework. But not use it, period?

Gack.

So how far does this go?

"I'm not sure we need a computer for this project, but when we do we'll dig up the coal to forge the steel to make the rack to mount the motherboard."

"I'm not sure we need a high-level programming language. We'll use assembly language and see what patterns emerge, then let the higher-level language evolve."

I think there is a spirit and a letter to the law, and the same goes for XP values. Just work in small pieces with concrete feedback. Avoid too much prediction and too many assumptions. Common sense works wonders.

Whence Dynamic Languages

Ryan Tomayko projects what might be necessary to get better late-bound languages acceptance in early-bound organizations...

I think we need something to break through on one of the VMs if we're ever going to move this into the enterprise. There's no way I could bring Python into my place of business on any serious level. This really comes down to it not being blessed as Enterprise Class (blech!) by Sun. It seems we need the VM to get in the door and then maybe we can just move quietly toward CPython with a Java/CLR bridge?

Maybe. The IT industry seems kind of in a "standardization" phase still, following the irrational exhuberence of the "anything goes" phase of the dot.com days. That can translate into the desire to choose one language to standardize on developer resources. I'm not sure getting CPython in would be any more difficult than Jython or IronPython.

We used to have these things called skunkworks. Do such things exist anymore where you can demonstrate value before being judged prematurely on cost? I think maybe but you may need radar detectors along the way.

Frameworks and Plug-ins and Loose Coupling

From ACM Queue about frameworks and plug-ins...

This article identifies some of the concepts around the foundation of pure plug-in architectures and how they affect various stakeholders when taking a plug-in approach. The intention is to discuss the general issues involved, but the article also presents some lessons learned from Eclipse (www.eclipse.org) to better illustrate some of the concepts.

The main problem with a framework like Eclipse is that so many unnecessary dependencies are forced on plug-in developers. You have to buy the full Eclipse approach to get even half the benefits.

What lessons have we learned in the last 15 years of building frameworks like this? Are we having trouble understanding what is important, what has been learned already in the software industry?

Why build a framework like Eclipse in the 21st century when you could build a framework like FIELD which was built in the 1980s? Certainly the full benefits of Eclipse could be built *using* an approach like FIELD's. Plug-ins that did not or could not support the full benefits of Eclipse would then have fall-back positions. New approaches could evolve that stretch beyond the imaginary boundaries of the current Eclipse framework.

What is it about FIELD? Well FIELD is based on integrated components loosely coupled via asynchronous message passing.

That may ring a bell for 21st century programmers.

Reconciling Modal Web Apps and Rest

Mark Baker comments on how continuation-based modal web applications fit the REST style...

"This is how most web applications work.". Actually, it isn't. Most Web applications are stateless. Of those that aren't, the vast majority are only stateful for authentication purposes.

Here's how I've started to view the relationship of the modal web app style and the REST style:

• Work-in-pogress interactions seem to be programmed AJAX-style, GOTO-style, Modal-style, or "richer-client" style.
• Modal web applications simplify programming controllers for work-in-progress over HTTP.
• Continuation-based interactions are not necessarily RESTful.
• As work-in-progress is "published" those pieces of work become resources with various representations, etc.
• Publishing (perhaps partially as a "proposal") completed work as resources can be RESTful.
• Accessing (perhaps partially) completed work as resources can be RESTful.

So I think modal and RESTful can be seen as somewhat complementary styles or at least aimed at different purposes. Perhaps they don't have to be completely unified if they meet their purposes with little risk of interference between implementations.

Extreme Refactoring

Again Keith Ray, this time on design, refactoring, and language choice...

Brian Button decided to explore refactoring by taking the Video Store example from Martin Fowler's book and refactoring it into methods one line long, no private methods, and aggressively removing all duplication, particularly duplication of iteration over containers.

His starting point (and Fowler's ending point) is the class named "Customer", containing a list of rentals. It has private methods for calculating rental cost and "frequent renter points", and a large method for printing the customer's statement. Brian's ending point is seven classes: Customer, RentalStatement, RentalCollection, Collector, LineItemPrinter, FrequentPointsTotaller, RentalCostTotaller.

If he had been programming in Ruby or Smalltalk instead of C#, he probably could have met his goals with only first two classes. Blocks and iteration methods built into Smalltalk/Ruby collection classes would have eliminated the need to create the other five classes.

Agile Data

Keith Ray writes about agile data...

He asked audience members to raise their hands if they could make a simple change (like renaming a column of a table) and re-deploy their database and associated apps in the same day. Only one guy, in a large room filled with database people, raised his hand (I bet that guy worked at Yahoo, Amazon, or Google.) That guy's DB had terabytes of data and 37 apps depending on it, but he could deploy a change in less than one day.

Ambler also asked audience members to raise their hands if their database and associated apps had extensive automated testing. The same guy raised his hand, and maybe one other.

Bill Maher

I saw Bill Maher live last night. He was recording an HBO special and DVD in Portland. Maher is a good observer and funny commentator of American politics, religion, and culture. He is irreverent and holds nothing back. He's more of a traditional humorist like Mark Twain or Woody Guthrie than Jon Stewart. But he goes beyond Stewart and Lewis Black topically and stylistically. (I missed seeing Black on stage when he was in Portland a few months ago.)

These three along with Harry Shearer are four people worth listening to for insight into the state of the union with varying degrees of subtlety and style.

Boo and IronPython

Edd Dumbill tried IronPython and now prefers Boo. Another Python friend was thinking of trying Boo too. The problem I have with that is I'd rather the bulk of my code not be tied to a specific platform, which Boo seems to do. Boo runs on MSFT dotnet as well as Mono, but still requires the CLR platform.

Edd raises significant concerns about IronPython. Rather than a somewhat handcuffed Boo, I would think a Python programmer would be very well off with regular CPython and bridging to dotnet via the Python.Net dll's.

Attitudes Toward SISC and Continuations

Chris Double used SISC running in the JVM to build a continuation-based modal web server. He wrote of the manager's and developers' experiences...

This approach has worked well for me and the performance is quite usable. Although I do get constant requests from the project manager to find a way to rewrite things in Java as he never was a fan of using Scheme and is concerned it may put customers of the system off it. The developers on the other hand seem to have enjoyed learning Scheme and consider it an advantage over Java.

Listen.

The Secret Sauce

(via Blaine Buxton) Eliot Miranda divulges...

VisualWorks and VW/GemStone combinations are used in sectors such as cpu manufacture, container shipping and derivatives trading on a world scale (i.e. they handle a substantial fraction of the world's activities in these sectors). But for nearly two decades the corporations who have built these applications have viewed their use of Smalltalk as a strategic advantage, and hence prevented the vendors from using the applications in marketing material.

Python Performance

Good news, but let's not get too sweaty over this...

IronPython is markedly faster on PyStone than CPython.

...remembering the CPython VM does not really push the envelope of VM techniques.

IronPython at PyCon

Technorati has dug up a list of items on IronPython at PyCon this week. I've not consumed any of them yet, but hopefully they will be enjoyable.

It looks like the new release of IronPython depends on the Yukon CLR. So for now it looks like CPython with the Python.Net assembly bridge is the approach of choice for "real" uses. With a bit of care Python code could then port over to IronPython.

Not sure what the language level of IronPython will be vs. CPython when Yukon is released.

Alan Kay (again) on Software Engineering

Kay's interview has made the rounds more than once, but with recent posts on software as engineering or craft, let's go back to the guru...

If you look at software today, through the lens of the history of engineering, it’s certainly engineering of a sort—but it’s the kind of engineering that people without the concept of the arch did. Most software today is very much like an Egyptian pyramid with millions of bricks piled on top of each other, with no structural integrity, but just done by brute force and thousands of slaves...

I would compare the Smalltalk stuff that we did in the ’70s with something like a Gothic cathedral. We had two ideas, really. One of them we got from Lisp: late binding. The other one was the idea of objects. Those gave us something a little bit like the arch, so we were able to make complex, seemingly large structures out of very little material, but I wouldn’t put us much past the engineering of 1,000 years ago...

I finally understood that the half page of code on the bottom of page 13 of the Lisp 1.5 manual was Lisp in itself. These were “Maxwell’s Equations of Software!” This is the whole world of programming in a few lines that I can put my hand over...

All of these ideas could be part of both software engineering and computer science, but I fear—as far as I can tell—that most undergraduate degrees in computer science these days are basically Java vocational training...

In a history of Smalltalk I wrote for ACM, I characterized one way of looking at languages in this way: a lot of them are either the agglutination of features or they’re a crystallization of style. Languages such as APL, Lisp, and Smalltalk are what you might call style languages, where there’s a real center and imputed style to how you’re supposed to do everything. Other languages such as PL/I and, indeed, languages that try to be additive without consolidation have often been more successful. I think the style languages appeal to people who have a certain mathematical laziness to them. Laziness actually pays off later on, because if you wind up spending a little extra time seeing that “oh, yes, this language is going to allow me to do this really, really nicely, and in a more general way than I could do it over here,” usually that comes back to help you when you’ve had a new idea a year down the road. The agglutinative languages, on the other hand, tend to produce agglutinations and they are very, very difficult to untangle when you’ve had that new idea.

One clear implication is that language you use has a significant impact on your ability to do software engineering. Your language is your most highly used tool.

Do Good for America - Use Lisp

(via Lemonodor)

François-René Rideau implores...

The obsession with parsing and syntax in computer science curricula is hurting America.

SPLJ Part 2.

Mark Baker suggests...

The resource oriented (RESTful) solution would exhibit greater degrees of architectural properties such as scalability, visibility, and simplicity than the service oriented solution, because RESTful solutions embody architectural constraints which induce those properties.

SPLJ

In the raw...

Applications remain largely discrete things, whether they’re wired together with RPC, SOAP or REST. Only slightly, (particularly around the blogosphere) are we seeing the pieces joining together.

Preventing Death

Somewhere around 13,000 children die of hunger-related causes every day around the world. I wonder when the US congress will subpoena them to come testify. Certainly they will get at least a snack on the airplane.

Certainly the congress can do something to save at least one of those lives each day. That's at least 364 more lives saved each year than can be saved in the case in Florida.

How much longer can our "culture of life" continue to ignore millions of deaths a year?

New for Who?

Depends on what the meaning of "new" is...

I firmly believe that the new advances in languages will borrow heavily from languages invented three decades ago such as Lisp and Smalltalk.

According to Alan Kay, computer professionals were smarter back then, because computing was still an esoteric technology. Now, we have a pop culture when colleges are essentially vocational schools.

...from Wesner Moise.

The Damage Done

James Robertson hopes for the best...

I guess the only good things about binary XML and WS* is that the people involved aren't doing greater harm somewhere else.

Seems like a fairly wide swath of damage from this vantage point.

I have a feeling things are just getting started too. Look out. CORBA took a long time to get to a moderately usable interoperable condition. For better or worse (we'll see) that increasing utility and popularity ultimately led to its current status, dying a slow death.

Math

(via Paul Graham)

I found, when I was studying mathematics, that 2 things were true: (1) the teacher was not too good and (2) the book was not too good. So I would always buy a half-dozen books on the topic and try to get the full picture by reading the same sections in each book. The combination helped me understand much more than the sum of the content. Also, I was never opposed to reading something as much as 10 times until I squeezed everything out of it.

I had the same problem, but I was not as diligent in overcoming it.

Old World Distributed Computing

Steve Loughran writes...

By adding all the functions of CORBA to SOAP, WS-* replicates the old world of distributed computing, only with underpinnings (HTTP, XML) not designed for that particular role.

Opposites

James Robertson writes about problems with the "common" language runtime. Some of the comments are suggesting you gotta take the good with the bad. I don't buy it and don't believe the commentators know about which they speak. Read on...

If you want the best of Smalltalk (or Python or Lisp) and access to dotnet as well then use a bridge between runtimes. (Smalltalk bridge, Python bridge, Lisp bridge)

I see no reason that all languages have to run in a crippled VM like the CLR or the JVM when you can get to these via bridges that keep them at arms length. The alternative is you cripple the language and lose most of the tools that make you productive in the first place.

I think any "common language runtime" has to be built at the foundation on good technology. Crippled languages can then be built on top rather than vice-versa. This was demonstrated on Lisp machines where Fortran, C, Pascal, Ada, etc. compilers were ported.

Funny that when good languages are ported to crippled runtimes, the good languages suffer. But when these languages were ported to the Lisp machine, the crippled languages became *more* capable than their original designs.

I've used C on the Lisp Machine. I can say, and everyone I know with the same experience agrees, this is the *best* environment for developing C programs. The same is never true for good languages ported to the CLR or JVM.

Maybe they'll get there some day. Maybe.

GoDaddy and Free DNS

Like Dave I use GoDaddy for inexpensive domain registration. I have for several years. I use it for sites they host (e.g. my son volunteers time and helps with web stuff for Arin's House) and sites I host elsewhere.

Dave's wondering about DNS for sites hosted elsewhere. I use ZoneEdit.

Croquet or Cannon?

I've got to put Croquet on my list of things to do. Every so often I see another post and look around, and wonder why I haven't tried it yet.

Here is an example of a window containing Flash content. Flash and other media forms may rendered on any polygon or combination of polygons within a scene. It should be kept in mind that it is not at all necessary for the frame of any window to be rendered. Therefore it is possible to have flash animations or media output of any size be rendered in any orientation or configuration within the Croquet space.

Being There

Update: added reference for W.Cunningham quote.

Bill de hÓra writes how keys to software success can be found in the business and operating models you adopt. Heed this advice.

We can be successful with the latest "service orchestration" software or with Smalltalk and relational databases, both decades old. (I think the latter may provide an advantage.) The keys are in how we form teams, communicate, and do work.

I would ban the phrase "service-oriented anything" until my organization understood this paragraph...

The reason in the past we needed to worry that 'it will never X' is because there is an assumption built into the procurement, requirements and development phases that systems of this kind get finished and that's it - no more money for active development. You have one shot to get it right. In turns out that one shot is an operational myth and a dangerous one at that - business systems are never really finished, but thinking that they are radically affects thereafter how budget is dispensed and for what.

Test-driven development is proving to be the operating model for continuously adding value to a system. We need equivalent models for the business that surrounds development.

• Whole teams.
• Communication.
• Frequent feedback.
• Test-driven.
• Continuous integration.

Ward Cunningham illustrates that the burden will land on middle and upper management as developers get their act together...

A common management mistake is to keep everyone busy by giving them different things to do. This basically tells people to work alone, and that is an easy way to fall prey to exactly [all] sorts of fears.

Agile asks the team to work together on only a few things at a time. That way, once someone gets over the wall, they can help everyone else over after.

Customers ask for a lot because they don't think they will ever get a chance to ask for more. Regular delivery of capability that doesn't go away will erase this fear in customers.

Dynamic Languages on the CLR

Jason Zander has a post that mentions dynamic languages on the Microsoft CLR. But don't get your hopes up yet.

Dynamic Languages

We had several questions about our philosphy on dynamic languages. I think I'll actually write a separate blog post on this one because it is such an interesting subject. A few quick notes for now:

I believe we already have a great platform for dynamic languages. You get the GC, exceptions, interop, and access to a huge set of libraries for free when you target the CLR.

That's nice marketing but not much information. Let's hope for the real post, and see what else he says here...

In Whibey we add Lightweight Code Gen (LCG) which further eases the ability to author the languages.

Alright. There's something. I wonder how the code generated supports dynamic languages.

Because we also built generics deeply into the runtime, you can leverage that support dynamically at runtime without having to "fake it out". This is a distinct advantage over techniques that use "erasure" to replace syntax with expanded code.

How is this an advantage for dynamic languages? "Because all our other programmer have to deal with rigid, unnecessary language constructs, we've preserved them at runtime so you can figure out how deal with them too!"

Thanks for that. What else have you done to make the CLR "compelling" on the Windows platform? Continuing...

Because you target our MSIL and type system, we also enable easy access to tons of tools that work across langauges seemlessly.

As opposed to the tons of cool tools that have been working in Lisp and Smalltalk for literally decades that do *not* require a "type system". Great fun... "Use dynamic languages in our world and as a bonus you get to understand our rigid type system."

By now I am feeling really frustrated and pessimistic. I am learning nothing about support for dynamic languages and only about how those languages will have to work with all the other rigid conventions of poorer languages. No news is good news compared to this.

I expect in the future to do a Project 7 style approach to adding more improvements to the engine, just like we did in the late 90's to form the current engine.

Any ideas what kind of improvements there will be for dynamic languages, your topic of conversation?

And finally a quick plug for Jim Hugunin who is doing a keynote at Pycon next week.

Finally there may be some information. Hugunin apparently has not posted since October. He's not participated in the IronPython email list. And recently someone *outside* Microsoft created a sourceforge site for IronPython fixes.

Maybe Sun can do better.

Programming

James Robertson writes...

There are a lot of developers who will have a really hard time letting go of the idea that all inbound CS people should take a compiler class. Ask yourself though - how many people are doing that kind of work, and how many are creating what amounts to glorified reporting interfaces?

I actually believe all programmers should have several compiler courses.

As it turns out a programmer friend with a mechanical engineering degree has been taking EE and CS courses over the last couple of years. He just completed a compiler course. He created an interesting SQL compiler for file systems as his final project.

I was reminded of the deeper value of learning the basics of compiling. We talked a few times over the semester, and his discovery process was familiar. The surprise he found building compilers are the regularity, and the patterns that decompose complex problems into combinations of simpler ones. Of course "back in my day" we did not have names for all these patterns, but the lessons are the same. The lessons learned building compilers can be applied to many other programming problems.

More advance compiler courses teach about the mathematics and engineering of representations and transformations of data. Denotational semantics, continuations, even "types" (gasp! I said it) are worthy topics that all programmers should understand.

Distributed Build Architecture

Vincent Massol proposes a distributed build architecture using JavaSpaces...

I personally prefer this architecture over one where the orchestrator directly sends build requests to build agents as I find it more scalable and more flexible.

Yeah, and robust as well. Just watch for that place where the build agent does a 'read' for a request. It should be a 'take'. Before the take though it should 'write' it's intent to take perhaps including an estimated time of completion. Then if it fails, another process can monitor the results and restore the request if it suspects or detects the agent failed to complete.

If the request included an estimate of time to build then build agents can predict their loads and which requests they should accept. These estimates can come from actual builds and improve themselves over time. Many possibilities for coordinated processes to evolve on their own through "metadata".

Would you like such a distributed build system?

Actually I'd rather use tools that did not require such massive builds, but the idea applies generally to many kinds of transformation activities. This is an ideal example of services built using simple, general coordination mechanisms.

You can build *all* kinds of coordination patterns using the simple Linda-like vocabulary.

Urp or ERP?

Notably the new Windows product VP has a background selling to customers with indigestion.
=8^o

Worth the Investment?

Alan Knight provides another story to guide your investment in software choices...

I spent last week in Cincinnati, at Cincom headquarters, in a variety of meetings. The meetings were quite good, as meetings go. The last day was spent in meetings with a large customer who has some very important systems dependent on VW. They had some very nice things to say about Smalltalk being the most stable piece of their information systems.

But it's so... different.

Yes, it is. And that's a *good* thing.

Coordinating Webs of Information

More from Jon Udell...

On today's Web, security, asynchrony, reliability, routing, and proxying are, in general, aspects of the data communications layer. On tomorrow's Web they'll be aspects of the application layer, and we'll operate more like weavers of business processes than plumbers of networks.

I'm not sure I understand this vision, but I'm also not sure I agree with my understanding of it.

The best way to build a web of information and processing appears to go back to blackboard architectures in the 1970's. These ideas also proved to be agile for a more broad set of coordination problems. Today's web is showing an approach along these lines may be effective at scaling to the world for machine-machine as well as human-machine.

I thoroughly doubt "we'll operate more like weavers of business processes than plumbers of networks". We need to hide business processes as much as possible, favoring coordination via semi-structured data showing up here and there on flexible networks of cooperating-yet-arms-length processes.

BKM's

Jon Udell asks...

Why must we make everything into a rivalry? Why can't we just enjoy the best of all worlds? That's how I use programming languages, that's how I use operating systems, and that's how I intend to use this stuff too.

Because we are confused. Or at least I am. About what *is* the best of all worlds. And there is a cost to all this. Combining Python, Smalltalk, and Java is one thing. But what are the Pythons, Smalltalks, and Javas of this new industry?

When you see so much, well, stuff, being flung at the wall, why wait to see what sticks? Go with what seems simple enough and works already. Doing that much is hard enough. Let others invest in prediction.

Prediction is very difficult, especially if it's about the future.
-Niels Bohr

Accretion vs. Evolution

Also from James Robertson, on accretion of features in popular programming systems, if not most software.

Java/JVM and C#/CLR evolution is interesting to watch as compared to the history of Smalltalk and Lisp. Consider how drastically Lisp changed from 1960 to 1970 or so, and the same for Smalltalk from 1970 to 1980. They had real developers using them in real situations, but the pressure to stagnate or the pressure to accrete cruft rather than replace it was not there.

Not only did Lisp and Smalltalk languages and tools support evolution as well as backward compatibility, the culture and organization encouraged even drastic swings in features.

J/J and C/C do not have those benefits, neither technical nor organizational. They are ultimately doomed, so the word to the wise would seem obviously to get off those platforms ASAP. Invest in systems that have proven themselves already for decades.

Do You *Like* Handcuffs?

Ramon Leon writes...

With or without Seaside, Smalltalk needs to return to the mainstream. The current crop of mainstream languages, pretty much feel like handcuffs in comparison.

Numbers Games

Jono Bacon writes...

Bear in mind that the headlines may simply be an illusion in an industry where the real success stories are happening all around you.

Yeah, but...

James Robertson is looking for a MSFT course correction...

You know what? As far as I'm concerned, MetaWebLog API sucks, Atom is pointless, and SOAP is a huge duplication of CORBA on port 80. But you know what else? It doesn't matter what I think, because I have to support these things anyway. Whoever runs Product Management/Marketing for IE - it's time to fire their sorry butt, get someone new in there, and slap them around with a cluestick until they get it. I'm tired of paying the IE penalty for CSS, and so is everyone else.

Yeah, but XAML and Avalon are going to be so, um, compelling.

Really.

Reference Relevance

As seen in an email list for dotnet programming...

Passing a reference type by value lets the callee manipulate the object that is referenced. Passing a reference type by reference lets the callee manipulate the reference to the object itself - to change where the caller's reference points.

Such are the things one gets to sweat about when the language is not as simple as possible.

Yeah, I know.

And ouch...

One thing to keep in mind when is that the 'ref' is very important when you cross application boundaries. Specifically this applies to .NET Remoting. If a parameter is labeled with 'ref' it will copy the changes back to the caller. This differs significantly from calling a member function locally.

We're Waiting Already

From CNET...

"It's a little depressing that developers are just now wrapping their heads around these things we shipped in the late 20th century," said Charles Fitzgerald, Microsoft's general manager for platform technologies. "But XAML is in a whole other class. This other stuff is very kludgy, very hard to debug. We've seen some pretty impressive hacks, but if you look at what XAML starts to solve, it's a major, major step up."

Please actually *deliver* something newfangled, and show us why its significantly better than the older technology. Then you might generate more buzz than Google.

But then again what about Firefox? Opera? Safari? Charles... I think you've got a lesson or two to go with XAML. You may know more than I do about how you'll win this one, but either show your entire hand or get back to work.

(via Jim)

Asynchronous SQLServer Messaging

High on my reading list, from MSDN on SQL Server 2005...

"Get an overview of how the new Service Broker functionality in SQL Server 2005 enables faster and easier development of SQL Server applications that rely upon asynchronous, queued operations."

The Future of Data

Martin Fowler writes...

If applications were more aware of version control then I think we would see more interesting things being done with it. I hope that a solid, usable, open source tool will spread the usage of version control to a wider audience - after all we are no longer short of disk space.

I have to agree, and think Subversion's support for WebDAV and API could help get there. Thinking back about a year, where I speculate...

From working on systems in a half dozen domains I have come to the conclusion there are three simple patterns we should lean on for making documents and messages persistent.

• Tuples Spaces (think Ruple Forums) for in-progress, state machine-like transactions and collaboration.
• Versioned document trees (think Subversion's file system) for long-lived, shared, document editing.
• Star Schema-based storage (think Sybase IQ, a simple, low-maintenance, scalable database technology) for read-only transaction history and analysis.

I deeply believe (it is my hypothesis) that these are the only patterns you need, and the implementations going forward can become far simpler and adaptable than all of today's cruft.

Emacszen

"Danny, after oscillating between NetBeans and Eclipse for several years, gets a surge of Zen and orders the Emacs Manual"

Success'ing

Ryan Tomayko writes... From the beginning, WS-* has been approached incorrectly. The approach that we're advocating is to first embrace and understand existing web architecture and then to gradually enhance (constrain) it to meet the needs of new requirements. We advocate this not because we think it's a better way to be successful, we think it's the only way to be successful. [That's some claim! Is "success" even defined in this case? -Patrick]

An example of the approach we're advocating is Bill de hÓra's recent proposal for reliable messaging over HTTP, (HTTPLR). The difference between the HTTPLR approach and the WS-ReliableMessaging approach is that Bill's proposal builds on simple proven web architecture without requiring additional apparatus.

This is interesting even inside the firewall.

April XPDX Code Sprint

The April XPDX code sprint (Extreme Programming in Portland, Oregon) is on for April 9-10.

We'll be using Python in several flavors: CPython, Jython, maybe IronPython, and Python.Net (CPython with the dll's to integrate with dotnet).

Also in attendance will be Smalltalk. Seaside with Squeak and VisualWorks, at the least.

The official April meeting, the following Tuesday evening, will be a retrospective on the sprint via Diana Larson.

Amazonian Art

Try these (via Lemonodor)...

• Lisp
• Smalltalk
• Python
• Ruby

D-Bate

Stu sez to Sean...

Does this cost a lot of money? Depends on what you define as "a lot".

And Avram Levinski adds...

Personally I think the whole WS-* is a gambit extended to outflank Java.

Before Tom closes with...

Stu mentioned: "Architecture groups are becoming keepers of the canonical schemas."

Kiss of Death. Just pack up and go home now.

Recombinant Gems

Carlos Perez keeps on putting out gems (with links to past gems as well). I am really enjoying these...

One key to a solution is the need for a recombinant architecture. That is a framework that will allow rapid assembly of components like we never seen before.

Apps

Danny Ayers writes...

I’d opt for a triplestore and Python, but even today we’re only just starting to see shrinkwrap apps moving to non-proprietary languages and interfaces.

The Race is On -- Where's the Goal Line?

From Infoworld...

What I thought would be a Java lovefest turned into a "How do we fend off Microsoft?" debate.

The consensus was that .Net is a strong technology but that Java provides the advantages and disadvantages of more community participation. While more amenable to community contributions than perhaps .Net is, it is thus more difficult to build a consensus on improvements than the monolithic .Net platform, which is controlled by Microsoft.

For the time being, both Java and .Net shall remain the dominant platforms. There may not be a victor, but continued coexistence for years to come.

I'm curious whether the interesting race is the emerging-yet-not-fully-realized race to "fully" support dynamic languages on these platforms.

Debasing

Danny Ayers writes...

It seems to me that a very large proportion of today’s user-oriented Web applications could run as CRUD. We’ve already got a fair idea how to separate content from presentation. But the business logic is usually all tangled up in a combination of SQL tables (and maybe triggers etc) and objects/procedures or other relatively hard-coded representation...

A part of the reason that middle layer has tended to grow over time is the mismatch between the logic provided by the database and the real-world data it has to manage (...much of this is to do with the design of SQL databases, not Codd’s model). Another problem is... up-front design of database schema... There isn’t much flexibility for continuous development at the back end, so things get pushed forward. Not very eXtreme.

...replacing the back end relational database with a triplestore, SQL with SPARQL and schemas with ontologies. Bit of RESTful HTTP and XML for the transport, bit of XSLT for the presentation layer (with HTML forms, XForms, XMLHttpRequest, whatever you like). There’s no reason why the whole lot couldn’t be generated from a set of statements, although a bit of PHP template hacking may be needed in the interim…

Yes, this is the old Semantic Web chestnut rewritten for a new generation…or rather, the generation that got quicker results with Dbase years back…

The Road Ahead

Mitch Kapor writes...

The challenge now is whether Ray and Groove, which represent forces of architectural innovation, can have a successful impact at Microsoft, which after all, is a large (58,000 person), middle-aged (30 year-old) company. It's hard to know whether the loss of nimbleness due to size and age is a greater challenge to Microsoft than is open source.

Groove and Croquet

Hugh Pyle writes some things about being owned by the borg and innovations...

What will become of Groove now? Good things, I think. After all, the only competitive architecture within a couple of decades is Alan Kay's mallet.

I wonder how Groove and Croquet line up. They seem fairly different on the surface.

Evolution

Sean McGrath writes about SOAP...

Behind the firewall, nobody can hear you scream and nobody can see your wallet bleed.

Success should also be measured by an ability to evolve. Building a system once is not the challenge.

Hypermedia as the Engine of Application State

From Roy Fielding's thesis...

"REST is defined by four interface constraints: identification of resources; manipulation of resources through representations; self-descriptive messages; and, hypermedia as the engine of application state."

(via the REST Yahoo group)

That last one has my attention.

Size Matters

Scott Hanselman writes (via Sam Gentile)...

The Size of the Framework - Sure, it's big. So was Win32, and so is sun.java.*. Programming isn't all Ruby on Rails, you know. :) The redist is 25 Meg? For what you get that's pretty cheap. That'll fit on any pen-drive and can be downloaded in a few minutes via broadband.

I was giving a tour of Squeak Smalltalk to a friend who'd never seen Smalltalk before. I did not have to explain much, just play with some objects, play with Seaside, show some browsers, a workspace, a debugger, look at some code. He got it, in bits and pieces. After a couple hours he was amazed.

He saw that the 3.7 image out of the box is about 15mb. The image I was using was about 22mb. The concept of an image, consisting of "live objects", being persistent, caught on and his eyes lit up. That 22mb contained the persistent objects that we were playing with: Powerpoint-like presentation objects, a 3D scriptable wonderland, rich text being flowed in real-time through multiple arbitrary shapes as I reshaped them. An internet browser, email, etc. A little car and steering wheel that really drives it around. A piano keyboard, and more elaborate instrumentation. All objects in the system, all the code available, all the time, multiple platforms identical, all in less than two dozen megabytes.

Live objects. *That* is amazing and he was amazed.

Web DNA

Wonders Jon Udell...

...despite Steve Ballmer's infamous monkey-dance chant, developers haven't flocked to TerraServer. What's Google's secret? Web DNA and no Windows tax.

Programming Languages

Philip Greenspun on programming languages...

SQL, Lisp, and Haskell are the only programming languages that I've seen where one spends more time thinking than typing.

There are some others to add to the list: Smalltalk, Python, Ruby, SML, etc. But the point is taken.

Bringing it all back home

Sean Mcgrath writes...

Here is the great irony of the Web. It was vastly easier to create a CRUD application (...Create, Report, Update and Delete...) in the days of Dbase II than it is today.

Sometime, somehow, we have to get that functionality back.

Hand-in-Hand

Radovan Janacek projects the downfall of JSR-xxx before WS-xxx.

But perhaps the decline of Java and C# will *coincide* with that of WS-Complexity. More agile approaches are gaining mindshare both for programming in the small as well as programming distributed systems.

Drawing

Drawing lines in the sand. Chris Ferris is going for a forest vs. trees argument. Carlos responds to Chris...

If this is the new definition, then I would like to hear a simple explanation on how all this baggage helps improve interoperability. Now if SOAP doesn't say anything about interoperability (i.e. it is orthogonal to REST) then a clear consistent and coherent explanation of WS-* would help.

The software worlds own version of March Madness. Now I'm drawing on too many metaphors.

Zenning

If we want to throw open the road ahead,
We must first push down the wall that’s facing us.
- Hong Yuan (~1533)

Pig Piling

WS-xxx, SOA, ... pig pile!

I agree with Sean. Carlos last few articles have really been a blast reading.

Then over on the other side of the room, speaking of blasts, James is pointing to Jonathan Schwartz's blasting away at WS-xxx hell as well.

From Herr Schwartz...

Web services may collapse under its own weight. No one at the conference said this. Those are my words. I'm beginning to feel that all the disparate web service specs and fragmented standards activities are way out of control. Want proof? Ask one of your IT folks to define web services. Ask two others. They won't match. We asked folks around the room - it was pretty grim. It's either got to be simplified, or radically rethought.

As you know, I also believe simplicity and volume always win - and that today's web services initiatives are in danger of vastly overcomplicating a very simple (really simple) solution.

Marketing

Kurt Cagle comments (to Sean Mcgrath)...

SOA is destined to become the has-been term of 2005, no doubt to be replaced by some other bit of marketing wizardry that is just as meaningless.

(Kurt Cagle works and writes about many things to do with XML.)

Steering

Ron Jeffries writes...

What is fascinating is that there is no aspect of life or business where things work perfectly, the first time, on time, without any cost. And yet they ask us for that. I believe that it is because, historically, they've had no way to steer, so they don't recognize right away that now they have one.

Languaging

Bill de hÓra writes...

You can think of it this way - if Lisp and Smalltalk represent some kind of maxima for expressivity in programming languages then speech act languages like KIF and FIPA-ACL represent a maxima for expressivity in Internet application protocols.

I guess I'll have to figure out what those are.

Enough research and experimentation has been conducted on software agent communication languages over the last twenty years to gives us an inkling of how this might work. The primary problem with this option is social - my guess is that it's more difficult to innovate with application protocols than content models since app protocols tend to be baked into the infrastructure as of the beginning of the 21st Century. And we do tend to think of protocols in engineering terms (interfaces, structures. bits, wires) rather than as linguistic phenomena. It's not called the Internet Language Task Force after all.

Hmm. Interesting. I've been of the mind that there are a small number of application protocols and then another level of protocols could evolve as sequences. (Kind of using DNA bases and sequences as metaphors.) So we need a basic technical structure that will enable a socially acceptable process for evolution.

Minimizing Obligations

Sam Ruby writes...

I personally like conclusions that minimize obligations for all concerned.

Interestingly, Sam is writing about legal issues, but this applies equally well to purely technical design issues.

The REST of Radovan Janecek's List

I've commented on one item from Radovan's list already. Now for the REST...

any other metadata has to be referenced from the representation of the resource

This is good. Semantic nets typically connect data (e.g. "frames" and "slots") and metadata (e.g. "facets" of a slot) directly. I would expect this to be so for a semantic web as well.

use of REST is not always appropriate (even if theoretically it can be used almost everywhere)

This is almost certainly so. But I suppose it is more broadly appropriate than its current range. And less clear is what is more appropriate in those other cases. Certainly there are cases where pre-existing systems are more appropriate. But what of the up-and-coming distributed systems technologies?

REST might be inappropriate especially if the integrated legacy system defines its own non-RESTful abstractions (e.g. SAP's BAPI).

Certainly there will be these cases. But I'm not sure SAP R3 can be put in that category. First, SAP has provided at least a couple of web-services-like interfaces to R3 and newer components. This is settling into the Netweaver/XI integration platform.

I don't know much about this yet, but I don't see why REST would be incompatible. One of the best presentations at OSCON 2003 was convincingly about a Perl and REST interface to R3.

use of REST is especially appropriate for generic system-level services that are expected to be used by robots (registry, security, management, reporting)

I think you are underestimating what a "robot" (i.e. an "interpreter" of semi-structured information) can do. See the Experiments with OVAL paper for a variety of examples using very modest software.

REST is appropriate for hypermedia, of course ;-)

Maybe a good discussion could be about what is suitable for representation as a hypermedia?

building REST applications, WS-* should be used wherever possible to ensure interoperability

Maybe this should be turned around a bit to say that WS-* should be defined to be compatible with the web.

RDF/OWL is more important for REST applications than it seems today

I think generally we need to learn how to build more evolvable application-specific semantic web interpreters. I know a little about RDF and less about OWL, although I am familiar with older semantic net languages. I think this statement could be so. But again referring to OVAL, a lot can be accomplished without getting to deep into heuristics.

Resources

Danny Ayers has a new blog on grid computing and such. One of his first posts is a look at WS resources and RDF. He lays it out nicely and has some good questions around how these seemingly related approaches might actually be compatible, or not.

Semantic Web Interpreters

A couple of inspirations come to mind that I periodically reference here for multiple reasons. Thinking about semantic web interpreters bring these to mind today.

• Adaptive Object Models
• OVAL by Malone, Fry, and Lai

These sources should be absorbed as fundamental background for building RESTful interpreters.

Good Enough

Carlos Perez on SOAP and REST...

If WS-Security is rarely used, HTTPS is good enough and interoperable web services is via XML Schema validation of documents then it becomes less clear what exactly SOAP brings to the table.

SOAP adherents say REST can't be made reliable. However, argue that point with Bill de Hora who recently published HTTPLR a reliable way of transmitting messages via HTTP. The spec is consistent with HTTP semantics rather than extending or abusing them. In otherwords, reliability without the extra baggage of SOAP.

Moreover

There are a couple threads that could be tied together.

Radovan Janecek writes about REST...

A resource has to return not only 'data' but also a contract that tells how to work with this resource further - i.e. what to POST and when. In case of browser-based applications, this is somehow solved by HTML. Otherwise, I think WSDL can be used.

I don't think this is quite so. The browser is interpreting representations at runtime. WSDL will present a description of the shape of the runtime data, but the application using the WSDL will still have to do the interpreting.

The choice is definitely not confined to a browser interpreting HTML or something else interpreting WSDL. No, there are many potential examples of other runtime interpreters. I think a good example is a "workflow engine" that interprets workflow descriptions. Let's say a workflow is stored on a server somewhere. The engine GET's a description of the workflow. Let's say the representation is a collection of RDF triples, some of which describe the orchestration of the workflow. Others describe the procedures being orchestrated, and still others may describe some of the work-in-progress.

The workflow engine, like the browser, is a runtime interpreter of state representations. The browser interprets HTML pages. The engine interprets workflows. The browser understands HTML. The engine understands the orchestration triples. When a procedure is encountered the engine interprets that by invoking the procedure at the given location (or perhaps just abstractly "on the grid") with the given work-in-progress triples as parameters of the procedure.

The RDF triples are runtime state representation, analogous to HTML. The workflow engine is the runtime interpreter, analogous to the browser. An infinite variety of runtime interpreters is imagineable. Mapping engines to interpret locations, ledgers to interpret sequences of transactions, purchasing agents to interpret items for sale.

So are there an infinite variety of state representations. The interpreters will vary by purpose, but their interfaces should be fairly constrained (i.e. RESTful). The state representations will be more broadly useful if they too are somewhat constrained, i.e. if your restful interpreter and mine can understand some of the same state then they can cooperate. This is the idea behind these interpreters sharing partial ontologies or even being able to translate parts of one to another.

Meanwhile Mark Baker quotes Hugh Winkler and then notes...

"When you sit down to write a description language for REST services (a IDL or WSDL for REST), you discover that doing so is unnecessary." Absolutely. The application state machine needs describing (at runtime), not the services (at design time).

The web is a dynamic, distributed runtime. We need to act like language designers and we need to understand dynamic languages.

Classic Rock

Paul Beard uses The Doors as his canary. Tim Bray uses Pink Floyd's The Wall.

Those both work for me. We have (at least?) three Classic Rock radio stations in my area, so I can bounce around those to avoid commercials as well as worn-out music (and beyond into jazz, community radio, progressive politics, sports, etc.)

The best station though has the weakest signal. Two of the stations are here in Portland. The best one comes out of Corvalis where I suppose there is less commercial pressure, or maybe they're just not owned by a conglomerate with a corporate play list.

In fact the Corvalis station, KLOO-FM, reminds me a lot of early 1970's album-oriented stations that don't just play the "hits". They play the songs they want, wherever the song shows up on the album.

Me too and eight-track tapes

Sam Ruby writes...

I'm old enough to remember when relational databases were controversial.

Me too. Sometime talking about the computer industry in the 21st century feels like talking about the iPod vs. eight-track tapes with my kids. (Not to mention reel-to-reel. Remember those?)

Commonality and Variation

I comment on Ted Neward's observations...

SOA as just another paradigm for commonality and variation... just so, even more generally. SOA is a new term, but people were arguing RPC vs. asynchronous message passing 20 years ago, not just in the labs but in production.

I used Apollo Domain workstations and their Aegis OS at Boeing and Mentor Graphics. In the late 1980's I was in discussions about their built-in distributed OS and mailbox channels vs. the emerging broader standards for RPC. People were arguing about RPC parameters vs. sending ASN.1 data structures (i.e. flexible trees of property/value pairs). Not much new in this SOA world but the terms and new standards which look a good bit like some of the older ones!

QOTD

"Intuition leads scientists astray every time."

A Secure Scheme-based OS

When I read this...

• A type-safe abstract instruction set (MSIL) as the only system binary interface.
• A unified extension mechanism for applications and the OS.
• A strong process isolation architecture.
• A ubiquitous metadata infrastructure to describes code, data, and communication.

(via this), I think of this. Obviously one has a large, ambitious team and the other represents work by a few people in a more narrow area. But a comparative analysis where they overlap would be worth reading.

What Engineers Do

Wayne Allen writes...

What is it that we are envious of from the "other" engineering disciplines?

I think we envy the respect and prestige more than anything else. The nature of software is different though. We have to build software worthy of respect, and it's very easy to build software that is not. We have unprecendented freedom, but...

Chui Tey writes in a comment...

I don't think the software that runs a satellite or one that powers weaponry would be written any less formally than the effort involved with engineering a bridge. Even at these lofty levels, there are elements of craft at the design level, one which comes from experience, and not one that comes from formal learning.

I think we have more freedom to ignore useful formalisms. Part of the challenge is to understand our formal tools and to push them into new capabilities. Test-first programming is an example of a formal tool not widely understood today. Model checking concurrent systems is an example of a current push into new capabilities.

Generally speaking though I think it is the softer communication issues among a development team that typically leads to failure.

More on common language runtimes

Ian Bicking from comments...

You mean, tried it like Guile? That was the whole premise of Guile, that you could translate other languages into Scheme. They never even got as far as translating elisp (which should have been the easy part) so that they could move Emacs to Scheme. That might not really be fair -- honestly, I don't know if anyone cared enough to try, Guile was born of a flame and never had a real champion...

I did not know that was a goal. I've neither tried Guile nor even looked at it. Sounds like more of an organizational thing than a technical issue. The history of Lisp is full of people writing compatibilities to move their code from one (usually older) Lisp system to another. This ultimately led to Common Lisp to avoid doing this so much.

Someone also tried translating Python to PLT Scheme, and they ended up giving up and just embedding Python in the Scheme interpreter...

Guile was crappy to debug Scheme in, but it offered absolutely nothing for any other language built on top of it. Can you annotate your Scheme source with information that allows you to reconstruct the native language environment when there's an error? Many languages deal poorly with incorrect programs, but Scheme implementations tend to be particularly bad, as is any implementation that comes out of an academic background (rather than being designed by a programmer that actually wants to *do* things with their implementation).

Gambit is a good choice because of its C interface, compiler to C, and good debugging capability. Again I am shooting from the hip, but for annotating Scheme source with information for reconstructing the source language information, the nice thing about this is it is Scheme and you can define all kinds of "meta-data" that does something or does nothing execution-wise, but just keeps information around (or eliminates it when desired). Everything is a list, remember. 8^) Just define syntax and/or structure that keeps the information you want, e.g. source text spans for highlighting an expression in a debugger or whatever. Seriously for non-Lisp programmers: everything is not a list. Scheme has "records" ("structs" in C), objects, vectors, etc. I would try defining records for the "meta-data" and the Scheme code would include references to these records.

And that doesn't even deal with the object mismatch issues. You don't actually have to resolve those issues if you just want to take advantage of the performance of the underlying implementation, but if you want to leverage it for interlanguage communication, translating it all to Scheme doesn't actually accomplish a whole lot.

I personally would not want to try reconciling object models too much. I would have each language keep its own definitions and see how easily the translation to Scheme data can support those languages. I would define some kind of interchange but that would not have to be too elaborate. Maybe start with YAML and leave it at that.

I'm highly sceptical of any academic projects, as I they tend to be mismotivated.

Yes, this is just a thought experiment for me. Like the Python and PLT Scheme observation, it may simply be unnecessary when the original language implementations can simply communicate with each other well enough.

The one time I could see attempting something like this would be when defining a new language from scratch. Lisp has always been a good vehicle for experimenting with new languages. They've tended to be more academic and less production too, but something like Gambit for performance is intriguing.

Craftiness

My response to a comment on Wayne Allen's blog...

True, craftiness appeals to many developers. But what is an example of cut and dry standard bits that don't involve craftiness?

Developing TV's? Cars? There is a craftiness to each of these. But less so. And that is where the communication comes into play with the many more informalities. And it is not just color or layout... it's also "What does a user want to do with this purchase order?" There are formal things to do with it, but also informal.

I think software is informal all the way down to the bits on the wire. Only then does it become informal. TV's and cars stay pretty much the same with inflection points here and there (e.g. Tivo, HDTV's, hybrid cars).

Factory, Engineering, Craft, or What?

As James Robertson points out, there are reasons to favor the Software as a Craft metaphor over the Software as a Factory metaphor. But he writes "software is still a craft" (emphasis mine).

That's almost an apology. I'm not sure how intentional his words align with my interpretation, but the notion is commonly expressed, as if someday it won't be a craft but for now it is. Usually the notion is that someday software will be more like "real engineering".

I've taken another view on these metaphors. Let's forget about factories because 10 seconds of thought informs us this does not fit well at all. So the candidates seem to be "craft" or "engineering".

My view is there are things software developers can learn from various kinds of engineers. Maybe there is something to be learned from legal engineering certification processes. But the lack of doing so is not what makes software development seem craft-like.

I think the "informality" of software makes it feel craft-like. If only we had more formalisms. Could be, but we'll always have "informalities". Every engineering effort, and every creative effort in general, involves many informal conversations.

The difference is perhaps software depends more on informal conversations than do many of the traditional engineering disciplines. The requirements in those disciplines are more formal. Most of software development is really requirements development. The interpretation of formal requirements is most easily expressed in software as tests and code that (correctly one hopes) runs those tests.

I think the key difference between software development and traditional engineering disciplines is not primarily the lack of discipline. Rather it is the inherent predominance of informal conversation and that software development fails mostly due to lack of communication. There is room to improve our formalisms for sure. But those improvements will not replace the focus on informal conversation the way standards and formalisms have in traditional engineering disciplines.

Those software development improvements have to support informal conversation. That's why the agile approaches have been successful... when they have improved communication for a specific team, they have improved the results without any new technology or formalisms. Of course some of that communication is in the form of "knowledge transfer" of how to use the available formal tools better. But in the aggregate it is *all* sorts of communication among the whole team that has made agile approaches successful. And when they have not been successful, in my experience it is not from failing to use a formalism well, it is failure to communicate well.

I'm a Broken Record

Update: Luke Gorrie comments...

Can you point me towards any examples that have gone "all the way" with a production language implementation based on translation to Scheme? The approach looks really appealing but I'm becoming skeptical of it.

I don't know. Kelsey went far enough to get a PhD. Feeley (with Erlang to Scheme) went far enough to "prove" the concept. Neither could be considered close to production worthy. The closest thing I can cite would be the compilers for Fortran, Ada, Pascal, C, and Prolog for the Lisp Machine. I'm not sure if they compiled to Lisp or to the Lisp Machine assembly language (which could be seen as a kind of Lisp as much or more than a typical "machine" language.)

I guess the point is not that this is proven, but that today, in 2005, why has it not been explored? People are still beginning VM projects left and right in the 1970's approach. Where does that leave us? Why not try something else, at a higher level, to *see* if it works?

End Update

As per James Robertson, Sriram Krishnan and some folks are building a new VM. It sounds ambitious and two minutes of looking around did not clarify for me why they want to build something from scratch. So rather than suggest they shouldn't, I'll try to be more positive and suggest some reading material for someone interested in these things, then speculate on what I'd do, then admit I have no intention of doing such a thing myself but you might listen to me anyway. Of course I think these ideas are worth considering, unless you *really* want to work on the lower level bits.

Right now, we're looking to copy from existing VMs as much as we can since we really don't have time to design a lot of stuff. This doesn't mean that we don't want to innovate - just that we don't want to reinvent every wheel. I'm personally looking forward to figuring out how to implement continuations inside the VM itself.

You could do worse than to look at the source code for Gambit Scheme and read the 20+ years of technical reports on compiling Scheme. By Sriram's second post, it looks like he is on this trail...

Everything worth inventing has been invented by the Lisp guys 2 decades ago. People like Guy Steele have done some really cool work

I guess if I wanted to work on a language project it would go more like this... start with the implementation of Gambit Scheme because you probably won't do better with less than 2-5 years of research and development. That's probably optimistic, based on the fact that Gambit has a decade or so of concentrated effort from a very bright person, and there's decades of work and discussions that went into Marc Feeley's own thought processes. You'd better talk with those guys because you'll be recreating their mistakes at best.

Then I'd build the translators for the languages I want to run to compile from source to Scheme. Along the way I'd do something interesting to demonstrate the payoff of using what's been done so well already: e.g. I might write translators for partial definitions of a couple languages , perhaps subsets of Smalltalk, Python, and Ruby, and then combine Scheme, Smalltalk, Python, and Ruby in a simple continuation-based modal web server based on Chris Double's tutorial.

The would demonstrate a better way of programming web applications on a well-designed kernel, supporting multiple popular languages. It would also demonstrate the value of continutations in languages that do not have them, e.g. Python (other than Stackless, I think). Pehaps others would want to add to the language fidelity. Another benefit of this approach then is that those folks don't have to understand my VM design, or track its quality, or track its design changes. All they need is to understand Scheme and write a translator from their language to Scheme, or enhance one of the initial prototypes.

GNU Smalltalk is fairly simple... a simple initial translator to Scheme does not seem too hard. I'm not sure what starting points to use for Python or Ruby. But I'm not proposing to do this... only suggest it to someone. I think it would be useful but it's not high on my personal list.

Just some thoughts if there's a Masters student looking for an interesting project.

Shortness

Jim Weirich writes about web apps, Ta Da, and Ruby on Rails...

Ta-Da lists uses XMLHttpRequest to interact dynamically with the host. Beautiful.

And to top it off, it is a Ruby-on-Rails application. Written in 579 lines of Ruby code, that’s less than the size of the XML config files used in many J2EE applications.

And John Wiseman writes about Groovy, Common Lisp, and the shortness of code for an RSS reader...

I had forgotten about when I said that the only way someone would be able to compete with the Groovy app was to integrate with Java or Cocoa, or use CLIM: He created a webapp and generated HTML.

It took me about 3 hours to write. The initial quick hack version took only about an hour and was about 120 lines, but it was not as nice as this one :).

The final version of the code, at about 200 lines, looks pretty good and wouldn't be hard to turn into a desktop GUI app.

The shortness is due to Ruby, Lisp, (or Python or Smalltalk or...) being able to hide any kind of functionality behind a few simple lines of code. We should expect these languages to provide roughly the same amount of abbreviation vs. more elaborate notations.

Maybe if you like writing code these languages aren't for you after all. 8^)

WTF? Solved: burnatonce

Update: The winner is burnatonce. I downloaded the software and had the CD burned in less than 1/2 hour. I got my refund from the 24-hour company and will apply it as a donation to burnatonce.

Not sure why Microsoft doesn't make their CD burner more easily available to XP users than to download a 12MB Windows Server 2003 package, while suggesting XP users download some guy's freeware that doesn't work. End of update

OK. I want to burn an ISO image onto a CD using Windows XP. The information from Microsoft says they don't have that capability, but to try some guy's free (beta) software! Thanks a lot. Glad my money's being put to good use up there in Redmond.

Hey, I've got an idea. Maybe I should just run some guy's free software from the ground up and skip this whole Windows thing. But OK, I install this guy's free software. But when I burn the CD an error immediately pops up... error code 1800183298837 or something. This guy must have worked at Microsoft recently. No wonder they recommend his stuff.

Then I try Nero Buring ROM. (Ha! Ha! I get it.) Problem is the software sucks so bad out the user interface that I want nothing more than to burn the Nero software itself.

Next I try MagicISO. Installs quick. Simple interface. BAM! The trial cannot burn an ISO greater than 300MB. Thanks. Seems useful enough that I decide to spend the $30 USD right then and there. The order process goes OK.

I get the email confirming my purchase. Great. But there is no key to register the software in the email. And there is no URL in the email to get a non-trial version.

Let's actually read the email.

Your order will be fulfilled directly by company that develops the software. Orders are generally fulfilled very quickly, however, please allow up to 24 hours.

%@($#^!& What is this? A 24-hour turn-around for an electronic purchase? (I chose not to spend the extra $10 USD to get a CD!)

I guess this time tomorrow I will be burning an ISO image on a CD.

With any luck.

Build Relationships

I guess it is Extreme Programming Day here at Making It Stick.

Kent Beck writes about planning...

We deliver functionality but we build relationships. Planning is more about building relationships than it is about delivering functionality.

I teach a "whole team" Agile/XP course a half dozen times a year. I emphasize values a much as practices. Communication is the value at the top of the list. We also spend a significant bit of time discussing how to adopt XP. My first (and often repeated) message is, if you can only take one thing out of this course then take the benefits of "clear communication with your whole team" into whatever practices you do choose.

Not only is communication the most important value it is also the easiest thing to adopt as an individual under any circumstances. Build relationships.

Slowing Down

Ron Jeffries on the pace and stress of software development...

If you're feeling under the gun, it might be time to slow down.

Amazingly counter-intuitive advice is found throughout Extreme Programming practices. I have *never* seen continued pressure and stress succeed in software development. On the other hand more than once I have seen big improvements when teams are able to step back, breath, make a couple of observations, and move forward gradually. Before long the team is working better than ever.