Message to Tim Bray

Larry Maccherone writes in a comment...

I wanted to post this to Tim Bray but his blog didn't allow posts. I've worked with a group here at Carnegie Mellon called the Fluid project. They have developed the ability to statically assure that your concurrency model matches your code. The tools also make it trivial for you to indicate your concurrency model by guiding you in adding minimal annotations in the code (think javadoc).

The fluid web site.

The publications (including Aaron Greenhouse's PhD thesis).

I post it here as a guide to how this might be done in dynamic languages but also to help Tim out with his java concurrency issues. Could someone see that it gets to Tim?
LMaccherone ..at.. cmu ..dot.. edu

Testing Interfaces

Ian raises some concerns about testing interfaces in Python...

Unit tests aren't going to test across the boundaries where people are proposing interfaces -- between loosely coupled components.

I am not sure I understand this point. Say I give you version 1 of a component and the tests deemed "sufficient" for that component. Then I give you version 2 of the component and the tests. If you are able to run version 1 of the tests against version 2 of the component, that should be a reasonable indicator that your own use of version 1 will be compatible with version 2 of the component.

As the provider of the component, I need to know nothing about your own use of that component, for reasonable definitions of "sufficient tests".

Note also that interfaces are more explicit about contracts between components than tests will be. For instance, an interface might demand a method which is not yet being used in your code or your usage; the interface makes it explicit that this method should still be defined, and so predicts future developments, not just current developments (half-implemented dictionary interfaces are a common example in Python, where interfaces would motivate people to finish the work).

The original scenario was an upgrade to a component breaking backward compatibility. If an upgrade provides a new feature then your old code has not used that feature yet. I would not expect that to break backward compatibility.

If an upgrade requires an old feature to require a new method of its old consumers, then that *would* break backward compatibility. However I would expect a reasonable set of tests from the component provider to demonstrate this failure on their part to keep their old customers happy.

Or two interfaces can match methods and signatures, but include extra information about the semantic differences between those interfaces; because it doesn't use duck typing, you won't accidentally mix semantics (you have to explicitly say which, if any, of those interfaces you support). This is particularly a problem with interfaces that define only a small number of methods, or use common methods like __call__. The only way to test these is with system tests, and system tests are always incomplete, there's always some code path or combination you haven't tested.

I guess I would need a specific example that cannot be tested by a unit test. Certainly there are "big" tests that require a lot of "units" to execute. But if each of those units were developed and updated by a series of "unit" tests then I would expect those tests to represent the "differences between those interfaces" as they exist within each unit.

I am having trouble addressing this issue in the abstract. If it is a frequent problem, I would look for ways to avoid it or break it into smaller problems. Maybe this is a scenario where changing the language would solve the problem. I just don't know, but I suspect there are simpler ways to solve the problem on the basis that I have used Python and several languages like Python and I have not run into this scenario. That doesn't mean it isn't real, just that I have not recognized it.

More Ways to Give and Help Children

In addition to my two favorites, Anders Jacobsen is asking bloggers to include these organizations on our blogs. For each of us that does, he'll donate to the British Red Cross. Thanks Anders! I think UNICEF is a good organization to have at the top, reading about their concern for child safety given the number of orphans resulting from the disaster. Pray to change the hearts and minds of people who would take advantage of children like this.

International aid organizations: UNICEF (United Nations Children's Fund) United Nations' World Food Programme Medecins Sans Frontieres / Doctors without Borders (donate!) CARE International The International Federation of Red Cross and Red Crescent Societies UK/Europe: Disasters Emergency Comittee (DEC) - comprises a raft of aid agencies, including the below and others British Red Cross Oxfam Save the Children UK North America: American Red Cross Canadian Red Cross Save The Children Oxfam America Anders Jacobsen: Webloggers: Give to tsunami victims and I'll give too!

Crying Out Loud

All of a sudden blogs are not about an ongoing discussion?

Gordon and Ian both write about the reaction to Guido's posting very preliminary thoughts on what would be a controversial change to the Python language.

The question they raise is this: does the reaction imply that Guido has to be somehow more "careful" with what he chooses to write?

This question puzzles me. I am assuming Guido published his thoughts in order to *get* a reaction. Should the kind of reaction he gets be considered a problem?

Dictators had better have thick skin. Let's assume he can handle it and continue to speculate. That's what blogs are for, right?

You publish your thoughts, I'll publish mine. The result will be a web of thoughts, and presumably this will be better for the whole community as compared to being afraid and secretive.

People don't "need to chill". People need to *write*!

I Must Be a Type Magnet

OK, I'm not resisting responding to some comments on the interface type declaration issue.

Interfaces are something that two of the major "framework" systems in Python (Zope and Twisted) have both found missing and wandered off in search of their own solution. If such a system existing and was codified into Python it would make it a lot easier for myself and others to actually rely upon bits of code crafted by other people; specifically feeling warm and fuzzy about you not going around and changing something deep in the guts of your modules that I am not going to find out about until five weeks after updating to the most recent release.

I have found that framework upgrades work best when there are sufficient tests available with the framework and sufficient tests for the system using the framework. These tests reduce the magnitude of lots of problems, especially the scenario described here.

Are you saying that even with sufficient tests, from the supplier and in your own code, you are still getting surprised a month into an upgrade?

If not, which do you think would be the lowest risk experiment for addressing the problem: writing more tests for the framework and your code, or changing the language?

(Note: I am *not* asking if you think type declarations would in some way address this problem. I *am* asking which of these two alternatives would be the lowest risk experiment.)

Freeze!

A big problem with Guido's considering type declarations for Python is that there are now at least three significant implementations of Python. Disregarding how you or I feel about this specific proposal, any big change in the language will put further distance between the implementations or impose a heavy fine on those implementations that are trying to keep pace.

Given the momentum in favor of Python over the last year, if I were the BDFL of Python, I would freeze the language at the 2.4 specification for a few years. This would allow the other implementations to catch up, and allow the CPython implementation to focus under the hood on implementation improvements.

I'd have to see real evidence of pain and suffering to want to extend the language significantly. Let other languages try new ideas, but Python should probably stay close to the current definition which has brought it to the threshold of something big.

Of course Guido's current thoughts just exist on his blog as far as I know. Real changes to the language based on these are a long way off. They could have an ill effect on the current momentum just the same. At the least they could suck energy away from more immediate concerns.

Comments are enabled for the concept of a moratorium. No type checking debates here please.

The Road to Ruin

The dictator is leading us astray and we're on the road to ruin. Sad just when things were looking up.

I thought I wasn't going to write about this anymore, but I can't believe my eyes. Just as Python is getting to the table with the superpowers.

I don't want to comment on this. Sorry, comments are disabled no matter how much fun more type checking debates might be!

Gartner Buys Meta

From The Register...

"This transaction is an exciting opportunity that will give us increased depth in key sectors, geographies and markets, and an increased ability to seize revenue opportunities with the addition of META Group's well-trained, successful sales force," said Gartner's CEO Gene Hall. "In sum, the acquisition will make Gartner a stronger company with increased opportunities for growth and greater resources to offer clients."

I was impressed with Meta's presence and insight at a conference they held a few years back, and with some recent content. My experience with Gartner has been spotty, but lately I've been finding some interesting white papers in their library.

James Robertson has more perspective on Gartner than I do.

Pimp My IT Ride

Chad Dickerson writes...

Smart IT shops should limit unneeded complexity at every turn, choose their customizations carefully, and turn a deaf ear to the siren song of the perfectly customized solution...

In the heat of the moment, no one thinks about what these one-of-a-kind customizations are going to mean down the road.

Do you think "service oriented architecture" is going to solve this problem? Can solving this problem be profitable?

Process Models

Bill de hOra writes...

There's no question that too much business logic is locked up in middleware and systems programming languages. But we need to avoid the industry standard hype cycle on these technologies and be clear on what they can and cannot do...

Process modeling has to balance agility and formality. We cannot allow to happen to process modeling what has happened to data modeling. And then we have to take back data modeling and fix *that* too. Related note.

Out of the Bottle and Into the Box

Daniel H. Steinberg writes...

Let's ship Jini, JXTA, and the Java APIs for Rendezvous along with the JVM. What if easy dynamic distributed networking was available from and to every Java enabled device.

He expands on this in a larger column.

3MLOC

Via Chris Double...

The runtime source code is 3 million lines of Java and C++. That's way too much for what it does...

This includes a mix of awk, sed, shell scripts, and indeed, Scheme! Yes, the JVM source includes jscheme.jar, used to generate some CORBA classes.

Games

I played a few traditional games with family and friends New Year's Eve and yesterday at a friend's New Year's Day open house.

• Rummy Royal -- We played for a couple of hours on New Year's Eve. It's been a long time since I've played this. The great thing about it is the variety of each round; you start with a chance to choose or bid on an alternate unseen hand. You can bet on an ace showing up in your hand. Then you play a poker hand by choosing five of your cards. Finally you play all your cards in a Rummy-like game. I pretty much broke even for the night.
• Scrabble -- This was about the worst game of Scrabble I ever played. At one point I had all consonants. I foolishly traded *all* of them in, and came up with all *vowels*. At this point I had three of the four 'u' tiles and two of the four 'i' tiles. Why didn't I keep some of the consonants? The biggest problem was the play went from the middle of the board down into the lower corners and none of is had a way to build upward.
• Sequence -- I had never played this game before even though my wife says we have it somewhere! This is a good game for adults and kids to play together. The rules are simple, the pace is moderate, and there's a good bit of chance, but opportunity for strategy to win out.

Some friends were telling me about "Speed Scrabble" but we didn't play. Next time; it sounds fun.

One Startling Sentence

PJE writes in a comment...

He's not saying adding enough negative numbers makes a positive -- although your phrasing sure made me click on the link out of mathematical curiosity!

Which is of course why I wrote that, and is drawn from the idea of One Startling Sentence.

But I chose two startling sentences. (One is the title itself.) I thought of a third, but thought then that was too much.

Take Back Your Data

Again Andrew Newman...

There is a common theme in the IT history towards more freedom. I don't mean free like in free speech, I mean free like in free will.

Lock Free Programming

Andrew Newman points to an article on atomic variables in Doug Lea's concurrent package for Java. Is the attention on concurrent programming going exponential?

More Than Two Wrongs Might Make a Right?

Oliver Steele seems to conclude that adding enough negative numbers may have a positive result. I'm not sure I get all the way to the same conclusion, but the argument is unique.

Update: OK, I definitely do not get to the same conclusion. Saying a type declaration "does a poor job of each scenario, and that means it does a good job of doing them all at once" is a leap I cannot make.

I don't see Oliver addressing the cost side of the equation sufficiently for his benefit side to come out greater. I think the costs can be overcome with creative work, but I'm not sure that's where I'd want my most creative thinkers working. Making type declarations optional alleviates costs, but why bother? I am just one who does not see the benefits.

Since Oliver already points out weaknesses in any one of the scenarios, there is no point attacking them individually. I cannot argue against the whole because I don't see a complete equation that can be analyzed. But I will point out some glaring problems he's passed over.

• Documentation Providing f(a: String, b: Integer): Boolean rather than just f(a, b) is a weak argument. What are f, a, and b? Those names, well chosen, will tell me more than String, Integer, or Boolean. Moreover using Smalltalk's keyword syntax you have even more expressiveness than this Pascal style.
• Early error detection Test-driven development eliminates the gap between coding and testing. There is no "earlier" time for detecting type errors.
• Discouraged from writing tests If your language is so bad that programmers are discouraged from writing tests, my friend, well, type declarations are not going to make up the difference.

This is the last post I will make on type declarations for the foreseeable future. Feel free to comment, but don't expect a response.

I have another post to make responding to PJE, which I have learned is *not* about type declarations per se. Coming soon.