"I have a mind like a steel... uh... thingy." Patrick Logan's weblog.

Search This Blog

Saturday, February 19, 2005

Courage and Trust in Software Development

Bill de hOra writes...

While refactoring is important for codebases that are expected to provide a product dynasty rather than a legacy mess, it's often devalued as mere tinkering by those removed from source code, as it doesn't provide new functionality - it can be very difficult to explain to a non-expert why not adding, or even delaying features now, will help with delivery of features later.
I think there are several ways to approach this devaluation. One is, do things in small pieces whenever possible. A little refactoring following a little vlaue adding is hardly noticeable.

A frequent question I get in the face of this kind of devaluing is, "Should we just do X and not tell them?" My sense is that sometimes may be advantageous when adopting a practice that has been unduly questioned. But my advice is not to make a habit of this. An important goal is to build an effective whole team, and deception in any significant, ongoing, way cannot be effective in the long run.

There is a more positive approach to this kind of behavior. That is to revisit the roles and responsibilities of developers and other stakeholders. In this case, developers have the responsibility to improve the design when they believe they'll get better at adding value in the longer run. Stakeholders having the courage to trust their developers is as important as those developers having the courage to admit they need to go through a significant refactoring for some reason.

Most software development practices over the last several decades have not brought courage and trust into the equation. Making these values part of the discussion is a clear advantage to agile methods and Extreme Programming. Scary, but better to be encouraged to courageously dive into the depths of what will haunt you anyway, even if ignored.

Friday, February 18, 2005

That's It

Bill de hOra...

We can see the market looking at 3 things to provide value in software solutions:
  1. Simple protocols and formats
  2. Open source infrastructure
  3. Agile methodologies
All of these are disruptive.

Wednesday, February 16, 2005

Skiplists in C# and Smalltalk

This on data structures recently from MSDN...

Skip lists are an ingenious data structure that turns a linked list into a data structure that offers the same running time as the more complex self-balancing tree data structures... In the latter part of the article, we'll be building a skip list class in C#, which can be downloaded.
Cool.

I recently had a use for a skip list in Smalltalk and could not find code I wrote a significant number of years ago. So I got out the trusty skiplist cookbook.

Fortunately I did not get far before discovering that skip lists are available "out of the box" in Squeak.

Tuesday, February 15, 2005

New Functional Geometry

Bill Clementson references a functional programming book chapter on functional geometry. (Part of a really interesting book from the early 1980's still worth reading.)

As it turns out the functional geometry presentation has been updated.

The Future of SOAP

James Governor writes...

One big question is why haven't IBM and Microsoft responded? The obvious answer is vested interest. When you have "bet the company" on a technology stack its kind of a drag to have to respond to something else. Its interesting that in a week when the bug guys, including Gartner, have trumpeted the arrival of UDDI 3.0, the world is quietly getting on with more interesting projects.

Smalltalk Evolution

The question is raised, how does Smalltalk evolve?

My answer is "very little". Smalltalk is not perfect, but is high up on the list of near-perfect languages. Any radical changes are better off being put into a new language. Otherwise the change can be put into one of the open source Smalltalk implementations and adopted ad hoc. That's your choice and right. If the change succeeds wildly the community will adopt it for you.

Meanwhile there is controversy over the leadership of Squeak. I don't know the whole story here but this is open source. They seem to be mostly supported and they've stepped up to the task, so good on them. It's not like they should be taking the language in radical directions.

Fork the language under a new name for radical changes. Let Smalltalk be Smalltalk at this point. I think the limitations of other languages lead their communities into a mindset of constantly adding features as if it were a Microsoft Office product.

Monday, February 14, 2005

Less is More

Blaine Buxton makes an often discounted observation...

Removing code means you are simplifying the code and the result is a net gain. So, next time, you're refactoring and deleting code. Don't think of it as deleting, but as gaining code via simplicity.
Blaise Pascal is quoted as saying...
I have made this letter longer than usual, because I lack the time to make it short.

Sunday, February 13, 2005

Shell Scripting

Danny Ayers is looking for a shell script to keep another process alive. I'm sure I won't do better... I do as little shell scripting as possible. Shell programming never seemed aimed at humans.

Everything I've wanted to do manually in a shell script I have done more easily in Emacs or some other interactive programming environment involving Lisp or Smalltalk. For automated tasks its as easy to schedule a Lisp script as easily as bash. The trick in the past was using a small Lisp like XLisp or one that generates small apps like Gambit Scheme. That's still the trick but there are more choices.

Many Lisp implementations these days understand the #! notation to run as a script. Another option is to use scsh, the Scheme Shell.

Saturday, February 12, 2005

Between a Rock and a Hard Place

Tim Brookins writes...

In case you've been living under a rock, Longhorn is platform for the next decade...

Software developers live to ship code. So delaying the first release of MBF is painful. However, as a Software Architect, I am absolutely convinced that this was the right choice. Shipping part of MBF in Whidbey and part in Longhorn would have created a system which spans two technology eras.

Interesting. There are several potential messages in this statement but I am not sure how a customer deciphers which should be heeded.

Friday, February 11, 2005

Objects and Processes

Ian Bicking on objects and processes (when processes are done right)...

That's when I thought about Erlang. Instead of objects, Erland has processes, and you send messages between processes. It seems obvious, but when I'd looked at Erlang before the parallel hadn't occurred to me. In that light Erlang processes make much more sense to me.
Something about Erlang design... if you need state, like an "object" would, then put it in its own process. Asynchronous messages to update the state and return state information. Like many small "databases" on the order of a table or so running in their own process.

Study the Work of Others

From Harold Carr's talk at the University of Utah...

  • No literature search in industry.
  • No encouragement for deep knowledge (want "code monkeys")
  • To climb ladder encouraged to manage or "architect" versus doing "the" work

  • Great painters and writers don't delegate
  • They continue to do the work themselves
  • And they continue to study the works of others

Not long after writing this, I come across an item from James Robertson with more ever-present evidence of this sad observation.

Adventure Ahead

Robert X. Cringely writes...

Right now, there is in the U.S. venture capital community about $25 billion that remains univested from funds that will end their lifespans in the next 12-18 months. If the VCs return those funds to investors they'll also have to return $3 billion in already-spent management fees. Alternatively, they can invest the money -- even if they invest it in bad deals -- and NOT have to cough-up that $3 billion.

...so get ready for the most furious venture investing cycle in history.

Thursday, February 10, 2005

Inexplicable: Truth Held Hostage

Where's the outrage over the recent 9/11 disclosure?

In April last year, President Bush said, "Had I any inkling whatsoever that the people were going to fly airplanes into buildings, we would have moved heaven and earth to save the country. ..." The 9/11 commission report apparently found that there were indeed such inklings, which should have "raised alarms about the growing terrorist threat to civil aviation throughout the 1990s and into the new century."

We're left with a pretty good inkling as to why the president moved heaven and earth to keep it quiet before the election.

And reading about Mark Dayton...
Also adding to his vulnerability, Dayton recently delivered a stinging denunciation against the confirmation of now-Secretary of State Condoleeza Rice.
I should think Rice would be the vulnerable one. Some National Security Advisor!

Speaking in Objects vs. "Code Complete"

I was listening to a presenter in a recorded session today refer to their project being "code complete". I did not expect any to refer to that term these days.

Coding never made sense to me as a milestone. I was on one of the first death march C++ projects ever, beginning in 1989. The previous couple of years were spent primarily developing in Lisp. Even with CFront, no debuggers, mangled names... or maybe especially because of these things, my preferred development style was to be unambitious about the results, and to work in small steps.

As a whole hubris had overcome the team. C++ would enable us to rule the world, the sooner the better. The intuition was to spend more time thinking and then coding. Compiling, etc. took too long, so should be avoided.

The problem is that "thinking in objects" is not nearly as productive as "speaking in objects". When you speak in objects, you have a dialog. You speak your intent into the editor and the objects you create respond right back, the sooner the better.

Today Phil Windley writes about a couple of projects still on that old beaten path. So that's three projects in one day I've come across.

I think Ward Cunningham said...

It's all talk until the tests run.

Ruby's Postulate

From a talk by Sam Ruby...

The accuracy of metadata is inversely proportional to the square of the distance between the data and the metadata.

Wednesday, February 09, 2005

What She Said

Kimbly...

I have more to say, about how democrats are a pathetic excuse for a progressive party.
Meanwhile, we should get on to the serious business at hand... starting with this guy...

Language Evolution

Philip Wadler's humorous take on language evolution. (pdf) Via Finding Lisp.

Not Getting It Any Way

Via Cheese Bikini, too bad Nintendo apparently doesn't get it. The DS is a nice device.

This is the sort of rich, simple innovation that I hoped would emerge with the wi-fi enabled Nintendo DS portable video game system. But Nintendo seems to have locked down DS development, limiting it to internal and professional developers. Such professionals have years of experience and training in building traditional games. This background cripples their ability to innovate, to see beyond the constraints of traditional game platforms.

Nintendo, learn from eBay and Google and Amazon: let customers and outsiders build value for you. Open your platform and let it thrive.

Software Factories

Clicking through from James Robertson's item about Smalltalk and C#(?), another item refers to Software Factories.

I will wager any system using a factory metaphor for software development will either not get off the ground or hover low to the ground for some time until they shift the metaphor to something lighter weight. Like balloons.


You're Getting It Our Way

I think Bill Venner in his item on static and dynamic is trying too hard to separate language and culture.

Python supports this multi-valued return technique better in its syntax than Java, but I realized that the availability of syntax isn't the main reason I do it in Python. The main reason is that's the way it's usually done in Python. By contrast, that's not the way it's usually done in Java. The cultures are different. The culture that guided the design of a language influences my attitudes and mindset when I use the language. The Java culture encourages me to build solid bricks...
I don't think you can untangle the culture and the language, even the syntax. Python the language was influenced by Lisp and Smalltalk. Those languages were developed by, and developed the dynamic culture that surrounds them.

Lisp was created in the late 1950s by people who needed agility more than they needed speed. (Yes, 1950s.) Smalltalk was influenced by Lisp, and Python by each in turn. They began is labs that gave us the earliest work in AI, collaboration, and usability. They were not computing pi or finding the next largest prime number.

This realization got me wondering, to what extent is this perceived increase in productivity with languages such as Python, Ruby, and Smalltalk due to the culture of those communities, and the mindset that the culture engenders in programmers, versus the actual languages themselves? What is the real source of the differences in the static versus dynamic language debate?
A community cannot sustain a specific culture unless the framework supports that culture. Programming languages are the framework that sustains the culture of the community that uses the language. They go hand in hand and nother will change that.

Today, and since the late 1980s, the programming community in general has needed agility for the most part more than we have needed speed. Our languages coming out of the "need for speed" community (C and Pascal, then C++, and now Java, C#) and the cultures that accompany them, have slowly been migrating to be more a culture of agility and less a culture of speed.

Those need for speed languages have features and those cultures have adopted practices that are more like the agile languages developed and used in the AI and collaboration labs in the 1970s. Finally we are seeing the last stages of the transition. Truly dynamic languages are gaining mindshare over the partially dynamic.

Meanwhile good implementations of dynamic languages and the advances of hardware have all but eliminated the speed side of the equation. There are fewer uses than ever for the languages coming out of the speed culture.

Update: A comment on Bill's blog...

Whenever you start to think of some Java problem as a general problem with static type checking, maybe rewrite it in Nice http://nice.sourceforge.net/
let boolean redirected, reply;
let long forumID;
let String subject, body;

(redirected, forumID, reply, subject, body) = ExamplePage.process(request, response, session);
I agree as far as that goes. The newer type systems are "dynamic" even though they represent a somewhat different development model than more traditional dynamic languages. But I don't know Nice and cannot say that it is dynamic or merely "partially dynamic", esp. if it is just a precompiler for Java. (Someone will correct me so I won't even google it.)

Being dynamic or agile is more than just having an explicit type system, or having a bad type system vs. a good one. For example, are relatively heavy mechanisms like "class loaders" needed?

Imagine That

From MSDN .Net Architecture Center...

The file has been the central container of work in software development for over thirty years. All of a project's structure and logic is ultimately reduced down to files and directories. The tools that surround software development are built around this concept, too. This article explores the concept of file structure, and contrasts it with an emerging view of Model Driven Architecture...

It seems clear that the file is a relic that has outlived its usefulness for software development...

The modern development environment has not yet fully caught up to the object-oriented shift. All of the tools still rely on a file to be the container of source code. While some contain modeling capabilities as well, the models exist as different entities than the file and its code.

Maybe some tools. Not all tools.

The article seems to conflate two issues that could benefit from being addressed independently:

  • Files
  • Models
By the end of the article I did not really learn anything new about the next generation of tools in Microsoft's Dynamic Systems Initiative, which seemed to be the intent.

Tuesday, February 08, 2005

TDD: Test-Driven Demo?

Perhaps preparation for a demo should be treated as preparing any kind of release for an Extreme Programming project...

  • Prioritize the stories.
  • Write the story tests.
  • Break stories into tasks.
  • Program test-first, writing the task tests and completing the tasks.
  • Run the story tests as the tasks are completed.
  • Any stories not completed by demo day will not be in the demo.
  • The stories are prioritized up front but can change as marketing needs change before the demo day.
If the person doing the demo sticks to the script there should be a good chance the demo won't crash.

When to Generate

Xach Beane is dissing maps.google.com...

Google Maps is essentially a large JavaScript application. Great hackers have an almost instinctual aversion to JavaScript. Google is betting its future on something a tasteful programmer's radar rejects.
I don't think JavaScript is that bad, but I've never been accused of being a great hacker. JavaScript has a few things going for it...
  • It is fairly simple and regular. Not a lot of obscure corners.
  • It is ubiquitous and familiar.
  • Most of the interesting uses for it in the browser can be written once or generated on the server!

Monday, February 07, 2005

Cygwin Heaven

I hadn't used Cygwin in some time. So nice. I found a tip for setting the TERM to rxvt which makes applications like emacs display well.

Gambit-C 4 beta 12 builds fine. The 'make check' test fails, maybe due to some CR-LF text/binary mode setting I need to consider. When the tests are diff'd ignoring whitespace they work fine. The tests for the C interface and dynamic loading need to be looked at. But the resulting executables seem fine so far.

Now I need to try the emacs mode for gambit.

Sunday, February 06, 2005

Theory: More than the Sum of Some Parts

More evidence a theory is more than the sum of some parts.

In Need of a Theory?

The notion of establishing a "metaphor" in Extreme Programming is similar to programming as "building a theory" as proposed by Peter Naur.

Reading how "Microsoft officials have characterized SharePoint as 'the killer app for XML'", I am looking for a theory to emerge in the way that perhaps continuations are beginning to form a theory around web server programs and s-expressions form a theory around manipulating XML.

SharePoint is a larger concept than continuations or s-expressions. So I would expect a larger theory, but a theory nonetheless, to associate SharePoint and XML conceptually. I did not see one emerge in this coverage of Bill Gates' keynote at a Microsoft Office conference.

Gates told keynote attendees to expect Microsoft to more tightly integrate SharePoint with other Microsoft technologies and point products, going forward. He said to expect SharePoint and Active Server Pages (ASP) to "become closer," and SharePoint and SQL Server to become more entwined, as well.

Gates discussed Microsoft's quest for unified storage of all types of data, ranging from files to e-mail and emphasized that SQL Server would be the ultimate repository for storing this type of data.

As "SharePoint evolves up on the server and WinFS evolves down to the client," Microsoft will be able to realize its unified storage goal, Gates told attendees.

Gates is attempting to associate a number of disjoint concepts in this quote. This approach is something I would expect of a salesman, rather than a Chief Software Architect. An architect should be a "theory builder". I would have trouble buying into this stated direction without a better theory, or even any theory at all.

Of course this opinion could be dismissed as just more Microsoft bashing from the likes of me. But think about it, and tell me the theory that is holding all this together. Maybe I am just missing the point. Microsoft provides the dominant architectures in my computing world. I have a vested interest in them making my life easier, and so I have the desire to point out where I think they're missing the mark.

From this, I think they're missing a theory that could propel their success for another decade. As it is, I suspect they're cobbling pieces together and I am asked to take their Word for it that there is a "there" there where there should be a theory.

Template-Based Programming

The last C++ program I wrote was long before templates became part of the language. Ted Neward writes...

There's a tremendous amount of power in template-based languages.
I have no idea what this implies for C++. I can agree that it's true for Lisp though.

Programming macros in Lisp perhaps has at least something in common with programming templates in C++. Where you go from there is up to you. I hope never to write another line of C++ even with templates.

Defending Applications

Phil Windley also writes about defending an application from OS attacks by running on multiple operating systems. An interesting idea that could be taking one or two steps further...

Better XML Scripting

Phil Windley writes...

I've written programs that use XML in Java, Perl, Python, XSLT, and now Scheme. Scheme has been the easiest. I think the reason is that with s-expressions, XML is essentially a native data type in Scheme. That's an incredibly powerful idea.

The Soul of a *New* Machine?

Ted Leung writes about a reinventing of capitalism to enhance its positive social effects.

I happen to be reading an arguably related book of essays, and poetry of all things. Poetry does not have the same cultural reception here in the US as it does in other parts of the world. This book, ("The Heart Aroused", by David Whyte, who I knew briefly 18 years ago when I lived on Whidbey Island just north of Ted's Bainbridge Island) addresses the problems of capitalism as they occur in our everyday work life.

Here is part of David Whyte's essay on Beowulf...

Business and politics profess to be hardheaded, but how many businesses, and even countries, have been ruined through decisions that were ostensibly hardheaded but which had more to do with the relutance of those in charge to face fears or vulnerabilities?

This reluctance to enter the deeper waters of the psyche is not confined to modern participants of corporate life. Fifteen hundred years ago in the Old English poem Beowulf, an anonymous bardic author confronted his listeners with a frightening image of this inner lake. His listeners were almost certainly rough warriors used to getting their hands dirty. He found them equally shy of that dark water.

Beowulf is a masculine story of descent into the waters of the unconscious, but where the restoration of a profound inner feminine power is essential to his survival. In that context it is a compelling story for both women and men working in a masculine workplace sorely in need of a commensurate balancing power. In a corporate culture still dominated by the image of the warrior archetype, Beowulf's plunge into the waters of the unconscious seems to be equally instructive for both sexes.

The early English teller of Beowulf asked his listeners to drop beneath the surface of their daily existence, where the rational mind continually prays for dry feet. Since that time the physical details of life may have changed. The elemental motifs have not.

Decentralization

John Robb writes...

My future opportunity, and yours, is being diminished by a state that doesn't value decentralized energy and prefers to spend its coin on military power.
The sad irony is these concerns and strategies were being addressed in the 1970's at least. I remember reading a pamphlet in the early 1980's called "The Indefensible Society" about this idea. As it turns out the web has an excerpt from that pamphlet.
Because moving in the direction of decentralization and the development of planetary villages is different from the direction our society has been heading during the past century or so, it is likely to be dismissed at first glance as impractical and utopian. Yet in reality it is eminently practical and well within our means as a society. And not only is it appealing and workable at a human level, it also can provide creative and effective solutions to problems that presently seem both unsolvable and out of control, such as the arms race.
Of course the original 20th century arms race is over. (On the other hand the US is spending $1 billion USD per week on Iraq.) The problem now is at least as dangerous as the arms race.

Saturday, February 05, 2005

Cooler than That

Web shopping in particular, and web user interfaces in general, just got a lot cooler. Clearly, 2005 is now officially the year of the rich thin client.

This is really inspiring work (try it), and the contact information shows Portland, Oregon. Anyone know who is behind this?

(Via Mark Baker)

Lessons Not Learned (Yet?)

Avi Bryant writes...

In Smalltalk, however, the only thing that's in C is a tiny virtual machine that understands a very simple set of bytecodes - it knows about pushing and popping objects from the stack, accessing indexed slots, blocks, and method dispatch. What it doesn't know anything about is the language syntax, standard library, compiler, debugger, profiler, thread scheduler, exception system, and so on, and so on - all of these things are implemented in Smalltalk.

[Smalltalk] doesn't support continuations and you want to add them? No problem; 10 lines of [Smalltalk] code and you're there.

...clearly you could implement the Ruby language with a Smalltalk-style VM (hopefully one with a Smalltalk-style JIT as well, which would bring a 20x or so speed increase to the current Ruby interpreter). But really what I'm talking about is a philosophical difference, not a technical one: to Smalltalkers, it's essential that as much of a system as possible be implemented in Smalltalk, whereas this simply isn't a priority for the scripting language community, and it's the priorities rather than the individual implementations that draw me to Smalltalk.

Schememonster

Beware.

Via Blaine Buxton

Modern Software Architecture

Gordon Weakliem writes...

Mozilla is still a C++ app though, and seems to be very stable. Architecturally, it seems to be built around JavaScript and XUL though. It's almost like a VM architecture in that respect. I've been messing with Mozilla Extensions lately, and though it's not the greatest development environment ever, it's very easy to write code for once you get your bearings. The XUL architecture seems to extend very far down into the browser, with most of the browser implemented in XUL and script. Again, it seems like an underlying VM is a prerequisite for a modern software architecture.
Which is ironic given not more than ten years ago the programmers using this architecture were fighting heated arguments about the very idea.

Friday, January 28, 2005

Lisp: an XML scripting language?

I guess if I had to sell someone on Lisp or Scheme today, I might describe it like this:

"Lisp is an XML scripting language."

More sexpr in Utah

Phil Windley is connecting language concepts, Scheme, and XML in his BYU programming language concepts course.

He has a simple excercise that makes the connection. The next step would be to blow their minds with SSAX.

Update: Phil Windley has a page for his students for running SSAX. To gauge how similar Lisp is to XML, how natural the mapping, the SSAX translation of Phil's original classroom example is identical except SSAX wraps Phil's translation in a list to indicate that is the top of the document...
(*TOP* (phil's data))

Naturally Dull

"Naturally dull, I’ve forgotten Zen activities"
- Isshi Bunshu (1608-1646)

Thursday, January 27, 2005

Testing Interfaces

Another blog item on testing.

This one asks whether the tests themselves should alter the interface of what's being tested. i.e. should the test have a back door for peeking into the black box.

My opinion is, emphatically, no. Not for the kinds of tests created while test-first programming.

An impulse to check on the internal state of an object while test-first programming is an indication of one or two things:

  • The test is being perceived too much as "testing" and not enough as "designing".
  • The object itself is probably too complex. Fix the design.

A Future so Bright?

Um...

Avid sailor Craig Venter has trawled for microbes in the Sargasso Sea and discovered more than a million new genes and 1,800 new species. Among them are organisms that thrive on carbon dioxide. Venter hopes to re-engineer some of these unique microbes genetically, into "designed species" that may reduce environmental CO2 levels, as well as provide new foods and energy sources. "Biology can do much more sophisticated chemistry than the best chemists," says Venter.
Hey, I suspect biology can do much more sophisticated *biology* than the best biologists as well!

I'm not sure I want people tinkering with CO2 consuming microbes any more than I want us putting fossil carbon into the atmosphere in the first place.

Test-First Programming

Frank Sommers writes...

I find myself violating this tenet of XP very often. I do test all my code, but often only after I've already written that code. Why does this one XP rule seem so out of place to me? ...

I like to think of refactoring and testing as editing. When writing an article, most writers first create an outline, then write the content, typically in a few sittings, and without regard to mistakes, typos, sentence structure, etc. Only then does an author spend most of his time on editing, fine-tuning, and polishing the article.

I think this seems out of place when "testing" is perceived to be a testing activity rather than a design activity. Rather than not testing first, try taking on a new perception of what it means to "test" software.

The way I view test-first programming exhibits the same outline -> content -> polish flow. In my perception, the "outline" is the short list of tests I think I want to write over the next hour or so. This list of tests is just a list of names, each name being some *design* feature that needs to be developed. Each name will show up in the design.

Then the "content" is the expansion of a name on the list into code:

  1. One or a few function calls or message sends.
  2. One or a few assertions about what is true afterwards.
  3. The other code needed to setup the calls, implement the assertions, and teardown.
  4. Then when this content breaks, the test framework tells me what's wrong.
  5. The rest of the content is missing... I have to write the code that implements the calls.
  6. When the "test" part of the content does not break, the initial draft is complete.
This content (the calls and assertions) is design information, an expression of a design contract. I am not testing something already conceived. I am in fact conceiving of something, creating something, designing something new: an expression about a system and an assertion about what is true given that expression.

As soon as the content is drafted, immediately I begin to polish that draft, consider how it fits with the other content, and how the content itself could be better expressed through reconsidering the names and parts and assertions.

So I see test-first programming as a way of designing. It happens to be a way of testing as well, which is good because a program has to be tested in ways that an informal text does not.

Wednesday, January 26, 2005

Exactly

A note to Sean McGrath from Steve Jensen in Sean's comment section...

Thanks for plugging Erlang: even if it doesn't end up being heavily adopted, it at least moves people in the right direction.
Exactly. I like what else Steve wrote...
We need to move from using semaphores and monitors to lock global state towards thinking of the world as partially ordered and asynchronous.
Someone asked Joe Armstrong at his Lightweight Languages 2002 presentation what synchronization mechanisms Erlang has. Joe responded, "Two. Send and receive."

Exactly.

Why...

...is Sharepoint the suckiest thing on this planet?

Monday, January 24, 2005

JSON: The Fat-Free Alternative to XML

JSON is the JavaScript Object Notation for writing "smart clients" like GMail in JavaScript without bulky XML messages. JSON-RPC is a simple RPC mechanism like XML-RPC. (Via Slashdot.)

I suppose this could also be based on YAML which is more rich than JSON, yet still not as complex as XML. I've not seen a mapping, in the few minutes I've looked, from JSON to XML, such as what's been defined for YAML.

Sunday, January 23, 2005

Communication

As seen on the XPDX Yahoo group...

Thanks for all the comments... One respondent said XP depends on communication, but I work in a corporation.
I guess that corporation does not "empower" their employees enough.

We Won't Get Fooled Again?

Scott Rosenberg writes...

The Bush inaugural marked the final transition of the Bush-family ideology from old-school conservatism, with its abhorrence of abstract schemes of human perfectibility, to a messianic idealism so divorced from reality it gives even sympathizers like Noonan the willies. Bush's vision of human perfectibility may be shaped by born-again fervor rather than socialist theory, but that difference doesn't make its collision with reality any less dangerous.
and writes...
This speech wasn't just soaring rhetoric. It was a lighter-than-air burst of helium verbiage -- lofty language untethered from the perplexing world we occupy and from the messy events of the last four years, sentences floating off into an empyrean of millennial vagaries.

The world is a simple place to Bush... His "untamed fire" is the cathartic inferno dreamed of by people who are confounded by a world they know is out of their control.

Thank You, Ken Anderson

Thank you, Ken Anderson, for your contributions to JScheme. There have been many times when I needed access to Java functionality but not the Java language per se. JScheme makes that kind of programming pleasant and productive. One of Ken's contributions was the Javadot notation which is a large part of ease of this Scheme/Java integration.

From Tim Hickey, another of JScheme's creators, via Chris Double...

I'm sorry to bring you the very sad news that Ken Anderson, one of the co-developers of JScheme, died last night...

Ken touched many lives and brought many communities together.

Saturday, January 22, 2005

Lisp and Concurrent Programming

Bill Clementson has a huge list of Lisp-related concurrent programming references. And if you haven't looked at Erlang yet, it is a list-based semi-functional language. Too bad it doesn't use Lisp syntax.

Bill refers to Dirk Gerrits, building an Erlang-like concurrency mechanism in Common Lisp. Something like this was done for Python as well, called Candygram.

Also related is the ETOS compiler for Erlang that generates Gambit Scheme, which scales as well as Erlang/OTP.

The mind spins, concurrent thoughts swirling all around.

This and That in Politics

I'm reading this and that near each other in my bloglines. They both alarm me for reasons that are not dissimilar.

PyCon DC 2005: IronPython

I didn't see anything on IronPython at the March conference, but Sam Ruby pointed out the keynote.

Structure and Notation of Computer Programs

Agreed...

We're not so much building on the programming state of the art as continually have each generation of programmers rediscover it.
-Bill de hOra

Thursday, January 20, 2005

OOP Is Much Better in Theory Than in Practice?

Lambda the Ultimate discusses the merit of an article as well as its content. Again against the object.

I think OOP is better in practice than in theory. But then I value practice more than theory.

Guessing from the Terminal

Mountains of wisdom from the Mountain of Worthless Information...

Next time you're in a meeting with folks and somebody asks you how long you think it'll take, be sure to tell them whether this is a "from the terminal" sort of guess, or a "from the ground" sort of guess. The difference? Until you've been-there-done-that, and have something against which to measure their requests (meaning you have some kind of experience with the things in question, ideally having built something like it before), your estimates are likely to be all "from the terminal" guesses, and highly likely to be wildly off base.

Smug Lisp Weenies

John Stephen Jacob Anderson, apparently a SmugLispWeenie if there ever was one, writes...

Python was the subset of Lisp that was understandable by C++ programmers.
Being at least a former SmugLispWeenie myself I am having trouble disagreeing. Which is not to say Python is bad, it just loses in certain significant ways that Lisp does not.

Update based comments:
"outside of the CS department"

Ouch.

I think Lisp is safely outside the CS dept. If we could get it *into* the CS dept. maybe things would improve a bit.

"Listing those significant ways"

Syntax extension, syntax itself, closures, continuations (Scheme in this case), to name the top of my list.

I would also include as a result of these, Lisp is a good vehicle for language experimentation (or just defining new object systems), whereas Python provides a simple object system it is more restricted in where you can take it.

Which is not to say that I would not choose Python over 90% of the industry's languages. I would.

Cairo as well as OpenGL

From "The (Re)Architecture of the X Window System" a progress report on Cairo...

This integration of the familiar PostScript operational model within the native application language environments provides a simple and powerful new tool for graphics application development...

Cairo's rendering algorithms use work done in the 1980's by Guibas, Ramshaw, and Stolfi [GRS83] along with work by John Hobby [Hob85], which has never been exploited in Postscript or in Windows. The implementation is fast, precise, and numerically stable, supports hardware acceleration, and is in advance of commercial systems...

Cairo is in the late stages of development and is being widely adopted in the open source community. It includes the ability to render to Postscript and a PDF back end is planned, which should greatly improve applications' printing support. Work to incorporate Cairo in the Gnome and KDE desktop environments is well underway, as are ports to Windows and Apple's MacIntosh, and it is being used by the Mono project. As with Xft2, Cairo works with all X servers, even those without the Render extension.

Also regarding X and OpenGL...
Using OpenGL as the basis for the X server itself will place X objects such as pixmaps and off-screen window contents inside OpenGL objects allowing applications to use the full OpenGL command set to manipulate them...

Whether the "core" X graphics will require any OpenGL extensions is still somewhat an open question.

In concert with the new compositing extensions, conventional X applications can then be integrated into 3D environments such as Croquet, or Sun's Looking Glass. X application contents can be used as textures and mapped onto any surface desired in those environments.

I think this is huge. (Yes, even huger than Avalon). Turning X, per se, into a legacy window system within a portable, modern vector and 3D graphics environment. Just remember, Don't Fidget With Widgets, Draw!

Where did those OpenGL patents go, again? Oh. Yeah. Now I remember.

Via Miguel de Icaza. (Microsoft must be wetting their pants trying to get de Icaza to sign up with them. I'd like to see the offer sheet.)

From the "What's That Again?" Department

From CNN...

"Poll: Nation split on Bush as uniter or divider"
Well then, apparently we have our answer.

Models for Backwards Navigation

Nice alternative implementation of a continuation-like web mechanism for languages without first-class continuations.

Unwittingly explains the *conceptual* model of continuation-based web servers. Languages with continuations offer a better *programming* model by simplifying the web programmer's *mental* model.

If you are familiar with web programming but unfamiliar with continuations, read this one first to get the concept, then read about continuation-based implementations.

Wiki and Comment Spam

I've not written my own wiki or blog system, and not thought about this problem at all. Given that, what if a form of "email authentication" was used to approve wiki edits and blog comments? (I think I've seen this somewhere but can't get google to remind me where.)

  1. Someone edits a wiki page or adds a comment.
  2. The change is suspended until "authenticated".
  3. At this point the person enters an email address (of some determinably accountable, non-transient nature).
  4. The change or some indication is sent to the email address.
  5. Upon receipt of the response or via an http form in the email, the change is accepted.
  6. In addition to other filtering and governing, the site now also has an email address.

Tuesday, January 18, 2005

Being FIT

Will Stott writes about using FIT via Excel.

Proof Carrying Code

I've only read a surface amount about Proof Carrying Code. In spite of my default stance on types, I am interested in ML and Haskell style type inference. So this book looks interesting, covering both topics and then some.

Via Lambda.

Can You Have It Your Way?

Guido's latest wandering around the world of type checking are disturbingly non-committal. I'd like to see his thinking get back to root causes and work forward to solutions. As it stands he's kind of wandering around the space of semi-solutions to unspecified problems.

Uninteresting and not a good use of my time at this point. If you've read this blog you know I am not interested in types much if at all. But whereever you are on the spectrum you most certainly would benefit from language changes being rooted in clearly outlined problems statements and solution explorations.

Monday, January 17, 2005

Objects are Easy? (Maybe Not)

A comment on Scott Rosenberg's blog post about concurrent programming...

Herb Sutter's analogy to object- oriented programming is completely bogus - OO is a bag of tricks for making it *easier* to write programs (often at the expense of making them a little slower), and especially easier to debug and maintain those programs; parallel programming makes it *harder* to write programs, and dramatically harder to make them correct, debuggable, and maintainable. It's just not going to happen.
First I would argue that programming with objects is not easier. But then the question is easier than what? The commentator does not say.

Objects are a code organization mechanism. As the complexity of a system grows, objects help through organizing the complexity. Progammers can use objects to focus on specific problems without being overwhelmed by uninteresting aspects of the system.

But objects are not easy. However I would agree that concurrency is even more difficult, especially given the languages and tools of the past several decades.

The commentator appears to be making an assumption though that the languages and tools of the past will not improve and there will be no incentive to improve...

As a software engineer with over 20 years experience, including a good deal on highly parallel machines, I can tell you for sure that there is not going to be a revolution in software development. Explicit parallelism is just too difficult to get right in general, and the applications that benefit from it have already been optimized for multi-cpu machines since the 1980's.
This sounds a lot like previous infamous predictions like no one needing more than 640k RAM.

Objects are hard, but they offer better notational organization. Processes are hard too, but they offer better dynamic organization. We just need better notations and tools for processes and someday someone may be writing something like "concurrency is a bag of tricks for making it *easier* to write programs".

The same commentator adds in a subsequent comment...

People don't get smarter, so this problem is not going to be solved no matter how much Intel would like it to be.
The same might have been said about programming sequentially before objects, or even modules and structured procedural improvements. The software industry has gotten smarter, perhaps not as quickly as any of us would like.

Friday, January 14, 2005

The Io Programming Language

A friend has been playing with the language Io. I spent an hour or so with him on it. Very interesting.

Io is small prototype-based programming language. The ideas in Io are mostly inspired by Smalltalk (all values are objects), Self, NewtonScript and Act1 (prototype-based differential inheritance, actors and futures for concurrency), LISP (code is a runtime inspectable/modifiable tree) and Lua (small, embeddable).
The language has an interesting evaluation, reflection, and delegation model. Io has a syntax that does not always require an explicitly denoted receiver for a message. Someone just learning Io can scratch their head once or twice figuring out what's going on.

On the whole though Io is a very simple language with what looks like a simple CPython-like implementation. For programmers looking to understand the "prototypes and delegation" approach to object-oriented programming (as opposed to "classes and inheritance"), then Io seems to be a simple way to dive in deep.

Playing a bit with Io felt a lot like the pre-CLOS days of Lisp in the 1980s when there were a lot of experiments with object systems like ObjVLisp, OakLisp, Loops, etc.

Wednesday, January 12, 2005

Scaling to Thousands of CPUs

From Shahin Khan in the Register...

A major shift is coming. Over the next few years, your ordinary applications will be able to tap into systems with, say, 7,000 CPUs, 50 tera bytes of memory, and 20 peta bytes of storage. In 2005, Azul Systems will ship compute pools with as many as 1,200 CPUs per a single standard rack...

What would change about application design if you could do this? Well, think back to what applications were like when you had just 128K of memory in your PC and a 512KB hard drive. The difference between the capabilities and flexibility of applications in those days and now is the level of improvement that we are talking about.

It may take more than a few years to get to these numbers for most of us, but I think at that point the change will be even more dramatic. Unless once again the software industry (you know, the "agile" industry) once again is too slow to keep up with hardware innovation.

One reason why I'm getting back into Erlang after tinkering with it several years ago. I want to understand how to think in terms of processes being about as cheap to create as objects are in other languages. One neat thing about Erlang is the sequential aspects of the language are so spartan, the programmer is forced to think about processes.

A single Erlang node on a single CPU today can comfortably get into the tens of thousands of dynamic processes. What would your system look like running hundreds of thousands or a million dynamic processes and lot of activity is spent collaborating with other systems also running at that scale?

See "Apache vs. Yaws" (Yaws is an Erlang-based web server)...

Yaws is still functioning at over 80,000 parallel connections.
The Java community is discussing a new protected-private keyword. This reminds me of the saying about rearranging Titanic's deck chairs.

Thursday, January 06, 2005

Message to Tim Bray

Larry Maccherone writes in a comment...

I wanted to post this to Tim Bray but his blog didn't allow posts. I've worked with a group here at Carnegie Mellon called the Fluid project. They have developed the ability to statically assure that your concurrency model matches your code. The tools also make it trivial for you to indicate your concurrency model by guiding you in adding minimal annotations in the code (think javadoc).

The fluid web site.

The publications (including Aaron Greenhouse's PhD thesis).

I post it here as a guide to how this might be done in dynamic languages but also to help Tim out with his java concurrency issues. Could someone see that it gets to Tim?
LMaccherone ..at.. cmu ..dot.. edu

Testing Interfaces

Ian raises some concerns about testing interfaces in Python...

Unit tests aren't going to test across the boundaries where people are proposing interfaces -- between loosely coupled components.
I am not sure I understand this point. Say I give you version 1 of a component and the tests deemed "sufficient" for that component. Then I give you version 2 of the component and the tests. If you are able to run version 1 of the tests against version 2 of the component, that should be a reasonable indicator that your own use of version 1 will be compatible with version 2 of the component.

As the provider of the component, I need to know nothing about your own use of that component, for reasonable definitions of "sufficient tests".

Note also that interfaces are more explicit about contracts between components than tests will be. For instance, an interface might demand a method which is not yet being used in your code or your usage; the interface makes it explicit that this method should still be defined, and so predicts future developments, not just current developments (half-implemented dictionary interfaces are a common example in Python, where interfaces would motivate people to finish the work).
The original scenario was an upgrade to a component breaking backward compatibility. If an upgrade provides a new feature then your old code has not used that feature yet. I would not expect that to break backward compatibility.

If an upgrade requires an old feature to require a new method of its old consumers, then that *would* break backward compatibility. However I would expect a reasonable set of tests from the component provider to demonstrate this failure on their part to keep their old customers happy.

Or two interfaces can match methods and signatures, but include extra information about the semantic differences between those interfaces; because it doesn't use duck typing, you won't accidentally mix semantics (you have to explicitly say which, if any, of those interfaces you support). This is particularly a problem with interfaces that define only a small number of methods, or use common methods like __call__. The only way to test these is with system tests, and system tests are always incomplete, there's always some code path or combination you haven't tested.
I guess I would need a specific example that cannot be tested by a unit test. Certainly there are "big" tests that require a lot of "units" to execute. But if each of those units were developed and updated by a series of "unit" tests then I would expect those tests to represent the "differences between those interfaces" as they exist within each unit.

I am having trouble addressing this issue in the abstract. If it is a frequent problem, I would look for ways to avoid it or break it into smaller problems. Maybe this is a scenario where changing the language would solve the problem. I just don't know, but I suspect there are simpler ways to solve the problem on the basis that I have used Python and several languages like Python and I have not run into this scenario. That doesn't mean it isn't real, just that I have not recognized it.

More Ways to Give and Help Children

In addition to my two favorites, Anders Jacobsen is asking bloggers to include these organizations on our blogs. For each of us that does, he'll donate to the British Red Cross. Thanks Anders! I think UNICEF is a good organization to have at the top, reading about their concern for child safety given the number of orphans resulting from the disaster. Pray to change the hearts and minds of people who would take advantage of children like this.

International aid organizations: UNICEF (United Nations Children's Fund) United Nations' World Food Programme Medecins Sans Frontieres / Doctors without Borders (donate!) CARE International The International Federation of Red Cross and Red Crescent Societies UK/Europe: Disasters Emergency Comittee (DEC) - comprises a raft of aid agencies, including the below and others British Red Cross Oxfam Save the Children UK North America: American Red Cross Canadian Red Cross Save The Children Oxfam America Anders Jacobsen: Webloggers: Give to tsunami victims and I'll give too!

Crying Out Loud

All of a sudden blogs are not about an ongoing discussion?

Gordon and Ian both write about the reaction to Guido's posting very preliminary thoughts on what would be a controversial change to the Python language.

The question they raise is this: does the reaction imply that Guido has to be somehow more "careful" with what he chooses to write?

This question puzzles me. I am assuming Guido published his thoughts in order to *get* a reaction. Should the kind of reaction he gets be considered a problem?

Dictators had better have thick skin. Let's assume he can handle it and continue to speculate. That's what blogs are for, right?

You publish your thoughts, I'll publish mine. The result will be a web of thoughts, and presumably this will be better for the whole community as compared to being afraid and secretive.

People don't "need to chill". People need to *write*!

Tuesday, January 04, 2005

I Must Be a Type Magnet

OK, I'm not resisting responding to some comments on the interface type declaration issue.

Interfaces are something that two of the major "framework" systems in Python (Zope and Twisted) have both found missing and wandered off in search of their own solution. If such a system existing and was codified into Python it would make it a lot easier for myself and others to actually rely upon bits of code crafted by other people; specifically feeling warm and fuzzy about you not going around and changing something deep in the guts of your modules that I am not going to find out about until five weeks after updating to the most recent release.
I have found that framework upgrades work best when there are sufficient tests available with the framework and sufficient tests for the system using the framework. These tests reduce the magnitude of lots of problems, especially the scenario described here.

Are you saying that even with sufficient tests, from the supplier and in your own code, you are still getting surprised a month into an upgrade?

If not, which do you think would be the lowest risk experiment for addressing the problem: writing more tests for the framework and your code, or changing the language?

(Note: I am *not* asking if you think type declarations would in some way address this problem. I *am* asking which of these two alternatives would be the lowest risk experiment.)

Monday, January 03, 2005

Freeze!

A big problem with Guido's considering type declarations for Python is that there are now at least three significant implementations of Python. Disregarding how you or I feel about this specific proposal, any big change in the language will put further distance between the implementations or impose a heavy fine on those implementations that are trying to keep pace.

Given the momentum in favor of Python over the last year, if I were the BDFL of Python, I would freeze the language at the 2.4 specification for a few years. This would allow the other implementations to catch up, and allow the CPython implementation to focus under the hood on implementation improvements.

I'd have to see real evidence of pain and suffering to want to extend the language significantly. Let other languages try new ideas, but Python should probably stay close to the current definition which has brought it to the threshold of something big.

Of course Guido's current thoughts just exist on his blog as far as I know. Real changes to the language based on these are a long way off. They could have an ill effect on the current momentum just the same. At the least they could suck energy away from more immediate concerns.

Comments are enabled for the concept of a moratorium. No type checking debates here please.

The Road to Ruin

The dictator is leading us astray and we're on the road to ruin. Sad just when things were looking up.

I thought I wasn't going to write about this anymore, but I can't believe my eyes. Just as Python is getting to the table with the superpowers.

I don't want to comment on this. Sorry, comments are disabled no matter how much fun more type checking debates might be!

Gartner Buys Meta

From The Register...

"This transaction is an exciting opportunity that will give us increased depth in key sectors, geographies and markets, and an increased ability to seize revenue opportunities with the addition of META Group's well-trained, successful sales force," said Gartner's CEO Gene Hall. "In sum, the acquisition will make Gartner a stronger company with increased opportunities for growth and greater resources to offer clients."
I was impressed with Meta's presence and insight at a conference they held a few years back, and with some recent content. My experience with Gartner has been spotty, but lately I've been finding some interesting white papers in their library.

James Robertson has more perspective on Gartner than I do.

Pimp My IT Ride

Chad Dickerson writes...

Smart IT shops should limit unneeded complexity at every turn, choose their customizations carefully, and turn a deaf ear to the siren song of the perfectly customized solution...

In the heat of the moment, no one thinks about what these one-of-a-kind customizations are going to mean down the road.

Do you think "service oriented architecture" is going to solve this problem? Can solving this problem be profitable?

Process Models

Bill de hOra writes...

There's no question that too much business logic is locked up in middleware and systems programming languages. But we need to avoid the industry standard hype cycle on these technologies and be clear on what they can and cannot do...
Process modeling has to balance agility and formality. We cannot allow to happen to process modeling what has happened to data modeling. And then we have to take back data modeling and fix *that* too. Related note.

Out of the Bottle and Into the Box

Daniel H. Steinberg writes...

Let's ship Jini, JXTA, and the Java APIs for Rendezvous along with the JVM. What if easy dynamic distributed networking was available from and to every Java enabled device.
He expands on this in a larger column.

3MLOC

Via Chris Double...

The runtime source code is 3 million lines of Java and C++. That's way too much for what it does...

This includes a mix of awk, sed, shell scripts, and indeed, Scheme! Yes, the JVM source includes jscheme.jar, used to generate some CORBA classes.

Sunday, January 02, 2005

Games

I played a few traditional games with family and friends New Year's Eve and yesterday at a friend's New Year's Day open house.

  • Rummy Royal -- We played for a couple of hours on New Year's Eve. It's been a long time since I've played this. The great thing about it is the variety of each round; you start with a chance to choose or bid on an alternate unseen hand. You can bet on an ace showing up in your hand. Then you play a poker hand by choosing five of your cards. Finally you play all your cards in a Rummy-like game. I pretty much broke even for the night.
  • Scrabble -- This was about the worst game of Scrabble I ever played. At one point I had all consonants. I foolishly traded *all* of them in, and came up with all *vowels*. At this point I had three of the four 'u' tiles and two of the four 'i' tiles. Why didn't I keep some of the consonants? The biggest problem was the play went from the middle of the board down into the lower corners and none of is had a way to build upward.
  • Sequence -- I had never played this game before even though my wife says we have it somewhere! This is a good game for adults and kids to play together. The rules are simple, the pace is moderate, and there's a good bit of chance, but opportunity for strategy to win out.
Some friends were telling me about "Speed Scrabble" but we didn't play. Next time; it sounds fun.

One Startling Sentence

PJE writes in a comment...

He's not saying adding enough negative numbers makes a positive -- although your phrasing sure made me click on the link out of mathematical curiosity!
Which is of course why I wrote that, and is drawn from the idea of One Startling Sentence.

But I chose two startling sentences. (One is the title itself.) I thought of a third, but thought then that was too much.

Take Back Your Data

Again Andrew Newman...

There is a common theme in the IT history towards more freedom. I don't mean free like in free speech, I mean free like in free will.

Lock Free Programming

Andrew Newman points to an article on atomic variables in Doug Lea's concurrent package for Java. Is the attention on concurrent programming going exponential?

More Than Two Wrongs Might Make a Right?

Oliver Steele seems to conclude that adding enough negative numbers may have a positive result. I'm not sure I get all the way to the same conclusion, but the argument is unique.

Update: OK, I definitely do not get to the same conclusion. Saying a type declaration "does a poor job of each scenario, and that means it does a good job of doing them all at once" is a leap I cannot make.

I don't see Oliver addressing the cost side of the equation sufficiently for his benefit side to come out greater. I think the costs can be overcome with creative work, but I'm not sure that's where I'd want my most creative thinkers working. Making type declarations optional alleviates costs, but why bother? I am just one who does not see the benefits.

Since Oliver already points out weaknesses in any one of the scenarios, there is no point attacking them individually. I cannot argue against the whole because I don't see a complete equation that can be analyzed. But I will point out some glaring problems he's passed over.

  • Documentation Providing f(a: String, b: Integer): Boolean rather than just f(a, b) is a weak argument. What are f, a, and b? Those names, well chosen, will tell me more than String, Integer, or Boolean. Moreover using Smalltalk's keyword syntax you have even more expressiveness than this Pascal style.
  • Early error detection Test-driven development eliminates the gap between coding and testing. There is no "earlier" time for detecting type errors.
  • Discouraged from writing tests If your language is so bad that programmers are discouraged from writing tests, my friend, well, type declarations are not going to make up the difference.
This is the last post I will make on type declarations for the foreseeable future. Feel free to comment, but don't expect a response.

I have another post to make responding to PJE, which I have learned is *not* about type declarations per se. Coming soon.

Saturday, January 01, 2005

Coming out of the woodwork

Another prediction...

10. Microsoft will increasingly play the patent card when competing with Linux. Although company officials have downplayed the extent to which Microsoft is threatening customers and partners with the risk of patent-infringement lawsuits if they go with Linux, we hear this is an increasingly key piece of Microsoft's competitive strategy. With Microsoft vowing to file 3,000 patent applications during fiscal 2005 alone, we expect the company to rattle the patent sabers even more in 2005.
Some background...
Microsoft has been slowly but surely deemphasizing its Passport digital-identity service for more than a year. But Passport increasingly is being pigeon-holed as a service for Microsoft properties only.
Another definition of the term "long tail"?

More Predictions

From Bill de hOra, tongue in cheek he says. We'll see...

java.util.concurrent will be a talking point and will produce a rash of articles and guidance as developers weep like children in the face of Doug Lea's 3rd edition of CPIJ.
Doug Lea's collection and concurrent packages have been one of my favorite features of Java for years. This would be a great prediction to come true.
Microsoft will try to hire Doug Lea.
I'd be surprised if this has not happened already. Does dotnet have anything like his concurrent package?
Everyone gets bored talking about AOP.
On the other hand, what if everyone realizes AOP is a nightmare from another era and moves to more simpler and expressive programming languages?
[Rich Internet Application] hacking hits the offline/online sweet spot as developers get fed up waiting for browsers, virtual machines and UI toolkits to evolve in response to Web Sites Which Are APIs... 2005 witnesses a revolution in how most developers are prepared to use obscure languages in production scenarios... The terms process-oriented, crash-first, concurrent message passing, and little language enter mainstream developer lingo. By the summer everyone gets closures and they replace IDE support as the popular distinguishing factor between languages. People start fooling with Erlang after they realise EJabberd is written in it and Herb Sutter's Fear and Loathing in Concurrency article scares the beejesus out of everyone.
Great predictions.
IM becomes a viable alternative to the heavier Grid and P2P technologies for integrators and data crunchers working at federation and Internet scales, but will initially be frowned on as a simplistic and inadequate for 'real work' - the debate will initially look like a rerun of WS-* v HTTP/XML
Nice. No tongue in cheek needed.

Finally...

Patrick Logan and Ehud Lamm become superstars.
Hmm. Tongue must be partially in cheek. Ehud certainly deserves credit and publicity for Lambda.

On the other hand, I don't look good in Spandex and I don't have much of a stage presence.

Friday, December 31, 2004

Recovery

Really just one hope comes to mind at the top of my list for 2005. That's a recovery to some new definition of peace and normalcy for the people affected by the tsunamis.

There are two local (to me) really great agencies involved in the recovery. They are Northwest Medical Teams and Mercy Corps, both here in Portland.

There are many agencies ready to take donations and hard at work on the recovery. If you're unsure of which agencies would be effective, these two have a long reputation of doing good work efficiently.

Beyond that effort, the rest of my list would be filled with hopes for more freedom, kindness, understanding, and generosity. Less war. Less terrorism. Less fear.

More health.

How's the Blogosphere?

A creative lyrical satire on blogging. I was concerned about the dialog portion going off on blogging in general as "saying nothing". I find "traditional" channels saying nothing lately. I'd rather page through a large number of individual(istic) blogs to keep up with my interests than a limited number of sanctioned channels.

David writes in the comments to this post...

It's not so much that I think blogging in general says nothing -- I read lots of blogs and gain a lot from them. The "random nothingness" stuff is a takeoff on the large number of blogs I've come across (mostly in the Java developer space) that have... disingenuously self-deprecating descriptions for people who certainly do have something to say...
So go for it, bloggers.

A (More?) Positive Outlook?

I consider myself kind of a critic. I think we need a lot more criticism in the industry as well as in society as a whole. I try to be constructive even though I miss that mark and have no editor to question me before I hit "Publish Post".

Given the New Year around the bend (as I write this, I'm about 11 hours away from 2005), I am going to post something positive about the mainstream of the industry. Shock!

Here's to a 2005 where we can learn more about Microsoft's Bigtop...

Bigtop consists of three components, all written in C#, according developers who said they were briefed by Microsoft. These are:
Highwire
Highwire is a technology designed to automate the development of highly parallel applications that distribute work over distributed resources, the aforementioned sources said. Highwire is a programming language/model that will aim to make the testing and compiling of such parallel programs much simpler and more reliable.
Bigparts
Bigparts is code designed to turn inexpensive PC devices into special-purpose servers, according to the sources. Bigparts will enable real-time, device-specific software to be moved off a PC, and instead be managed centrally via some Web-services-like model.
Bigwin
According to sources close to Microsoft, Bigwin sounds like the ultimate manifestation of Microsoft's "software as a service" mantra. In a Bigwin world, applications are just collections of OS services that adhere to certain "behavioral contracts." These OS services can be provided directly by the core OS or even obtained from libraries outside of the core OS.
These are terrific goals. Market forces can detract from goals like this. Perhaps better markets can emerge from staying on track; the benefits could be huge.

More Agility in 2005?

Why didn't they teach us in math that integers can "overflow"...

The most serious of the three vulnerabilities involves the Windows LoadImage API Function.

That bug was described as an integer overflow that could be exploited via browsers or e-mail client software. Users who open an HTML message or Web page bearing the image could face security risks.

Here's to a New Year where more programmers use agile languages where integers behave more like you learned in math class.

*Experience* Music Project

This week I went to the Experience Music Project for the first time. Our family joined for the year because (1) it was about the same cost as general admission considering the Bob Dylan T-Shirt they throw in with the membership; and (2) we are in Seattle a lot and (try to, some of us) play various instruments.

One thing I should not is that I am not a fan of the exterior of the building (or of Frank Gehry's architecture generally), but the interior of the building seems to work pretty well for moving a lot of people through a lot of content in a small amount of square footage. Some of the doorways and passages were inexplicably narrow.

The content is what really matters. I'd like to see a wider emphasis than rock and roll, but if you look around a bit you'll find broader content. I'd like to get back up there for Jazz in January.

The best thing about the "museum" is the experiential opportunities. The sound lab is a great place to learn how to make music on various instruments or just to play around. A number of sound-proof booths and rooms have keyboards, guitars, drums, sound equipment, etc. with guided software for absolute beginners or just for free form fun.

Dang if I wasn't embarrassed as a software developer though. In the few hours I was in the sound lab my family and I experienced a handful of frozen computers and handheld devices. More than once "rebooting the computer" was the recovery method.

These are embedded systems and when they break down they spoil the experience even more than a typicalhome computer problem. There's no reason I can see these systems could not fail fast and recover quickly without falling back to the main screen most of the time.

We need to make great new strides in this as programmers beginning in 2005.

Languages and Data Integration

Years of endless debates about static type annotations regurgitating the same information over again have led me to shy awayfrom them in recent years. I've done pretty well, yet here I am in the gravity field of another one. I'm not sure how long this will last but for this day, I am somewhat interested in continuing.

PJE writes in a comment...

Um, just because a given use case can be accomplished without a given tool, doesn't mean the tool is useless, or isn't useful for that use case. If that were true, we'd never use anything but machine code.
I have not found any useful scenarios where I want static type annotations. There's not much more to say than that. I'd like to hear of scenarios others have found useful so I can compare them to my own.
Meanwhile, I notice you haven't responded to the point that solid integration with statically typed languages often requires type annotation; e.g. jythonc's use of special patterns in docstrings. Integration with other languages, documentation, and runtime introspection/adaptation are my use cases for type annotation.
I'm not sure what you mean by "solid" integration. I have done production-quality integration between "agile" languages like Lisp, Smalltalk, and Python with "rigid" languages like Pascal, C, C++.

There are situations where the more rigid languages have required, say, a 32-bit integer or a specifically formatted sequence of values (i.e. "struct" or "record"). "Agile" languages tend to provide integers that are not limited by the word size of the underlying hardware. In scenarios like this then, on the agile side of the integration a check is necessary that the integer will fit in the rigid size of 32-bits and so on. Likewise a conversion is necessary to actually format the scalar data and structured data into the kind expected on the rigid side.

I would not call these expressions "static type annotations" in the agile language though. Rather they are hoops I'd not have to jump through as much if the rigid language were more agile. The language systems I have used that I can recall have tended to allow me a way to express these in the agile language itself rather than in a comment or doc string, but that's a minor point.

Languages like Java and C# are less rigid and given their capabilities for reflection, integration with more agile languages tend to be more automatic. And so Jython and Python.Net among others tend to do most of the conversion automatically without special annotations or coding.

Corner cases require more programmer attention but that's also a minor point. They could agree on a common model for data exchange such as is done in CORBA and it's descendent WS-xxx. Doing integration via CORBA using Smalltalk is so easy I believe had more programmers been working this way the WS-xxx fiasco would never have been. Rather improvements to CORBA's relationship with HTTP would have been getting all the intention. These are market force issues, not software.

Another source of feedback on this is the Inter-Language Unification work done at Xerox. Related to CORBA but more focused on integration aspects and RPC, where CORBA was beginning to branch out into a variety of distributed programming models.

I don't see how type inference helps with either documentation or introspection, either. If it's pure "duck typing", how is the system going to infer that I want an IAtomicStream, say, versus an IRawStream, if the two have the same methods? Duck typing isn't sufficient to distinguish different *semantics* of a type/interface, only its method signature -- and that is not sufficient for documentation!
If you need an IAtomicStream on the rigid side then there is probably enough information to infer that from the code on the agile side. If the rigid specification is ambiguous (well, then it's not really a "specification" is it?)... then, yes, you need to have some kind of an expression on the agile side to disambiguate.

The point is, this is a burden rather than a feature. Why would I *want* to work at this level of detail?

A good bit of information on the power of inference for agile languages can be found in the implementation of the Stalin Scheme compiler, see "Flow-Directed Lightweight Closure Conversion" by Jeffrey Mark Siskind.

Documentation needs to present the semantics of the interface, so that if you are implementing that interface yourself, you know what semantics to implement.
The documentation I want to see is (1) executable examples or (2) technical memos that augment executable examples. Period.
Likewise, for introspection, I don't see how an implicit signature helps. Let's say I want to create an XML schema for a document format that can contain messages that get converted to method calls on some object. If each of those methods uses a different set of methods on their arguments, I'm going to end up defining new types for each method, which might be okay for the computer to process, but for a human being it's going to be a mess to understand.
I am not sure what this example is about. I agree there are integration scenarios where something like an XML Scheme is required to specify the kinds of messages that can be exchanged between language-independent processes. That does not in any way imply that static type annotations are a benefit to the languages I use to implement the independent processes.
Type inference, IMO, is just type checking. It doesn't seem to me to be useful for anything else -- and like you, I'm not interested in type checking! (I suppose it could be used for optimization in some cases, but only if you can statically prove a concrete type - not just an operation signature.)
Type inference is better than type annotation. The real benefit I see is in getting even more capable agile modeling and inferencing tools by extrapolating from recent work in model checking. Simple, even compound, type checks are uninteresting. We do need a lot of help with concurrent state machines however. Tools that can tell me more about *real* behavior problems rather than simple type checks should be getting all this attention.
Now, perhaps I could be wrong, and there is some other way to accomplish these use cases that's just as clean. If so, I'd sure like to see them. I personally don't see that any of my uses for type annotation *can* be satisfied by inference; there's just not enough information. Heck, I don't see how a *human* could do the inference in the absence of added documentation, let alone a computer. It would be little more than blind guessing.
I think I see one use case here... how to integrate languages that have different data models. I agree system integration requires some kind of agreement on specifying data integration. I'm not sure what else I am seeing here.

OK. I am getting tired of writing about type checking. I think we're in agreement on this use case of language-independent data integration.

Prediction of Hope

Don Box has some predictions for 2005. I like reading people's ideas, but I only have hopes, not predicitions. One of Don's predictions leads to a hope of mine...

The term SOA will have been beaten to death and the software industry will invent or recycle some equally vague term to replace it. The term has zero differentiation value at this point and marketing teams across the globe are looking to coin a replacement that will give them something more interesting to say about their middleware than "we move messages around really well." If "Business Agents" become the buzzword de l'année, I expect Mary Jo Foley to hunt me down and kill me.
Perhaps more end users in 2005 will demand that their programmers find better ways to move messages around really well and dispense with the buzz words.

Another interesting set of predictions from Carlos Perez' Managability blog. Especially those on Rich Internet Applications, Grid Architecture and REST, and Scripting Languages.

Thursday, December 30, 2004

Dynamic Types and Naked Objects

PJE writes in a comment...

There *are* other uses for type declarations besides type checking. For example, the Naked Objects framework in Java uses method signatures to figure out what actions might apply when one object is dragged-and-dropped onto another object in a GUI.
Jayson Vantuyl responds in another comment...
In response to the above comment, I would point out that there are other (better) ways to do what the Naked Objects Framework (NOF) is doing in Python.
I wrote and demonstrated a Naked Objects framework in Jython at the 2003 XML DevCon. One reviewer...
14:46 - Patrick Logan - "Point & Click XML Applications" - an interesting discussion and demonstration (using Jython) of point and click application development... to me one of the most interesting points was that at the end of his presentation he pointed out that most of these concepts are extensions of things started 15+ years ago such as Lotus Notes, Agenda, Naked Objects, Tuple Spaces, etc.
I didn't allocate as much time as I'd hoped, and I wasn't so happy with the result. Nevertheless, the system did not require type annotations in the Python. Rather the end user is able to define their own data types in the GUI itself (including "Any").

If I were to continue with that work, I'd consider making these "end user annoations" optional and provide inference as well as modifying the inferred type dynamically as users create more objects. (With some kind of confirmation of the user's intentions.) But I would not think language extensions to Python would be worthwhile.

Back to PJE...

Documentation, of course, is an even more immediately useful use case. A good syntax would be more precise and more brief than docstring explanations.
I think there are more useful kinds of documentation to write other than type annotations. I'd like the system to infer types and tell me what it determines the current types might be for some method. Moreover, I'd like the inferencing mechanism to consider the test code and data as well as the application code.

Given those inferences I could correct my code, add new tests or test data, or have the system keep the inferences around as automatically generated documentation if so desired.

The documentation I write myself would be more useful in the form of technical memos (see the complete Episodes pattern language).

Is there a danger of over-declaration by newbies? Sure. I'd probably find it really annoying to use code with excessive declarations, if those declarations actually restricted anything. However, if they were *just annotations* (or were adaptation specifiers per PEP 246), then I could still pass their code any darn thing I pleased, and we could all continue to be happy.
Yikes! What would be worse, type annotations that *are* enforced or type annotations that are *not* enforced?

Smalltalk Keeps Going

Here's how Michael Lucas-Smith keeps his Smalltalk servers humming with new capabilities.

Blog Archive

About Me

Portland, Oregon, United States
I'm usually writing from my favorite location on the planet, the pacific northwest of the u.s. I write for myself only and unless otherwise specified my posts here should not be taken as representing an official position of my employer. Contact me at my gee mail account, username patrickdlogan.