Sociology

Quote from someone named Jerry back in 2003...

It is for sure a VHS world when people flock to XML, XML namespaces, XML parsing, and DOM and then flinch at a parenthesis.

No, this is not related to any predictions for 2008.

Milemeter: Buy Your Insurance By the Mile

Brady Forrest writes about Milemeter, a new insurance startup. First, the words "insurance" and "startup" seem to form an oxymoron. But I don't see them listed, so I read on.

An interesting aspect of insurance to me is that the "product" is completely information-based (most of my career has been in software for the design and manufacture of tangible, electronic products; I currently work in the insurance industry). Unlike some other information-based products in financial services, though, insurance seems to be more heavily regulated, and the period between events is long (days, weeks, months). The most successful sale of a product (i.e. no claims, no changes to the coverage) has one significant event a year, (re-)issue, and some minor billing/payment events.

Compare this to financial portfolios, which can be wide ranging and nearly unregulated (cf. the current sub-prime crisis and obscure bundling as supposedly secure instruments), and the period between trading events can be measured in sub-seconds and/or distributed around worldwide markets.

On the one hand there does not seem to be a lot of pressure to change the way information technology works in insurance. On the other hand all these aspects seem to open up new opportunities for change.

Brady observes...

As you may have guessed, they are built on AWS (you can see a video discussing their usage on their blog). They are also using Ruby on Rails with Postgres...

This is what I want to see, large, black-box industries being taken down and made consumer-friendly. (Can the health system please be next?) I don't really know what I pay for with my current insurance, but with Milemeter I'll have a much better understanding.

The internet will sooner or later affect all these industries. (Amazingly much of the current B2B transactions in insurance takes place over proprietary networks, that is, when they are automated at all.)

The established insurance IT has to get its cost of change significantly lower. The best way to do this is to copy the way software is developed for the internet. As Milemeter demonstrates, this will come from the "outside" whether or not the "inside" is ready for it.

I had a chance to visit with Steve Loughran and some of his local friends, when Steve was in Oregon last week. We had a good talk about all these changes, where they are trending, and which kinds of organizations are doing what along those trend lines.

There is no doubt the cost of change is the limiting factor in established organizations from following those trends as aggressively as possible. The opportunities are there and the pressure to change will increase.

Chris Gay of Milemeter notes in the video, linked above, "Amazon Web Services is a pay-as-you-go infrastructure and Milemeter is a pay-as-you-go insurance provider". The ability to use nimble infrastructure(s) will aid the product itself to remain nimble.

Patterns of Change

Reg Braithwaite knows...

Of course I recommend reading the original. But may I add, please do not get sucked into arguing whether Design Patterns are good, or whether IDE refactorings really work, or any of the other technical points that are so much fun to rehash for the millionth time.

Instead, consider the cultural forces at work. Cultural problems cannot be solved with technology. If you are an advocate for change, ask yourself what sort of cultural change is needed, not what sort of technical problems need to be solved.

RubyCamp 2008 in Vancouver

This announcement came around...

RubyCamp 2008 in Vancouver on Saturday, January 26th.

RubyCamp is an one-day gathering for Rubyists and Railers.

When and Where:

WorkSpace in downtown Vancouver, B.C., Canada
January 26th, 2008 from 9:00 to 5:00

We've been getting into the JRuby here, and a couple days in Vancouver might be nice. This may go on the list of game-time decisions.

Where's the Beef?

Subbu Allamaraju shows that a significantly more RESTful api than Amazon's SimpleDB is not so difficult to conceive.

The SimpleDB API is neither resource oriented nor HTTP friendly. Having said that, how should such an API be designed in a resource-oriented manner? Here is my take, a version-0.1 of a RESTful SimpleDB. In the design below, I tried to keep the semantics of this version as close as possible to the official SimpleDB API

The Wayback Machine: Internet Archive

James Robertson and Robert Scoble lament the loss of one's data one old sites they themselves do not own. But the Internet Archive's Wayback Machine does a reasonable job of rescuing this. (Could wayback rescue your facebook if that were to disappear?)

e.g. my old Radio Weblog is no longer around: http://radio.weblogs.com/0100812/

However the Wayback Machine can find it: http://web.archive.org/web/*/http://radio.weblogs.com/0100812/

Note that the wayback machine continues to archive the "not found" page. Actually whatever service is running the old Radio is returning a 300, "multiple choices", rather than a 404, "not found".

So using the archive you can actually see when my Radio went off the air, sometime between December 7, 2003 and April 5, 2004.

It looks like the archive has not been able to retrieve at least all the longer essays, which Radio stored under a "story" URL.

Also without exhaustively searching the archive, the earliest criticism of WS-* that I have found of mine is August 25, 2002. See "Protocols, Documents, and Transactions"

Thud

How did Amazon allow this design out into the wild? Apparently they've already had a private beta period with non-Amazon developers. No one suspected this use of HTTP GET to be a poor choice?

It's 2007. There are all kinds of basic REST reference and getting started materials. Didn't the whole net go through that "Google is pre-caching everything it can GET" episode a couple years ago. That's the episode where the Rails folks learned that GET should not have side effects.

Wow. Hopefully Amazon will POST a fix asap.

Subbu Allamaraju calls this "SOAPy REST". Didn't even the SOAP folks learn to avoid these kinds of GETs a couple years ago?

IBM published an article recently on using HTTP to access DB2 and IDS. The URLs they provide need improvement, embedding the operation in the URL. This despite that they do seem to be distinguishing between GET and POST. From the information at that site and another bit I could get behind their developerWorks login, it's not clear to me yet whether they allow the use of GET with a URL that includes a destructive operation.

Maybe the RESTful Web Services book will begin to turn these things around. But that Amazon did this in 2007 is a bit of a shock.

Update: Steve Loughran nominates SimpleDB for a new award...

I nominate it for the 2007 Restless awards, in the much contested category of

"things that claim to be RESTful but do side effects in their GETs" along with the ever popular "SOAP endpoint in disguise" category

I know this mailing list has not, historically, had such awards, but now is as good a time to start as any.

Yeah, maybe there should be annual (or more frequently, on Internet-time, awards for "RESTless" and "RESTful" web services that degrade or enhance, RESTpectfully, the web.

Column-Oriented RDF Storage

A couple years ago it occured to me that a database with column-oriented storage, such as Sybase IQ, might make a reasonable database for storing RDF data, where a star schema can be seen as a way to represent related tuples together.

Now it turns out some folks have been working on such a thing... C-Store is an open source, column-oriented databse. The paper "Scalable Semantic Web Data Management Using Vertical Partitioning" (pdf) discusses using C-Store for RDF...

Efficient management of RDF data is an important factor in realizing the Semantic Web vision. Performance and scalability issues are becoming increasingly pressing as Semantic Web technology is applied to real-world applications. In this paper, we examine the reasons why current data management solutions for RDF data scale poorly, and explore the fundamental scalability limitations of these approaches. We review the state of the art for improving performance for RDF databases and consider a recent suggestion, “property tables.” We then discuss practically and empirically why this solution has undesirable features. As an improvement, we propose an alternative solution: vertically partitioning the RDF data. We compare the performance of vertical partitioning with prior art on queries generated by a Web-based RDF browser over a large-scale (more than 50 million triples) catalog of library data. Our results show that a vertical partitioned schema achieves similar performance to the property table technique while being much simpler to design. Further, if a column-oriented DBMS (a database architected specially for the vertically partitioned case) is used instead of a row-oriented DBMS, another order of magnitude performance improvement is observed, with query times dropping from minutes to several seconds.

Release It Again

Pete Lacey -- what he said, about Release It and Michael Nygard's blog.

If you find yourself in a panic over some centralized resource, wonder if the full costs are accounted for. What alternatives might exist for decentralizing, and how do the cost/benfits really add up over time?

The cost of operations is dropping. The cost of change is still too high for many to take advantage of that though. By the time we can get our systems onto more budget-friendly architectures... well, I guess heading in that direction puts you on the path toward even better things.

Depending on the business, if one extrapolates out from one's current position, through the point where more open/available/scalable systems are in use... well, then is this evidence that for most of us, our ultimate position is out in "software as a service/utility" land? Exactly who should be in the data center business five to ten years from now?

Another data point shows up. (Oh, and speaking of cost of change, it's in Erlang and it provides a REST api available from any language.)

From the SimpleDB FAQ...

Q: Where is my data stored?

Amazon SimpleDB stores your data in our multiple data centers in the United States. We anticipate adding other geographies over time.

Q: Does Amazon store its own data in Amazon SimpleDB?

Yes. Developers within Amazon use Amazon SimpleDB for a wide variety of projects. Many of these projects use Amazon SimpleDB as their authoritative data and query store and rely on it for business-critical operations.

So, "business-critical" seems kind of reliable. Not just scaled out databases, but scaled out data centers. Those are even more expensive to operate on your own.

Yahoo Flex Skin, Other Flex News

Among other recent Flash/Flex/Air/Adobe news, Yahoo released an open source skin for Flex.

Elsewhere new versions of Flash/Flex/Air from Adobe have been released and in some cases newly opened up.

I don't feel compelled to use the Flex Data Services, but for those who do, or when I do, today's news should be encouraging. You can use their open source implementation, or use their open specification for an alternative, e.g. in some other non-JVM language.

Embedded Lightness

I was out with a long-time friend and former co-worker last night. He's been working the last year or so on software for managing blade servers. Technically this is an "embedded system". Some veteran embedded system developers on his team have been learning just how much their field has changed.

Essentially the system consists of linux, lighttpd, php, and sqlite. Where the veterans were set on writing a lot of C and custom protocols for handling events in the server (blades coming and going, for example), they've been able to do everything using the web's architecture. They wrote some C to enhance the php interface to snmp.

They had some pre-existing C code that has a memory problem after some number of days. No problem, they just kill and restart the process well before that. When that code becomes the most important problem, they'll look at it.

Job Opportunity

Mike's got a job posting up at his new gig. You'd be working with a smart, funny, curious (in the good sense!) project manager/developer, on an interesting problem, and a high-powered executive team.

You don't have to live in Portland, although why would you want to live anywhere else?

Safety Dance

Douglas Crockford at XML Conference 2007. You can dance if you want to...

The current browser implementations have problems because they share all of the information between the current sessions (problems with cookie stealing, replay attacks, and chrome changes). That’s the dangerous web 1.0. Now, we’re trying to intentionally mashing stuff up (which we’d always tried to prevent when unintentional).

Meet the new Reg...

Paul Fremantle on WSO2's registry work...

For a while I've been thinking that the SOA registry space has been a little overcomplicated...

So fundamentally the approach we have taken is to build a registry/repository based on REST concepts. And as we looked at the REST space, we kept noticing how close the Atom Publishing Protocol (APP) is to our needs, so we've made that the public remote API to access the repository. Of course, if you are just browsing the registry, you only need a browser - APP is mainly there to support updating resources.

Of course, using Atom and APP gives some really nice benefits too - like being able to subscribe a feed of new resources that meet your search criteria.

Glen Daniels was talking about this registry in the hallway at QCon a few weeks ago. Nice. You could definitely see Glen's gears turning during the sessions and discussions.

Reg

Some folks have been quoting Reg Braithwaite lately. (Steve Vinoski did just a bit ago.) Here's another...

I have never met someone who desperately wanted to be great but failed to be at least decent.

That's a good observation. And almost Goethe-like.

The name of this band is Talking Heads

I was poking around youtube for the Late Night with David Letterman show (the one back on NBC), the episode back around 1983-84, where the entire screen rotated clockwise over the course of the hour.

I've not found a clip yet. But I've been watching other clips, like this one...

Amazing. There was a time as an early 20-something when we *had* to watch Late Night and talk about it the next day. The current Letterman is more mild, but still bits like, "Is this anything" and "Will it float" provide glimpses back to the original show, which at the time seemed like "anything goes".

I wish I could find a clip of that episode with the screen rotating slowly over the hour.

Andy Kaufman and Elayne Boozler

Andy Kaufman and Elayne Boozler...

Boozler I think has gone under appreciated over the years. She keeps up with Kaufman here. Painfully hilarious improv.

"http is like air"

Quote of the day from an IBM exec on the importance they are placing on rest, http, atom being everywhere.

And Then There Were Three... Then Five Again

Mike? He gone.

He formed the band, a little over a year ago, chartered to launch an open source project within a large "vertical". That was four charters, a CIO, one viking, and Joanne ago.

Don't ask. That's the nature of large "verticals" and being a small team, unlike any other, weaving in and out of the established structures. We got pretty good at writing charters on the wiki.

But in between we've done some pretty good work across a couple tables of old Dells running the Ubuntu server. And we're still being asked to be creative, agile, and infectious, which is not often the nature of large "verticals".

This fourth charter could stick. We're back to a full complement of five. And a real budget. And simple tools for restful web services.

Good luck, Mike. You made a lot of good things happen and made them a lot of fun. And took more than one for the team. Now your original charter has come around... nice.

Bird of prey.

Learning Scalable Internet Services

Via Planet Trapexit, and this interesting post from RightScale on the network performance of Amazon EC2 and S3, there is this fascinating page from a UC Santa Barbara course...

"CS290F - Scalable Internet Services"

The project consisted of building a transactional dynamic web site in Ruby on Rails and running on Amazon's Elastic Compute Cloud (EC2). Each site had to hold >100'000 database records that could be searched and explored, have user accounts, and include some form of transaction, such as a shopping cart check-out.

Each project then had to be deployed on multiple servers on EC2 and the groups had to use httperf to demonstrate that they could scale the performance of their site by running a front-end load balancer server, a database server, a memcached server, and up to 10 application servers. All this had to fit into a 10-week quarter, with none of the students knowing either Ruby or Rails at the outset!

More information from the instructor's RightScale blog. The wiki link there is broken, but at least pieces are still around, including the course material.