The Processing Around Restful Web Services

JJ Dubray asks and Stefan Tilkov responds...

Is your solution also about implementing complex processes that are likely to change on sophisticated business objects such as an insurance policy or claim?

I’ll happily agree that if you’re after a BPM-style solution, REST is not the first choice... clearly there’s no equivalent of BPEL for plain RESTful HTTP.

I think this is another false dichotomy. The most complicated business process could still incorporate GET, POST, PUT. Why on earth not?

As it happens, I work on Restful Web Services for insurance, including policies and claims. So if JJ or anyone wants to work out some details, that collaboration could be educational for both of us.

Update: I created a wiki for this collaboration.

Scala Buzzzzings

All the buzz about Scala again, suddenly.

Via Stefan Tilkov, from Doug Pardee...

I’d long felt that combining a powerful static type system with functional coding would be almost ideal. Now that it’s been done—and done quite well—I find that it’s “not all that” after all.

The documentation is weak... A book is in the works... I haven’t read any of it, but I don’t have much hope...

I’ve already discussed the complexity of its type system... But with Scala’s abstract types, a variable or parameter or whatever can be declared to be of a type that is defined somewhere in the inheritance tree… and not just superclasses, it could be defined in a subclass as well, including the possibility of an anonymous subclass. And since Scala allows classes to be defined darned near anywhere, tracking down all of the possibilities is not fun...

def sum(l: List[int]): int = (0/:l){_+_} Isn’t that lovely?

I agree with Doug's complaint about weak, out of date information. But I have the current book draft and have read a good deal of it already. The book seems a good place to start to learn functional programming, especially for someone with Java or C# experience. There are some good introductory books for Haskell as well, and this books seems about as good. Unfortunately about twice the size as the Haskell books, though, as this book attempts to explain most of the Scala language, while the Haskell introductions I've been through do not cover all of Haskell.

I've written several times already how I am at best unused to thinking with more modern function type systems. This dynamic language programmer is used to sending messages to objects and applying any number of any kind of argument to a function without a darn compiler insisting that it must have a valid type theory for what's bounding around in my head.

As for that stretch of Scala syntax above, I do think sometimes Scala, like Haskell, goes way over the top with its cute, concise notation. I can handle this ok, since I am used to the concepts like "fold left" and "fold right". If your a Smalltalker, think "inject:into:". Then once those little bits of syntax can be translated into a familiar vocabulary, it's not so bad. But not great for beginners, I'd agree.

DRM Disasters: The Song Remains The Same

Jimmy Guterman describes a DRM / installation "catastrophe" with Adobe's Creative Suite on Windows. (I'm not sure if the problem is with MSFT, or Adobe, or both).

He concludes...

Yet another case in which copy protection and productivity don't go hand-in-hand. The problem is solved, but the great error message lives on.

Reading this brought up memories of the security dongle fiascos back in the 1980s.

The Good Examples Would Help

Update: Stu has a good response in the comments.

Stu says there are some good things in ws-* that could or should be translated into the Rest world. That may be true, but I cannot tell just from his descriptions. The missing pieces that would help are really the missing pieces that had led to my doubts in ws-* all along.

Those pieces are really good examples.

I can see that WS-Security, WS-Trust, WS-SecureConversation, and Security Assertions Markup Language specify a *lot* about security and trust, very little of which (admittedly) I do not have much of a grasp of. Why do I need them though? Are there really clear examples of real world usage that I can follow to understand why these were used over other capabilities, and how they were used in a complete path to production, and evolution in production over time, and what else was needed to provide all the security and trust of the complete solution?

Multiply that by the rest of Stu's list: transactions, "coordination", "choreography", BPEL. That's a whole lot of stuff that needs explaining. I am a reasonably bright person. I can even curiously dig into things I don't understand and begin to make sense of them, when I put my mind to it.

Years after first being turned off of the seemingly impenetrable ws-* specs, and years after first (of many times) being told that as an application developer, I simply don't need to understand much of the specs because the vendors understand them on my behalf, I still cannot rationalize why I would need anything on Stu's list, let alone anything on the rest of the ws-* heap (because ws-* is way messier than a "stack" from my observations).

Lists of capabilities are insufficient for me. Maybe that's just me, but oh well, it still frustrates me because I feel as a fairly experienced developer I keep getting the sense that I *should* at least understand these things better even if I am still opposed to them.

Good examples would help me. I can find all kinds of them in the Rest world. I can figure out how to write my own without too much time or effort. Why can I not say the same thing about even Stu's abbreviated list?

That Hardware / Software Analogy Applied to Closed / Open Source Software: Bzzt from this vantage point

Via Stefan Tilkov, some observations on closed vs. open source software from Jaron Lanier...

A closed-software team is a human construction that can tie down enough variables so that software becomes just a little more like a hardware chip—and note that chips, the most encapsulated objects made by humans, get better and better following an exponential pattern of improvement known as Moore’s law.

Yeah, and we're seeing so many innovations occurring on closed software today, compared to open software, it is frightening. (insert "sarcastic" emoticon here)

Even the iPhone is based on an open source Unix. I don't think Jaron is being fair to compare the iPhone to Linux. Maybe compare the iPhone to Android-based systems in a few years. That would seem fair. Even then I would expect some of them to have various "closed" aspects.

And let's not take that hardware analogy too far. I think it was Edsger Dijkstra who observed about 30 years ago or so that CPUs iterate over the same design but different process technologies. That's more or less true still, but where it is less true today what also is true is that the design complexity and design approaches are more like software than they are like the hardware of yesterday.

Penultimately, let me note from personal experience that with either closed or open source, the challenges of getting "better and better following an exponential pattern of improvement" in software is damn hard. I've not seen many systems that do that well, but the best candidates at this point seem to be open in various ways.

Finally Jaron's entire debate does not seem worth having. Questioning whether closed or open source is "the right" model is a false dichotomy, a source of info-pollution. The answer is clearly, from observation, somewhere in the middle. Sometimes a little this way, other times a little that way.

CouchDB Finds a Home

Damien Katz is going to IBM to work on CouchDB full time. CouchDB is now intended to become an Apache project.

That's a great development. Good for IBM picking it up, too.

Not knowing CouchDB too deeply, but the basic gist of it, I wonder how well the approach would translate into a cloud-based system, not too unlike Simple DB?

The database world is suddenly (to me at least) looking significantly different than even a year ago, given CouchDB, Simple DB, and the research at MIT on C-Store, H-Store, etc.

Last year at this time my co-workers and I were diving into a serious look at Jini/Javaspaces for some intra-data-center messaging and persistence needs. The community was what we decided was the biggest lack, but the complexity was high on the list, and the Java-centric nature was ok, but not... (alright Jini community... I know it can be used with/from other non-Java systems...)

Given systems like those above, I wonder which direction we would have favored back then.

Scalagain

Respectable people (1, 2) are still looking at the Scala programming language for the JVM. I took a reasonably deep dive on it back in the summer. As I wrote back then, I've tinkered back in the day with SML, Haskell (and a bit of other functional languages like Clean). But being significantly deeper into "dynamic" languages like Lisp, Smalltalk, and now somewhat Python and Ruby, I never really got over the curve of "thinking in types".

There is a difference between "type-driven programming" and "test-driven" (or in Lisp, repl-driven, or in Smalltalk, workspace-driven). I'm not sure what "type-driven" feels like in real settings. And so after a week of kicking Scala's tires off and on, I decided it's nice but not enough in my wheelhouse to get excited about.

Reading folks blogging about their own Scaladventures and reading that a new book (400+ pages!! That could be good or bad) is available (currently as a draft), I purchased the book and will tinker again, at least for a few more days, with Scala.

I just got my access key to Amazon's Simple DB, so maybe I'll dream up something interesting there with Scala. Integrating the dynamic, semi-structured nature of systems like Simple DB with the compile-time rich type checking of a language like Scala is an ongoing area of, em, interest. Dynamic languages never get in my way with things like Simple DB.

MSFT and the Clouds

Bob Warfield writes about Microsoft and their "rift with the web"...

While I am again a whole-hearted advocate, giving your software over to a hosted environment and all of its attendant API’s (RESTful or not) is a big step towards lock-in, no matter how you look at it. This is again an area where Microsoft’s old school monopolist behavior won’t serve it well. There will be fear, perhaps unreasonable, that Microsoft will take unfair advantage if handed the keys to your kingdom by hosting on their cloud infrastructure. The problem is that they’ve failed to conduct themselves as the Swiss do in matters of banking. They are voracious competitors and seemingly always will be. It isn’t enough for them to win, others must lose.

Initial reaction: avoiding, or at least reducing the amount of, lock-in with any service provider will be important. My additional reaction to a MSFT service would be, yeah, especially with MSFT. But that's probably largely irrational. If the service is something like the ones we've seen so far, with comparable pricing, who cares if it is MSFT as the provider?

This is just an opportunity to bring a couple of related thoughts that have been bouncing around my head lately...

One is that the pricing of Amazon's services, and their payment services, really usurp what I think has been a long-held dream at MSFT to institute some kind of micro-transactional payment mechanism. Apparently now they have a model to emulate, so we should start seeing these kinds of services from Microsoft.

But wait, isn't one of their recent services a sort of "Biztalk in the Clouds"? Maybe they will completely rift with the web. Astoria is also referred to as "ADO.Net Data Services" but they say it has a Rest api and can be used from AJAX front ends. I assume this means it can be used by any HTTP client, but the documentation is Word and I dislike having to download that just to find this out.

But all this leads to another question: can anyone *other* than MSFT afford to write a cloud service using MSFT products? I assume Amazon and others use primarily open source tools and products, and when they do not then they are still Unix-based.

MSFT obviously has no licensing issues around building a large cloud based on windows and dotnet. Would anyone else choose to do so?

Big IT and the Clouds

Tying a couple folks blog posts together. James Robertson on SimpleDB, etc....

Bypassing IT just went from being somewhat painful to being a line item in your budget.

And Bill de hÓra on pretty much the same subject...

This seems to me like the desktop computer revolution of the eighties or the email/web of the ninties - consumer tech that infects the enterprise.

Yep. Like the Apple II and Viscalc just showing up in the office because they could get a job done. Like the Mosaic browser just showing up in the office because... oh, wait, that was for porn. But like the sales dept signing their own deal with Saleforce because someone's IT could not get a job done.

I suspect large enterprises will find their way to the clouds in fits and starts, mostly from outsiders being able to get a job done and that job can be paid for on a meter. At some point there will be pressure from Sales, or from Marketing, or even Accounting, to integrate with the cloud.

Certainly by that time CEOs or someone should be asking IT why their costs are so damn high and progress so awfully slow compared to all those new systems that run in the clouds. In 2007 we could finally see that ws-* will end sooner rather than later. Now we are starting to see that the big IT data center will also be shorter lived than at least I imagined a year or two ago. It won't be soon, and it won't be painless, but at least now we have the end in our sights and we can start taking aim for that goal.

Sociology

Quote from someone named Jerry back in 2003...

It is for sure a VHS world when people flock to XML, XML namespaces, XML parsing, and DOM and then flinch at a parenthesis.

No, this is not related to any predictions for 2008.

Milemeter: Buy Your Insurance By the Mile

Brady Forrest writes about Milemeter, a new insurance startup. First, the words "insurance" and "startup" seem to form an oxymoron. But I don't see them listed, so I read on.

An interesting aspect of insurance to me is that the "product" is completely information-based (most of my career has been in software for the design and manufacture of tangible, electronic products; I currently work in the insurance industry). Unlike some other information-based products in financial services, though, insurance seems to be more heavily regulated, and the period between events is long (days, weeks, months). The most successful sale of a product (i.e. no claims, no changes to the coverage) has one significant event a year, (re-)issue, and some minor billing/payment events.

Compare this to financial portfolios, which can be wide ranging and nearly unregulated (cf. the current sub-prime crisis and obscure bundling as supposedly secure instruments), and the period between trading events can be measured in sub-seconds and/or distributed around worldwide markets.

On the one hand there does not seem to be a lot of pressure to change the way information technology works in insurance. On the other hand all these aspects seem to open up new opportunities for change.

Brady observes...

As you may have guessed, they are built on AWS (you can see a video discussing their usage on their blog). They are also using Ruby on Rails with Postgres...

This is what I want to see, large, black-box industries being taken down and made consumer-friendly. (Can the health system please be next?) I don't really know what I pay for with my current insurance, but with Milemeter I'll have a much better understanding.

The internet will sooner or later affect all these industries. (Amazingly much of the current B2B transactions in insurance takes place over proprietary networks, that is, when they are automated at all.)

The established insurance IT has to get its cost of change significantly lower. The best way to do this is to copy the way software is developed for the internet. As Milemeter demonstrates, this will come from the "outside" whether or not the "inside" is ready for it.

I had a chance to visit with Steve Loughran and some of his local friends, when Steve was in Oregon last week. We had a good talk about all these changes, where they are trending, and which kinds of organizations are doing what along those trend lines.

There is no doubt the cost of change is the limiting factor in established organizations from following those trends as aggressively as possible. The opportunities are there and the pressure to change will increase.

Chris Gay of Milemeter notes in the video, linked above, "Amazon Web Services is a pay-as-you-go infrastructure and Milemeter is a pay-as-you-go insurance provider". The ability to use nimble infrastructure(s) will aid the product itself to remain nimble.