"I have a mind like a steel... uh... thingy." Patrick Logan's weblog.

Search This Blog

Saturday, May 24, 2008

Steve Vinoski on RPC and Messaging

There's been a stream of messages on the erlang list about RPC. Steve Vinoski's put an explanation of RPC and his position on his blog. Now he's followed up on the email list with a position on integration generally, and pub/sub messaging in particular...

One of the most effective forms of enterprise integration I've seen over the years is publish/subscribe messaging. I worked many years for CORBA vendors, and we'd often lose potential deals to messaging systems. Message queuing systems work well because (in no particular order):
  • they don't pretend to be programming language procedure or method calls, so they avoid the associated impedance mismatch problems
  • they don't try to hide distributed systems issues
  • coupling is low -- drop a message into a queue here, pick up a message from a queue there
  • queues can be persistent, or more generally, delivery guarantees can be varied as needed
  • asynchrony
  • payloads need not conform to some made-up IDL type system
  • getting two different messaging systems to interoperate is easier than getting two different RPC or distributed object systems to interoperate
The problem with messaging systems, though, is that traditionally they've been quite expensive. Thankfully, I believe AMQP (http://amqp.org/) solves that issue nicely, and of course Erlang is the perfect way to implement it, which Alexis, Tony, and the rest of the RabbitMQ guys have already done (http://www.rabbitmq.com/).
I agree with Steve's take. The best pub/sub mechanism I have experience with has been Tibco's rv. All the Tibco layers above it have been enterprisey bits to make a sale. The basic rv bus is super-easy to just use from C, Python, Java, anything. Horribly expensive back in the day, and I assume it still is.

AMQP should have helped address this. But I've not seen much on the internets about AMQP over the last many months. I wonder if this is going to go anywhere? Or am I just missing the traffic?

I'm not sure XMPP pub/sub in the enterprise is going anywhere too quickly either, but that's another candidate for getting beyond vendor lock-in for enterprise messaging.

Why on earth Cisco is defining a new RPC protocol is beyond me.

And to any AMQP folks: the redirect from http://amqp.org/ seems broken.

Enterprises should be moving toward HTTP first and foremost, figuring out what's happening with XMPP for various other kinds of messaging, and/or AMQP in situations where HTTP and XMPP don't seem to cut it.

9 comments:

Nat Pryce said...

AMQP and TibcoRV are in different spaces. AMQP is for guaranteed messaging, RV for low-latency, highly scalable, best-effort messaging. The nearest Tibco product to AMQP is Tibco EMS (which is an excellent messaging platform).

XMPP could be seen as a similar to Tibco RV, since it is also a best-effort messaging protocol, but it's in no way a serious competitor.

Patrick Logan said...

"AMQP and TibcoRV are in different spaces. AMQP is for guaranteed messaging, RV for low-latency, highly scalable, best-effort messaging."

That's technically so. Tibco rv is UDP without "routing daemons" across subnets and EMS as I understand it is a JMS superset built on TCP. Tibco rv does have some level of guarantee with ledgers.

At the end of the day when you look at the overall architecture of a system built *around* any of these in at least the IT shops I have seen, there is little to distinguish them. They all implement some level of scalability, performance, and reliability.

So I don't think most potential development communities would put them into completely different spaces nor should they not consider XMPP a serious competitor to AMQP except in the most extreme situations.

Patrick Logan said...

Em, Tibco rv is UDP *with* "routing daemons" across subnets. My bad, so to speak.

dbt said...

If you're looking for something in the low cost, low latency, "best effort" space, I'd strongly recommend 29west's LBM. Used it at my last job---a real pleasure.

Unknown said...

Depending on your requirements I believe that Amazon SQS is very fine choice and has the interesting aspect of being HTTP based, meaning probably easier for developers to pick up and integrate within existing architecture.

Again it's more a point of using the right tool for the job. SQS, AMQP, XMPP have all their strengths and domain where they shine.

Anonymous said...

"At the end of the day when you look at the overall architecture of a system built *around* any of [RV or EMS] in at least the IT shops I have seen, there is little to distinguish them. They all implement some level of scalability, performance, and reliability."

What does that mean? All systems have some level of scalabilty, performance and reliability, even if that level is very low.

If IT shops are using RV and EMS in the same way, then they are incompetent. The two systems are aimed at clearly different usage scenarios, and therefore have different engineering trade-offs.

Patrick Logan said...

"If IT shops are using RV and EMS in the same way, then they are incompetent. The two systems are aimed at clearly different usage scenarios, and therefore have different engineering trade-offs."

Most IT situations I've seen boil down to ease of use. There are a few that I've seen that have critical reliability or timing requirements. Those I've seen in assembly and fabrication in factories.

In all the other IT departments I've worked with (e.g. finance, sales, marketing, distribution) they've been sold rv and jms as solutions to the same problem. And so that's what's behind what I wrote, "I don't think most potential development communities would put them into completely different spaces".

Only in very specific cases does performance engineering become a significant part of the decision. That's just what I've seen. Some people spend their entire careers working with trading floors or demanding factory floors. But that's rare relative to the total set of customers these products are marketed to.

Anonymous said...

For some reason, various middleware pundits always forget about the CORBA Notification and Event service when criticizing CORBA as strictly RPC and brittle.

I've used these services with great success over the years in messaging systems that I've built.

The COS Notification service is very close to JMS, which of course is used in many of the popular messaging systems.

You can even sling XML messages around as a string if you'd like to get the desired "flexibility"; you don't need to use type-safe binary messages.

I'd love to see the software industry stop hyping various flavor-of-the-month technologies and start arguing the merits of each one based on the real facts and data. (performance benchmarks, APIs, realistic use cases, etc.)

Patrick Logan said...

"I'd love to see the software industry stop hyping various flavor-of-the-month technologies and start arguing the merits of each one based on the real facts and data. (performance benchmarks, APIs, realistic use cases, etc.)"

Yeah, well. I am not unsympathetic to the motivation. But that's unlikely to happen, and to the extent it does, unlikely to work.

Something we can hope for, pragmatically, is to get more such comparisons performed by individual teams, or small collaborating groups, under narrow, limited conditions.

Blog Archive

About Me

Portland, Oregon, United States
I'm usually writing from my favorite location on the planet, the pacific northwest of the u.s. I write for myself only and unless otherwise specified my posts here should not be taken as representing an official position of my employer. Contact me at my gee mail account, username patrickdlogan.