Middleware Matters

It's that darned word "protocol" again! You can't compare HTTP to TCP, IIOP, or SOAP, because it's higher level than all of those; one level below your data. See;

http://www.markbaker.ca/2002/09/Blog/2004/10/29#2004-10-protocols

Hi Mark, I agree that the word "protocol" is overloaded, but worry not: I do understand the difference between an application protocol and a transport protocol.

I read your blog entry you pointed me to, and unfortunately it still doesn't answer my question. Maybe I wasn't clear, so let me phrase the question another way: since REST proponents often ask WS proponents why they want protocol independence, does that mean that REST proponents actually seek protocol *dependence*? And what does that even mean? Does that mean that my financial trading application, for example, should be littered with knowledge of the protocol over which it communicates? If so, doesn't that represent a poor separation of concerns?

Ah, ok, good questions Steve.

If I were building a distributed financial app, I wouldn't commonly litter it with knowledge of the HTTP *syntax*; existing libraries generally do a good job of hiding that, and for the most part, that's a good thing to hide. But I would definitely tightly couple the app to HTTP *semantics*. So, for example, I would have the application know it was doing the semantic equivalent of a GET by using some aspect of the API that made that apparent, e.g. the java.net UrlConnection stuff.

But no, I don't think that's a poor separation of concerns, since I don't think there's any concerns to separate.

Hi Mark, I agree. HTTP is an application protocol, so having an application use it makes total sense. From that perspective, it's no different than having an application be aware of the verbs in an IDL interface.

But I wonder whether this is the only reason that REST proponents wonder about WS proponents' desire for protocol independence. Does the protocol independence question always come back to disagreements surrounding application protocols vs. transport protocols, or is there more to talk about in that area?

Yes, exactly (re IDL). I'm so glad you agree Steve, though I admit to not understanding how that agreement relates to your position for protocol independence. Were you thinking about protocol syntax perhaps?

And yes, I do think it just boils down to application vs. transport. Most Web services proponents hear "protocol" and automatically think "transport", and therefore it follows naturally that you'd want your app to be independent of them.

Hi Mark, the reason I think protocol independence is important goes back to the separation of concerns argument I made in a previous comment.

Before I explain that, however, note that I've written extensively about protocol independence over the past few years (all my columns are on my home page at http://www.iona.com/hyplan/vinoski ) and also note that Artix, IONA's ESB product (see http://www.iona.com/products/artix/ ), supports a fairly high degree of protocol independence in production today for a variety of customers. It allows web services applications to use, for example, SOAP over HTTP or MQ, fixed format over MQ, and native calls to CORBA, TIBCO, and Tuxedo apps, each based completely on their native protocols and formats/syntaxes (i.e., no changes to existing services/servers required). You can even have multiples of these all in use by a single application, chosen via configuration, with no dependencies in your code on any of the particular mechanisms you choose to configure.

Now, back to the separation of concerns issue: there's no need IMO to tie the core of your business logic to any protocol, application or otherwise. It goes back to my business card analogy that I describe awhile ago in my blog:

http://www.iona.com/blogs/vinoski/archives/000120.html

Your business card lists multiple ways to reach you, and whether someone does so over mobile, by email, or by instant messenger, they're getting the same service: you. Similarly, isolating the fundamentals of your application away from the details of Tuxedo and TIBCO and MQ and CORBA and whatever else, and instead hiding all that behind an abstraction, makes for an application that can live across technology trends, which invariably come and go.

My earlier comment about HTTP and IDL fits perfectly into this picture, because what I'm saying is that you isolate those kinds of protocol details away from the core of your business logic. It all comes down to practicality. Yes, somewhere in your application, someone has to be aware of the HTTP application protocol or those IDL interface details. But that doesn't belong in your core logic -- rather, it belongs in a plug-in, away from the code running your business. And no, not everything is interchangeable from one protocol to another, but you'd be surprised at just how much impedance mismatch can be avoided in practice based on the fact that most people never use half of the features of these middleware systems, instead sticking to a relatively simple and common core. This is all exactly what Artix does. And it does it fast, and it does it now, in production today. (I don't mean to sound too commercial, but to already have it working today while many argue about it as if it's hypothetical is, well, kinda silly.)

Hi Steve. Ok, that's pretty good, we're getting close I think. One question to clarify ...

This abstraction you want your application to be written to; does it include operations? Mine does, which means, for example, that if my application wanted to submit a document for processing, that it could use HTTP and SMTP because both of them support operations for that purpose (POST and DATA respectively). *But*, if my application were performing, say, a request for data, then HTTP could be used, but SMTP couldn't, because HTTP has an operation that means that (GET), while SMTP doesn't.

I certainly agree that having a single application able to support multiple protocols is desirable. But the argument for "protocol independence" goes far beyond that, to the point where the abstraction has no operations and therefore application protocols are treated as transport protocols.

Are we in synch?

Yep, Mark, I think we're in synch, since I don't see how the abstraction layer can avoid having operations. It is, after all, abstracting inter-application communications. Even with the business card analogy, for example, there are messages flowing to and from you, whether they're getting there by phone, by voicemail, by IM, etc.

Awesome.

FWIW, the operation-free abstraction is what I've observed most Web services proponents to be after when they speak of "protocol independence". In that view of the world, the WSDL defines the abstraction, not the application protocol.

Oops, I should clarify that the first abstraction I speak of there - the operation-free one - is the abstraction presented of the underlying protocol. The abstraction used by the application is the one presented by the WSDL.

Steve, if I understand Artix correctly, it uses WSDL as the common ground and abstracts away the transport and/or transfer protocol as a binding. I.e. I always describe my RPC-style service interface, generate stubs and skeletons and can, via configuration, decide which protocol to use for communication.

Correct?

Hi Stefan, thanks for the prompt to get back to this (we have a product release coming out so I was too busy this week to continue commenting). What you say is mostly correct. WSDL is an important aspect of Artix, in that applications are generally written against WSDL, but Artix is not limited to only RPC, given that it supports systems such as MQ Series, TIBCO, and JMS as well.

That's not really the level I was talking about in my thread with Mark, though. We've had significant success with interceptor-based architectures at IONA, which we started developing nearly a decade ago. In our Adaptive Runtime Technology (ART), the architecture underneath both Orbix 6 and Artix, messages flow through chains of interceptors to get from sender to receiver. Interceptors have a uniform interface, which I suppose is kind of like Jim's and Savas's ProcessThis. The flexibility this style affords is tremendous, and when you do it right, it's very fast as well. (Coincidentally, I read something earlier today about channels in Indigo, and they seem similar to our interceptors.) It is the interceptor abstraction that I was talking about in my previous comments.

Now, back to a question for Mark: do REST proponents believe that HTTP is the "Ultimate Protocol" that can completely serve all applications? If not, why pretend that it is? To solve this WS-REST rift, then, would the REST proponents' answer be to write a generic application interface in some abstraction language that can map directly to application protocols like HTTP, but also map to suitable intermediate abstractions that sit over transport protocols?

(BTW, I don't mean that as a trick question or antagonistic comment. Rather, I ask that question quite genuinely because, for the most part, what I personally see from REST proponents is constant criticism of WS proponents' approaches, but I personally never see REST proponents offering any practical solutions themselves. Granted, I may very well just be missing such proposals or solutions, because maybe I'm not tied into the right REST blogs or websites or mailing lists or wikis or whatever. That's precisely why I ask the question.)

I admit I know a little bit (but only a little bit) more about Artix than my question suggests, and if I understand it correctly, interceptors are like JaxRpcHandlers (in fact in the Java version, they are), i.e. they can be introduced in the RPC-style processing chain. And even if I use MQ to transport messages, my programming API doesn't change - it's still stubs and skeletons, like in the good old Orbix days.

That's why I'm wondering whether the two of you are really talking about the same thing. To me, Artix is the perfect embodiment of WSDL-centrism -- whether you use JMS, MQ, HTTP, or SOAP or IIOP, is only a matter of configuration. That means that it takes protocol independence to its extreme.

As an aside, I believe Jim's and Savas's idea of a programming model, translated to Artix, would mean to have interceptors only - i.e. stop "below" the binding, which doesn't seem possible with Artix right now.

Hi Steve.

"do REST proponents believe that HTTP is the "Ultimate Protocol" that can completely serve all applications? If not, why pretend that it is?"

I've never personally said it's good for all applications, because it clearly isn't. For example, I've often said that I wouldn't build a telnet-like application with it, or any that requires lots of small control messages like that, since the latency introduced by HTTP's per-message overhead would be entirely unsuitable. For document oriented applications though, I think HTTP's probably suitable for 97% of them or thereabouts.

"To solve this WS-REST rift, then, would the REST proponents' answer be to write a generic application interface in some abstraction language that can map directly to application protocols like HTTP, but also map to suitable intermediate abstractions that sit over transport protocols?"

Regarding the first part of that comment, there's already a number of multi-protocol supporting software libraries that do that, e.g. libwww. With them, even if you want to do an FTP RETR, you'd do it by invoking the logical equivalent of "GET" on an HTTP-like API bound to a ftp URI (usually with the URI part of the API). But the interface the developer sees is always HTTP semantics. As for the second part of that comment, I don't understand what you mean. Perhaps a concrete example of how you think a transport protocol would fit into that picture?

I understand the motivation for such questions, since you're right, I do mostly criticize. But I've often pointed out that some of what WS proponents seek you already get For Free with the Web. For example, you get large amounts of reliability free just from the architecture that you don't get with Web services (e.g. with known safe and idempotent operations). I'm not saying the Web currently has all the things WS proponents need, so there's still *much* more work to be done. But IMO it'll be done as additional architectural constraints applied to the Web, not by tearing down well proven ones. Have you read Rohit's ARRESTED dissertation yet?

Hi Mark, what I meant by the second part of my comment was that if it's HTTP semantics you want at the app level, could you achieve that by writing a layer to implement those semantics over whatever the underlying transport protocol happens to be, just as HTTP itself is such a layer written over TCP? Would this be the REST form of transport protocol independence?

I think you would definitely be able to design your own RESTful application protocol, semantically equivalent (or at least similar) to HTTP and put it on top of different transport protocols (and abuse some transfer protocols as transport protocols on the way).

The question is: Why would you want to do that? Wouldn't you just have replaced a very well-known, popular application protocol (HTTP) with a proprietary one that nobody knows?

That, IMO, is the problem with this kind of protocol independence -- it's just an illusion: you always depend on some protocol. Best to admit it.

Another related question for Mark: as Rich Salz reminds us here:

http://webservices.xml.com/pub/a/ws/2004/11/17/salz.html

Internet services are often implemented as a single application servicing multiple ports, with different protocols distinguished by the different ports. Some of the protocols that these service applications deal with are application protocols. This implies that it's not uncommon for a single application to simultaneously handle multiple application protocols. From a REST POV, is building applications in this manner a good idea or not? Does having multiple application protocols handled by a single application break the model?

Oops, sorry for the delay.

I agree 100% with Stefan's response to your question to me.

"From a REST POV, is building applications in this manner a good idea or not?"

I don't think REST has much to say about that, but I've personally done it both from a client POV (see that libwww example above) and from a server POV using both an SMTP and HTTP Servlet to receive incoming documents to a single "app". Very clean and intuitive.

"Does having multiple application protocols handled by a single application break the model?"

Not AFAICT. I'm sure there could be some "gotchas" in implementing that support that might, but I don't think it necessarily - nor even commonly - breaks anything.