REST != HTTP

Roy
Fielding has weighed in on the recent "buzzwordiness" (hey, if Colbert can make
up "truthiness", then I can make up "buzzwordiness") of calling everything a "REST
API", a tactic that has become more en vogue of late as vendors discover
that the general programming population is finding the WSDL-based XML services stack
too complex to navigate successfully for all but the simplest of projects. Contrary
to what many RESTafarians may be hoping, Roy doesn't gather all these wayward children
to his breast and praise their anti-vendor/anti-corporate/anti-proprietary efforts,
but instead, blasts them pretty seriously for mangling his term:

I am getting frustrated by the number of people calling any HTTP-based interface a
REST API. Today’s example is the SocialSite
REST API. That is RPC. It screams RPC. There is so much coupling on display that
it should be given an X rating.

Ouch. "So much coupling on display that it should be given an X rating." I have to
remember that phrase--that's a keeper. And I'm shocked that Roy even knows what an
X rating is; he's such a mellow guy with such an innocent-looking face, I would've
bet money he'd never run into one before. (Yes, people, that's a joke.)

What needs to be done to make the REST architectural style clear on the notion that
hypertext is a constraint? In other words, if the engine of application state (and
hence the API) is not being driven by hypertext, then it cannot be RESTful and cannot
be a REST API. Period. Is there some broken manual somewhere that needs to be fixed?

Go Roy!

For those of you who've not read Roy's thesis, and are thinking that this
is some kind of betrayal or trick, let's first of all point out that at no point is
Roy saying that your nifty HTTP-based API is not useful or simple.
He's simply saying that it isn't RESTful. That's a key differentiation. REST
has a specific set of goals and constraints it was trying to meet, and as such prescribes
a particular kind of architectural style to fit within those constraints. (Yes, REST
is essentially an architectural pattern: a solution to a problem within a certain
context that yields certain consequences.)

Assuming you haven't tuned me out completely already, allow me to elucidate. In Chapter
5 of Roy's
thesis, Roy begins to build up the style that will ultimately be considered REST.
I'm not going to quote each and every step here--that's what the hyperlink above is
for--but simply call out certain parts. For example, in section 5.1.3, "Stateless",
he suggests that this architectural style should be stateless in nature, and explains
why; the emphasis/italics are mine:

We next add a constraint to the client-server interaction: communication must be stateless
in nature, as in the client-stateless-server (CSS) style of Section 3.4.3 (Figure
5-3), such that each request from client to server must contain all of the information
necessary to understand the request, and cannot take advantage of any stored
context on the server. Session state is therefore kept entirely on the client.

This constraint induces the properties of visibility, reliability, and scalability.
Visibility is improved because a monitoring system does not have to look beyond a
single request datum in order to determine the full nature of the request. Reliability
is improved because it eases the task of recovering from partial failures [133]. Scalability
is improved because not having to store state between requests allows the server component
to quickly free resources, and further simplifies implementation because the server
doesn't have to manage resource usage across requests.

Like most architectural choices, the stateless constraint reflects a design trade-off.
The disadvantage is that it may decrease network performance by increasing the repetitive
data (per-interaction overhead) sent in a series of requests, since that data cannot
be left on the server in a shared context. In addition, placing the application state
on the client-side reduces the server's control over consistent application behavior,
since the application becomes dependent on the correct implementation of semantics
across multiple client versions.

In the HTTP case, the state is contained entirely in the document itself, the hypertext.
This has a couple of implications for those of us building "distributed applications",
such as the very real consideration that there's a lot of state we don't
necessarily want to be sending back to the client, such as voluminous information
(the user's e-commerce shopping cart contents) or sensitive information (the user's
credentials or single-signon authentication/authorization token). This is a bitter
pill to swallow for the application development world, because much of the applications
we develop have some pretty hefty notions of server-based state management that we
want or need to preserve, either for legacy support reasons, for legitimate concerns
(network bandwidth or security), or just for ease-of-understanding. Fielding isn't
apologetic about it, though--look at the third paragraph above. "[T]he stateless constraint
reflects a design trade-off."

In other words, if you don't like it, fine, don't follow it, but understand that if
you're not leaving all the application state on the client, you're not doing REST.

By the way, note that technically, HTTP is not tied to HTML, since the document sent
back and forth could easily be a PDF document, too, particularly since PDF supports
hyperlinks to other PDF documents. Nowhere in the thesis do we see the idea that it has to
be HTML flying back and forth.

Roy's thesis continues on in the same vein; in section 5.1.4 he describes how "client-cache-stateless-server"
provides some additional reliability and performance, but only if the data in the
cache is consistent and not stale, which was fine for static documents, but not for
dynamic content such as image maps. Extensions were necessary in order to accomodate
the new ideas.

In section 5.1.5 ("Uniform Interface") we get to another stinging rebuke of REST as
a generalized distributed application scheme; again, the emphasis is mine:

The central feature that distinguishes the REST architectural style from other network-based
styles is its emphasis on a uniform interface between components (Figure 5-6). By
applying the software engineering principle of generality to the component interface,
the overall system architecture is simplified and the visibility of interactions is
improved. Implementations are decoupled from the services they provide, which encourages
independent evolvability. The trade-off, though, is that a uniform interface degrades
efficiency, since information is transferred in a standardized form rather than one
which is specific to an application's needs. The REST interface is designed to be
efficient for large-grain hypermedia data transfer, optimizing for the common case
of the Web, but resulting in an interface that is not optimal for other forms of architectural
interaction.

In order to obtain a uniform interface, multiple architectural constraints are needed
to guide the behavior of components. REST is defined by four interface constraints:
identification of resources; manipulation of resources through representations; self-descriptive
messages; and, hypermedia as the engine of application state. These constraints
will be discussed in Section 5.2.

In other words, in order to be doing something that Fielding considers RESTful, you
have to be using hypermedia (that is to say, hypertext documents of some form) as
the core of your application state. It might seem like this implies that you have
to be building a Web application in order to be considered building something RESTful,
so therefore all Web apps are RESTful by nature, but pay close attention to the wording:
hypermedia must be the core of your application state. The way most Web apps
are built today, HTML is clearly not the core of the state, but merely a way to render
it. This is the accidental consequence of treating Web applications and desktop client
applications as just pale reflections of one another.

The next section, 5.1.6 ("Layered System") again builds on the notion of stateless-server
architecture to provide additional flexibility and power:

In order to further improve behavior for Internet-scale requirements, we add layered
system constraints (Figure 5-7). As described in Section 3.4.2, the layered system
style allows an architecture to be composed of hierarchical layers by constraining
component behavior such that each component cannot "see" beyond the immediate layer
with which they are interacting. By restricting knowledge of the system to a single
layer, we place a bound on the overall system complexity and promote substrate independence.
Layers can be used to encapsulate legacy services and to protect new services from
legacy clients, simplifying components by moving infrequently used functionality to
a shared intermediary. Intermediaries can also be used to improve system scalability
by enabling load balancing of services across multiple networks and processors.

The primary disadvantage of layered systems is that they add overhead and latency
to the processing of data, reducing user-perceived performance [32]. For a network-based
system that supports cache constraints, this can be offset by the benefits of shared
caching at intermediaries. Placing shared caches at the boundaries of an organizational
domain can result in significant performance benefits [136]. Such layers also allow
security policies to be enforced on data crossing the organizational boundary, as
is required by firewalls [79].

The combination of layered system and uniform interface constraints induces architectural
properties similar to those of the uniform pipe-and-filter style (Section 3.2.2).
Although REST interaction is two-way, the large-grain data flows of hypermedia interaction
can each be processed like a data-flow network, with filter components selectively
applied to the data stream in order to transform the content as it passes [26]. Within
REST, intermediary components can actively transform the content of messages because
the messages are self-descriptive and their semantics are visible to intermediaries.

The potential of layered systems (itself not something that people building RESTful
approaches seem to think much about) is only realized if the entirety of the state
being transferred is self-descriptive and visible to the intermediaries--in other
words, intermediaries can only be helpful and/or non-performance-inhibitive if they
have free reign to make decisions based on the state they see being transferred. If
something isn't present in the state being transferred, usually because there is server-side
state being maintained, then they have to be concerned about silently changing the
semantics of what is happening in the interaction, and intermediaries--and layers
as a whole--become a liability. (Which is probably why so few systems seem to do it.)

And if the notion of visible, transported state is not yet made clear in his dissertation,
Fielding dissects the discussion even further in section 5.2.1, "Data Elements". It's
too long to reprint here in its entirety, and frankly, reading the whole thing is
necessary to see the point of hypermedia and its place in the whole system. (The same
could be said of the entire chapter, in fact.) But it's pretty clear, once you read
the dissertation, that hypermedia/hypertext is a core, critical piece to the whole
REST construction. Clients are expected, in a RESTful system, to have no preconceived
notions of structure or relationship between resources, and discover all of that through
the state of the hypertext documents that are sent back to them. In the HTML case,
that discovery occurs inside the human brain; in the SOA/services case, that discovery
is much harder to define and describe. RDF and Semantic Web ideas may be of some help
here, but JSON can't, and simple XML can't, unless the client has some preconceived
notion of what the XML structure looks like, which violates Fielding's rules:

A REST API should be entered with no prior knowledge beyond the initial URI (bookmark)
and set of standardized media types that are appropriate for the intended audience
(i.e., expected to be understood by any client that might use the API). From that
point on, all application state transitions must be driven by client selection of
server-provided choices that are present in the received representations or implied
by the user’s manipulation of those representations. The transitions may be determined
(or limited by) the client’s knowledge of media types and resource communication mechanisms,
both of which may be improved on-the-fly (e.g., code-on-demand). [Failure here implies
that out-of-band information is driving interaction instead of hypertext.]

An interesting "fuzzy gray area" here is whether or not the client's knowledge of
a variant or schematic structure of XML could be considered to be a "standardized
media type", but I'm willing to bet that Fielding will argue against it on the grounds
that your application's XML schema is not "standardized" (unless, of course, it is,
through a national/international/industry standardization effort).

But in case you'd missed it, let me summarize the past twenty or so paragraphs: hypermedia
is a core requirement to being RESTful. If you ain't slinging all of your application
state back and forth in hypertext, you ain't REST. Period. Fielding said it, he defined
it, and that settles it.

Before the hate mail comes a-flyin', let me reiterate one vitally important point: if
you're not doing REST, it doesn't mean that your API sucks. Fielding may have
his definition of what REST is, and the idealist in me wants to remain true to his
definitions of it (after all, if we can't agree on a common set of definitions, a
common lexicon, then we can't really make much progress as an industry), but...

... the pragmatist in me keeps saying, "so what"?

Look, at the end of the day, if your system wants to misuse HTTP, abuse HTML, and
carnally violate the principles of loose coupling and resource representation that
underlie REST, who cares? Do you get special bonus points from the Apache Foundation
if you use HTTP in the way Fielding intended? Will Microsoft and Oracle and Sun and
IBM offer you discounts on your next software purchases if you create a REST-faithful
system? Will the partisan politics in Washington, or the tribal conflicts in the Middle
East, or even the widely-misnamed "REST-vs-SOAP" debates come to an end if you only
figure out a way to make hypermedia the core engine of your application state?

Yeah, I didn't think so, either.

Point is, REST is just an architectural style. It is nothing more than another
entry alongside such things as client-server, n-tier, distributed objects,
service-oriented, and embedded systems. REST is just a tool for thinking about how
to build an application, and it's high time we kick it off the pedastal on which we've
placed it and let it come back down to earth with the rest of us mortals. HTTP is
useful, but not sufficient, so solve our problems. REST is as well.

And at the end of the day, when we put one tool from our tool belt "above all others",
we end up building some truly horrendous crap.

Enterprise consulting, mentoring or instruction. Java, C++, .NET or XML services.
1-day or multi-day workshops available. Contact
me for details.