REST is the architecture of the web and the web has seen unprecedented growth and evolution since its inception over 30 years ago. A web of static documents gave rise to the read/write web–the so-called web 2.0. New media types and formats have evolved, protocols have become more powerful and more secure. In short, the web has only grown bigger in scale, more powerful in terms of capabilities, and continues to evolve. The human web is a marvel of software engineering and its longevity is a testament to the vision its founders and architects. Unlike much of the software I use today, I have never opened a web browser to a message that the back-end of the web has undergone a breaking change and I would need to download new software before continuing.

At some point in the first decade of the 21st century the web crossed an inflection point; machine-to-machine API calls eclipsed human traffic on the web. Many of these APIs have been labeled REST APIs despite many–perhaps even most–of these APIs fail to exhibit the qualities elicited by truly following the REST architectural style. Consequently, the machine web consists of a lot of brittle integrations that require countless person-hours to craft and maintain. To be clear, I’m not here to get on my soapbox about how all these sinners are “doing REST wrong”; I really don’t care. Not every API needs to–or should–be a REST API. The REST architectural style is a very specific tool to solve specific problems.

“Some architectural styles are often portrayed as “silver bullet” solutions for all forms of software. However, a good designer should select a style that matches the needs of a particular problem being solved.” -Dr Roy Fielding Architectural Styles and the Design of Network Based Architectures

Fundamentally I want to talk about some of the overlooked aspects of this architectural style and how they can be implemented to eliminate problems with versioning, evolution, and flexibility. Today we’re focusing on content negotiation.

REST and Abstraction

The purpose of abstraction is not to be vague, but to create a new semantic level in which one can be absolutely precise.

—Edsger Dijkstra

The REST Architectural Style makes the deliberate decision to decouple concepts like identity, representation, semantics, behavior, and implementation. Let’s look at these concepts in a more detail.

When the proper separation and abstraction of these concepts, we get a very flexible, evolvable, and agile API. Many, if not most, “RESTful” APIs provide very leaky abstractions. Here is a particularly egregious example:

http://example.com/api/v1/json/getPersonById.php?id=1234

To get the architectural benefits of REST, the URI should function solely as an identifier for a resource. In this case, the URI identifies a specific representation of the Person resource; specifically the V1 syntax, serialized in JSON. It also leaks implementation details (in this case, php). The implementation of this particular API example does an awful lot of work to reinvent what REST and underlying protocols give us for free while losing many of the architectural benefits the REST style would otherwise provide. Although this approach works reasonably well in very narrow, tightly-coupled, centrally controlled, client-server applications, increasingly we are seeing a need to get much more out of our APIs as organizations are discovering new requirements to connect information across enterprise application silos and increasingly adopt platform thinking. One such example….

Clearly this API contains the concept of a Person resource. In an underlying data storage mechanism, a particular person is assigned some kind of surrogate key to identify a specific instance of Person. From both a resource and business perspective, a Person with id=1234 is the same person regardless of whether it is served up with the V1 attributes or the V2 attributes. It is the same person whether the data is serialized in xml or json. It is the same person, regardless of whether the serialization of the person is served up using a php script or some other server-side technology. By using different URIs to represent variations of the exact same ‘thing’. This violates many of the core constraints of REST and scores zero on the Richardson Maturity Model (RMM). Almost any change is a breaking change to a client and if we want to begin to get the increasingly-important aspect of compossibility, we have to do even more work to try to resolve this mess of identifiers that will only get worse as the back-end of this system evolves.

Composability is to software as compounding interest is to finance

As sir Tim Berners-Lee said in his essay, Cool URIs don’t change. We can stabilize this URI by eliminating the variable elements from the URI and focus solely on identifying the given resource. If we adopt the Hierarchical URI Pattern, the Patterned URI and the Literal Keys Pattern we can construct a new, stable identifier for this resource.

http://example.com/person/id/1234

This now globally and unambiguously identifies the same, specific person resource. It won’t change if we want to add/remove/rename fields in a given version of a serialization, it won’t change if we want to change the backend implementation, and it won’t change regardless of serialization. This is the foundation for a much more useful abstraction. Many legacy backends can support this type of URI scheme using URI rewriting rules. We now have, at least, a level 1 API (according to the RMM), but likely a level 2 API (assuming you’re using GET, PUT, PATCH, DELETE, etc. correctly).

Adding Versioning and Serializations the RESTful Way

While we have a well-thought-out and stable URI to identify our resources, the original example clearly communicates a need to support some kind of versioning and multiple serializations. This is where content-negotiation comes in. Let’s look at serializations first.

Original Example

GET /api/v1/json/getPersonById.php?id=1234
Accept: */*

200 OK
{
    id: 1234,
    name: "Michael Carducci",
    ...
}

In this case, the client is accepting any response mimetype, and the server is hardcoded to respond based on part of the URI. The developer has to know what serializations are available at design-time, how to construct this URI, and hardcode this. If new serializations become available, the developer must learn this from out-of-band sources like documentation and code new capabilities into the client.

RESTful Example

GET /person/id/1234
Accept: application/json

200 OK
{
    id: 1234,
    name: "Michael Carducci",
    ...
}

Changing that Accept: header to application/xml would cause the server to either respond with an XML serialization, or a HTTP 406 error (Not Acceptable), or a simply return the default mimetype.

What About Versioning?

While changing serializations is useful, we’re left with the pesky need to evolve the syntax within representations. Changing business requirements often result in particular attributes being added, removed, or renamed. Most client implementations follow syntactic integration rather than semantic integration and these mappings can be quite brittle. There is a need to request specific versions of a given serialization. Content-negotiation helps here, as well.

The http protocol and supporting standards have quite a few capabilities to get creative with mimetypes. You may define vendor specific mimetypes or create variations using a registered mimetype suffix. For example, specifically requesting the V1 attributes might look like this:

GET /person/id/1234
Accept: application/v1+json

200 OK
{
    id: 1234,
    name: "Michael Carducci",
    ...
}

Omitting the v1+ would result in default behavior defined by the server (typically the current/latest serialization).

Client-Specific Optimizations?

One of the arguments against REST I hear is the problem of network efficiency. There are either “too many requests” necessary to hydrate a template in a client-side app, or the concern of over-fetching data for a narrower view (e.g. a mobile client). The former can be solved with proper up-front resource design to create resources (and composite resources) with the appropriate granularity; the latter can be solved with content negotiation.

An increasingly popular solution is the Backends for Frontends (BFF) pattern which requires a custom API layer for each front-end (mobile, browser, desktop, etc.) with optimized/right-sized/expanded payloads. REST allows for the same flexibility without the overhead of creating and maintaining multiple, substantially similar APIs while dramatically increasing future flexibility and agility.

This is where I would recommend taking advantage of vendor specific mimetypes to create the same client-optimized representations.

Beyond Media Types

There is a lot of power in being able to provide multiple mimetypes in your API or HTTP application, but there are actually four dimensions of content negation:

• Media Type - Request the best representation of a resources using the Accept header.
• Language - Request localized representations based on the Accept-Language header.
• Encoding - Request a content encoding scheme (usually a compression algorithm) that the client can understand using the Accept-Encoding header.
• Character Set - Request a specific character set using the Accept-Charset header (although wide support for UTF-8 makes this less common and some browsers have abandoned it. See this video for some fascinating history.)

Conclusion

This is just one area where the REST architectural style along with the supporting protocols and enabling standards can yield a truly flexible and agile API strategy. The success, growth, and evolution of the World-Wide Web (which is both the inspiration of REST and a paragon implementation) has shown how this architectural style supports mind-boggling growth, evolvability, and agility. The exponential growth of data and the increasingly competitive business climates require us to think about agility not just in terms of short-term developer productivity, but also investment in long-term acceleration and flexible, agile architectures. These investments will often pay handsome dividends in the future.