Table of Contents

• 1. Overview
• 2. REST and ETags
• 3. Client-Server communication with curl
• 4. ETag support in Spring
• 5. Testing ETags
• 6. ETags are BIG
• 7. Conclusion

1. Overview

This article will focus on ETags – the Spring support, integration testing of the RESTful API, and consumption scenarios with curl. This is the ninth of a series of articles about setting up a secure RESTful Web Service using Spring 3.1 and Spring Security 3.1 with Java based configuration.

The REST with Spring series:

• Part 1 – Bootstrapping a web application with Spring 3.1 and Java based Configuration
• Part 2 – Building a RESTful Web Service with Spring 3.1 and Java based Configuration
• Part 3 – Securing a RESTful Web Service with Spring Security 3.1
• Part 4 – RESTful Web Service Discoverability
• Part 5 – REST Service Discoverability with Spring
• Part 6 – Basic and Digest authentication for a RESTful Service with Spring Security 3.1
• Part 7 – REST Pagination in Spring
• Part 8 – Authentication against a RESTful Service with Spring Security

2. REST and ETags

From the official Spring documentation on ETag support:

An ETag (entity tag) is an HTTP response header returned by an HTTP/1.1 compliant web server used to determine change in content at a given URL.

ETags are used for two things – caching and conditional requests. The ETag value can be though as a hash computed out of the bytes of the Response body. Because a cryptographic hash function is likely used, even the smallest modification of the body will drastically change the output and thus the value of the ETag. This is only true for strong ETags – the protocol does provide a weak Etag as well.

Using an If-* header turns a standard GET request into a conditional GET. The two If-* headers that are using with ETags are “If-None-Match” and “If-Match” – each with it’s own semantics as discussed later in this article.

3. Client-Server communication with curl

A simple Client-Server communication involving ETags can be broken down into the steps:

- first, the Client makes a REST API call – the Response includes the ETag header to be stored for further use:

curl -H "Accept: application/json" -i http://localhost:8080/rest-sec/api/resources/1HTTP/1.1 200 OK
ETag: "f88dd058fe004909615a64f01be66a7"
Content-Type: application/json;charset=UTF-8
Content-Length: 52

- next request the Client makes to the RESTful API includes the If-None-Match request header with the ETag value from the previous step; if the Resource has not changed on the Server, the Response will contain no body and a status code of 304 – Not Modified:

curl -H "Accept: application/json" -H 'If-None-Match: "f88dd058fe004909615a64f01be66a7"'
-i http://localhost:8080/rest-sec/api/resources/1HTTP/1.1 304 Not Modified
ETag: "f88dd058fe004909615a64f01be66a7"

- now, before retrieving the Resource again, we will change it by performing an update:

curl --user admin@fake.com:adminpass -H "Content-Type: application/json" -i
-X PUT --data '{ "id":1, "name":"newRoleName2", "description":"theNewDescription" }'
http://localhost:8080/rest-sec/api/resources/1HTTP/1.1 200 OK
ETag: "d41d8cd98f00b204e9800998ecf8427e"
<strong>Content-Length: 0</strong>

- finally, we send out the the last request to retrieve the Privilege again; keep in mind that it has been updated since the last time it was retrieved, so the previous ETag value should no longer work – the response will contain the new data and a new ETag which, again, can be stored for further use:

curl -H "Accept: application/json" -H 'If-None-Match: "f88dd058fe004909615a64f01be66a7"' -i
http://localhost:8080/rest-sec/api/resources/1HTTP/1.1 200 OK
ETag: "03cb37ca667706c68c0aad4cb04c3a211"
Content-Type: application/json;charset=UTF-8
Content-Length: 56

And there you have it – ETags in the wild and saving bandwidth.

4. ETag support in Spring

On to the Spring support – to use ETag in Spring is extremely easy to set up and completely transparent for the application. The support is enabled by adding a simple Filter in the web.xml:

<filter>
<filter-name>etagFilter</filter-name>
<filter-class>org.springframework.web.filter.ShallowEtagHeaderFilter</filter-class>
</filter>
<filter-mapping>
<filter-name>etagFilter</filter-name>
<url-pattern>/api/*</url-pattern>
</filter-mapping>

The filter is mapped on the same URI pattern as the RESTful API itself. The filter itself is the standard implementation of ETag functionality since Spring 3.0.

The implementation is a shallow one – the ETag is calculated based on the response, which will save bandwidth but not server performance. So, a request that will benefit from the ETag support will still be processed as a standard request, consume any resource that it would normally consume (database connections, etc) and only before having it’s response returned back to the client will the ETag support kick in.

At that point the ETag will be calculated out of the Response body and set on the Resource itself; also, if the If-None-Match header was set on the Request, it will be handled as well.

A deeper implementation of the ETag mechanism could potentially provide much greater benefits – such as serving some requests from the cache and not having to perform the computation at all – but the implementation would most definitly not be as simple, nor as pluggable as the shallow approach described here.

5. Testing ETags

Let’s start simple – we need to verify that the response of a simple request retrieving a single Resource will actually return the “ETag” header:

@Test
public void givenResourceExists_whenRetrievingResource_thenEtagIsAlsoReturned() {
// Given
Resource existingResource = getApi().create(new Resource());
String uriOfResource = baseUri + "/" + existingResource.getId();
// When
Response findOneResponse = RestAssured.given().
header("Accept", "application/json").get(uriOfResource);
// Then
assertNotNull(findOneResponse.getHeader(HttpHeaders.ETAG));
}

Next, we verify the happy path of the ETag behaviour – if the Request to retrieve the Resource from the server uses the correct ETag value, then the Resource is no longer returned.

@Test
public void givenResourceWasRetrieved_whenRetrievingAgainWithEtag_thenNotModifiedReturned() {
// Given
T existingResource = getApi().create(createNewEntity());
String uriOfResource = baseUri + "/" + existingResource.getId();
Response findOneResponse = RestAssured.given().
header("Accept", "application/json").get(uriOfResource);
String etagValue = findOneResponse.getHeader(HttpHeaders.ETAG);
// When
Response secondFindOneResponse= RestAssured.given().
header("Accept", "application/json").headers("If-None-Match", etagValue)
.get(uriOfResource);
// Then
assertTrue(secondFindOneResponse.getStatusCode() == 304);
}

Step by step:

• a Resource is first created and then retrieved – the ETag value is stored for further use
• a new retrieve request is sent, this time with the “If-None-Match” header specifying the ETag value previously stored
• on this second request, the server simply returns a 304 Not Modified, since the Resource itself has indeed not beeing modified between the two retrieval operations

Finally, we verify the case where the Resource is changed between the first and the second retrieval requests:

@Test
public void givenResourceWasRetrieved_whenRetrievingAgainWithEtag_thenNotModifiedReturned() {
// Given
T existingResource = getApi().create(createNewEntity());
String uriOfResource = baseUri + "/" + existingResource.getId();
Response findOneResponse = RestAssured.given().
header("Accept", "application/json").get(uriOfResource);
String etagValue = findOneResponse.getHeader(HttpHeaders.ETAG);
existingResource.setName(randomAlphabetic(6))
getApi().update(existingResource.setName(randomString));
// When
Response secondFindOneResponse= RestAssured.given().
header("Accept", "application/json").headers("If-None-Match", etagValue)
.get(uriOfResource);
// Then
assertTrue(secondFindOneResponse.getStatusCode() == 200);
}

Step by step:

• a Resource is first created and then retrieved – the ETag value is stored for further use
• the same Resource is then updated
• a new retrieve request is sent, this time with the “If-None-Match” header specifying the ETag value previously stored
• on this second request, the server will returns a 200 OK along with the full Resource, since the ETag value is no longer correct, as the Resource has been updated in the meantime

Next, we test the behaviour for “If-Match” – the ShallowEtagHeaderFilter does not have out of the box support for the If-Match HTTP header (being tracked on this JIRA issue), so the following test should fail:

@Test
public void givenResourceExists_whenRetrievedWithIfMatchIncorrectEtag_then412IsReceived() {
// Given
T existingResource = getApi().create(createNewEntity());
// When
String uriOfResource = baseUri + "/" + existingResource.getId();
Response findOneResponse = RestAssured.given().header("Accept", "application/json").
headers("If-Match", randomAlphabetic(8)).get(uriOfResource);
// Then
assertTrue(findOneResponse.getStatusCode() == 412);
}

Step by step:

• a Resource is first created
• the Resource is then retrieved with the “If-Match” header specifying an incorrect ETag value – this is a conditional GET request
• the server should return a 412 Precondition Failed

6. ETags are BIG

We have only used ETags for read operations – a RFC exists trying to clarify how implementations should deal with ETags on write operations – this is not standard, but is an interesting read.

There are of course other possible uses of the ETag mechanism, such an for an Optimistic Locking Mechanism using Spring 3.1 as well as dealing with the related “Lost Update Problem”.

There are also several known potential pitfalls and caveats to be aware of when using ETags.

7. Conclusion

This article only scratched the surface with what’s possible with Spring and ETags. For a full implementation of an ETag enabled RESTful service, along with integration tests verifying the ETag behaviour, check out the github project.

If you read this far, you should follow me on twitter here.