Table of Contents
1. Overview
This article will focus on ETags – the Spring support, integration testing of the RESTful API, and consumption scenarios with curl. This is the ninth of a series of articles about setting up a secure RESTful Web Service using Spring 3.1 and Spring Security 3.1 with Java based configuration.
The REST with Spring series:
2. REST and ETags
From the official Spring documentation on ETag support:
An ETag [16] (entity tag) is an HTTP response header returned by an HTTP/1.1 compliant web server used to determine change in content at a given URL.
ETags are used for two things – caching and conditional requests. The ETag value can be though as a hash computed out of the bytes of the Response body. Because a cryptographic hash function is likely used, even the smallest modification of the body will drastically change the output and thus the value of the ETag. This is only true for strong ETags – the protocol does provide a weak Etag [17] as well.
Using an If-* header turns a standard GET request into a conditional GET. The two If-* headers that are using with ETags are “If-None-Match [18]” and “If-Match [19]” – each with it’s own semantics as discussed later in this article.
3. Client-Server communication with curl
A simple Client-Server communication involving ETags can be broken down into the steps:
- first, the Client makes a REST API call – the Response includes the ETag header to be stored for further use:
curl -H "Accept: application/json" -i http://localhost:8080/rest-sec/api/resources/1HTTP/1.1 200 OK
ETag: "f88dd058fe004909615a64f01be66a7"
Content-Type: application/json;charset=UTF-8
Content-Length: 52
- next request the Client makes to the RESTful API includes the If-None-Match request header with the ETag value from the previous step; if the Resource has not changed on the Server, the Response will contain no body and a status code of 304 – Not Modified:
curl -H "Accept: application/json" -H 'If-None-Match: "f88dd058fe004909615a64f01be66a7"'
-i http://localhost:8080/rest-sec/api/resources/1HTTP/1.1 304 Not Modified
ETag: "f88dd058fe004909615a64f01be66a7"
- now, before retrieving the Resource again, we will change it by performing an update:
curl --user admin@fake.com:adminpass -H "Content-Type: application/json" -i
-X PUT --data '{ "id":1, "name":"newRoleName2", "description":"theNewDescription" }'
http://localhost:8080/rest-sec/api/resources/1HTTP/1.1 200 OK
ETag: "d41d8cd98f00b204e9800998ecf8427e"
<strong>Content-Length: 0</strong>
- finally, we send out the the last request to retrieve the Privilege again; keep in mind that it has been updated since the last time it was retrieved, so the previous ETag value should no longer work – the response will contain the new data and a new ETag which, again, can be stored for further use:
curl -H "Accept: application/json" -H 'If-None-Match: "f88dd058fe004909615a64f01be66a7"' -i
http://localhost:8080/rest-sec/api/resources/1HTTP/1.1 200 OK
ETag: "03cb37ca667706c68c0aad4cb04c3a211"
Content-Type: application/json;charset=UTF-8
Content-Length: 56
And there you have it – ETags in the wild and saving bandwidth.
4. ETag support in Spring
On to the Spring support – to use ETag in Spring is extremely easy to set up and completely transparent for the application. The support is enabled by adding a simple Filter in the web.xml:
<filter>
<filter-name>etagFilter</filter-name>
<filter-class>org.springframework.web.filter.ShallowEtagHeaderFilter</filter-class>
</filter>
<filter-mapping>
<filter-name>etagFilter</filter-name>
<url-pattern>/api/*</url-pattern>
</filter-mapping>
The filter is mapped on the same URI pattern as the RESTful API itself. The filter itself is the standard implementation of ETag functionality since Spring 3.0.
The implementation is a shallow one – the ETag is calculated based on the response, which will save bandwidth but not server performance. So, a request that will benefit from the ETag support will still be processed as a standard request, consume any resource that it would normally consume (database connections, etc) and only before having it’s response returned back to the client will the ETag support kick in.
At that point the ETag will be calculated out of the Response body and set on the Resource itself; also, if the If-None-Match header was set on the Request, it will be handled as well.
A deeper implementation of the ETag mechanism could potentially provide much greater benefits – such as serving some requests from the cache and not having to perform the computation at all – but the implementation would most definitly not be as simple, nor as pluggable as the shallow approach described here.
5. Testing ETags
Let’s start simple – we need to verify that the response of a simple request retrieving a single Resource will actually return the “ETag” header:
@Test
public void givenResourceExists_whenRetrievingResource_thenEtagIsAlsoReturned() {
// Given
Resource existingResource = getApi().create(new Resource());
String uriOfResource = baseUri + "/" + existingResource.getId();
// When
Response findOneResponse = RestAssured.given().
header("Accept", "application/json").get(uriOfResource);
// Then
assertNotNull(findOneResponse.getHeader(HttpHeaders.ETAG));
}
Next, we verify the happy path of the ETag behaviour – if the Request to retrieve the Resource from the server uses the correct ETag value, then the Resource is no longer returned.
@Test
public void givenResourceWasRetrieved_whenRetrievingAgainWithEtag_thenNotModifiedReturned() {
// Given
T existingResource = getApi().create(createNewEntity());
String uriOfResource = baseUri + "/" + existingResource.getId();
Response findOneResponse = RestAssured.given().
header("Accept", "application/json").get(uriOfResource);
String etagValue = findOneResponse.getHeader(HttpHeaders.ETAG);
// When
Response secondFindOneResponse= RestAssured.given().
header("Accept", "application/json").headers("If-None-Match", etagValue)
.get(uriOfResource);
// Then
assertTrue(secondFindOneResponse.getStatusCode() == 304);
}
Step by step:
Finally, we verify the case where the Resource is changed between the first and the second retrieval requests:
@Test
public void givenResourceWasRetrieved_whenRetrievingAgainWithEtag_thenNotModifiedReturned() {
// Given
T existingResource = getApi().create(createNewEntity());
String uriOfResource = baseUri + "/" + existingResource.getId();
Response findOneResponse = RestAssured.given().
header("Accept", "application/json").get(uriOfResource);
String etagValue = findOneResponse.getHeader(HttpHeaders.ETAG);
existingResource.setName(randomAlphabetic(6))
getApi().update(existingResource.setName(randomString));
// When
Response secondFindOneResponse= RestAssured.given().
header("Accept", "application/json").headers("If-None-Match", etagValue)
.get(uriOfResource);
// Then
assertTrue(secondFindOneResponse.getStatusCode() == 200);
}
Step by step:
Next, we test the behaviour for “If-Match” – the ShallowEtagHeaderFilter does not have out of the box support for the If-Match HTTP header (being tracked on this JIRA issue [20]), so the following test should fail:
@Test
public void givenResourceExists_whenRetrievedWithIfMatchIncorrectEtag_then412IsReceived() {
// Given
T existingResource = getApi().create(createNewEntity());
// When
String uriOfResource = baseUri + "/" + existingResource.getId();
Response findOneResponse = RestAssured.given().header("Accept", "application/json").
headers("If-Match", randomAlphabetic(8)).get(uriOfResource);
// Then
assertTrue(findOneResponse.getStatusCode() == 412);
}
Step by step:
6. ETags are BIG
We have only used ETags for read operations – a RFC exists [21] trying to clarify how implementations should deal with ETags on write operations – this is not standard, but is an interesting read.
There are of course other possible uses of the ETag mechanism, such an for an Optimistic Locking Mechanism using Spring 3.1 [22] as well as dealing with the related “Lost Update Problem” [23].
There are also several known potential pitfalls and caveats [24] to be aware of when using ETags.
7. Conclusion
This article only scratched the surface with what’s possible with Spring and ETags. For a full implementation of an ETag enabled RESTful service, along with integration tests verifying the ETag behaviour, check out the github project [25].
If you read this far, you should follow me on twitter here [26].
Links:
[1] http://www.baeldung.com#overview
[2] http://www.baeldung.com#rest
[3] http://www.baeldung.com#curl
[4] http://www.baeldung.com#spring
[5] http://www.baeldung.com#testing
[6] http://www.baeldung.com#other
[7] http://www.baeldung.com#conclusion
[8] http://www.baeldung.com/2011/10/20/bootstraping-a-web-application-with-spring-3-1-and-java-based-configuration-part-1/
[9] http://www.baeldung.com/2011/10/25/building-a-restful-web-service-with-spring-3-1-and-java-based-configuration-part-2/
[10] http://www.baeldung.com/2011/10/31/securing-a-restful-web-service-with-spring-security-3-1-part-3/
[11] http://www.baeldung.com/2011/11/06/restful-web-service-discoverability-part-4/
[12] http://www.baeldung.com/2011/11/13/rest-service-discoverability-with-spring-part-5/
[13] http://www.baeldung.com/2011/11/20/basic-and-digest-authentication-for-a-restful-service-with-spring-security-3-1/
[14] http://www.baeldung.com/2012/01/18/rest-pagination-in-spring/
[15] http://www.baeldung.com/2012/12/20/authentication-against-a-restful-service/
[16] http://en.wikipedia.org/wiki/HTTP_ETag
[17] http://tools.ietf.org/html/rfc2616#section-13.3.3
[18] http://tools.ietf.org/html/rfc2616#section-14.26
[19] http://tools.ietf.org/html/rfc2616#section-14.24
[20] https://jira.springsource.org/browse/SPR-10164
[21] http://greenbytes.de/tech/webdav/draft-reschke-http-etag-on-write-latest.html#rfc.section.1.1
[22] http://blog.furiousbob.com/2011/12/06/hateoas-restful-services-using-spring-3-1/
[23] http://www.w3.org/1999/04/Editing/
[24] http://www.mnot.net/blog/2007/08/07/etags
[25] https://github.com/eugenp/REST
[26] http://twitter.com/baeldung