Using Google Protocol Buffers with Spring MVC-based REST Services

Using Google Protocol Buffers with Spring MVC-based REST Services

This week I'm in São Paulo, Brazil presenting at QCon SP. I had an interesting discussion with someone who loves Spring's REST stack, but wondered if there was something more efficient than plain-ol' JSON. Indeed, there is! I often get asked about Spring's support for high-speed binary based encoding of messages. Spring's long supported RPC encoding with the likes of Hessian, Burlap, etc., and Spring Framework 4.1 introduced support for Google Protocol Buffers which can be used with REST services as well.

From the Google Protocol Buffer website:

Protocol buffers are Google's language-neutral, platform-neutral, extensible mechanism for serializing structured data – think XML, but smaller, faster, and simpler. You define how you want your data to be structured once, then you can use special generated source code to easily write and read your structured data to and from a variety of data streams and using a variety of languages…

Google uses Protocol Buffers extensively in their own, internal, service-centric architecture.

A .proto document describes the types (_messages_) to be encoded and contains a definition language that should be familiar to anyone who's used C structs. In the document, you define types, fields in those types, and their ordering (memory offsets!) in the type relative to each other.

The .proto files aren't implementations - they're declarative descriptions of messages that may be conveyed over the wire. They can prescribe and validate constraints - the type of a given field, or the cardinatlity of that field - on the messages that are encoded and decoded. You must use the Protobuf compiler to generate the appropriate client for your language of choice.

You can use Google Protocol Buffers anyway you like, but in this post we'll look at using it as a way to encode REST service payloads. This approach is powerful: you can use content-negotiation to serve high speed Protocol Buffer payloads to the clients (in any number of languages) that accept it, and something more conventional like JSON for those that don't.

Protocol Buffer messages offer a number of improvements over typical JSON-encoded messages, particularly in a polyglot system where microservices are implemented in various technologies but need to be able to reason about communication between services in a consistant, long-term manner.

Protocol Buffers are several nice features that promote stable APIs:

• Protocol Buffers offer backward compatibility for free. Each field is numbered in a Protocol Buffer, so you don't have to change the behavior of the code going forward to maintain backward compatability with older clients. Clients that don't know about new fields won't bother trying to parse them.
• Protocol Buffers provide a natural place to specify validation using the required, optional, and repeated keywords. Each client enforces these constraints in their own way.
• Protocol Buffers are polyglot, and work with all manner of technologies. In the example code for this blog alone there is a Ruby, Python and Java client for the Java service demonstrated. It's just a matter of using one of the numerous supported compilers.

You might think that you could just use Java's inbuilt serialization mechanism in a homogeneous service environment but, as the Protocol Buffers team were quick to point out whent hey first introduced the technology, there are some problems even with that. Java language luminary Josh Bloch's epic tome, Effective Java, on page 213, provides further details.

Read full article from Using Google Protocol Buffers with Spring MVC-based REST Services