What's the best practice to for Thrift file (api) versioning?

Question

I have an API written in thrift. Example:

service Api {
  void invoke()
}

It does something. I want to change the behavior to do something else but still keep the old behavior for clients that expect the old behavior.

What's the best practice to handle a new API version?

Answer 1

Soft versioning

Thrift supports soft versioning, so it is perfectly valid to do a version 2 of your service which looks like this:

service Api {
   void invoke(1: string optional_arg1, 2: i32 optional_arg2) throws (1: MyError e)
   i32 number_of_invokes()
}

Since the newly added arguments are technically optional, an arbitrary clients request may or may not contain them, or may contain only parts of them (e.g. specify arg1 but not arg2). The exception is a bit different, old clients will raise some kind of generic unexpected exception or similar.

It is even possible to remove an outdated function entirely, in this case old clients will get an exception whenever they try to call the (now non-existing) removed function.

All of the above is similarly is true with regard to adding member fields to structures, exceptions etc. Instead of removing declarations from the IDL file, it is recommended to comment out old removed member fields and functions to prevent people from re-using old field IDs, old function names or old enum values in later versions.

struct foobar {
  // API 1.0 fields
  1: i32 foo
  //2: i32 bar   - obsolete with API 2.0

  // API 2.0 fields
  3: i32 baz
}

required is forever

Where you need to be careful is the use of the keyword required. Once you publish an API with a struct containing an required member, you will need to carry this one until the structure as a whole is removed. Same is true with adding new required fields later on. Otherwise you risk breaking changes, because mixing old and new clients and servers will sooner or later produce a situation where one end absolutely expects a certain required member field, but the opposite end can't deliver, simply because it does not know anything about it.

This is not a problem with normal or optional fields, since Thrift is designed to skip over unknown fields (the type ID is contained in the wire data), and just ignore missing fields. In contrast, additional checks are applied for required fields to ensure they are present in the wire data.

Endpoints

Although soft versioning is a great tool, it comes at the cost of cumulating burdens due to the need to be compatible. Furthermore, in some cases your API will undergo breaking changes, intentionally without being backwards compatible. In that case, it is recommened to set the new service at a different endpoint.

Alternatively, the multiplex protocol introduced with Thrift 0.9.2 can be used to offer multiple services and/or service versions over the same endpoint (i.e. socket, http URI, ...)