Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Where is the Thrift format specification?

Tags:

Although Thrift format is relatively widely used (and project still alive), I haven't been able to find specification for the underlying data format. Or formats, given that there are apparently at least two formats; standard one, and compact format.

Given its wide usage I am hoping it is just my poor google-fu, and a specification or specifications exist (outside of source code). If so, please share!

EDIT: Java source code is not too bad for figuring out format; see TBinaryProtocol.java and TCompactBinaryProtocol.java; which contain primitives, called by generated classes (which extend TBase)

EDIT2: While it does not contain format spec, "the missing guide" is good reading.

like image 959
StaxMan Avatar asked Feb 15 '11 21:02

StaxMan


People also ask

What is Thrift format?

Thrift is an interface definition language and binary communication protocol used for defining and creating services for numerous programming languages. It was developed at Facebook for "scalable cross-language services development" and as of 2020 is an open source project in the Apache Software Foundation.

Does Facebook still use Thrift?

Since then, usage of Thrift at Facebook has continued to grow. Today, it powers more than 100 services used in production, most of which are written in C++, Java, PHP, or Python.

What is the IDL for Apache Thrift?

0. The Thrift interface definition language (IDL) allows for the definition of Thrift Types. A Thrift IDL file is processed by the Thrift code generator to produce code for the various target languages to support the defined structs and services in the IDL file.


2 Answers

I had the same problem and therefore decided to write a specification of the Thrift binary and compact protocol.

You can find it here: https://erikvanoosten.github.io/thrift-missing-specification/

like image 91
Erik van Oosten Avatar answered Oct 19 '22 06:10

Erik van Oosten


The format for a Thrift message depends on the serialization protocol chosen. You can serialize a Thrift object using one of many protocols, including the TBinaryProtocol, TCompactProtocol, TJSONProtocol, and other obscure options. The protocol is described abstractly in the original Thrift Whitepaper PDF in terms of the API, but is not currently defined anywhere in the Thrift apache project.

The TCompactProtocol protocol is slightly better defined on the thrift wiki but not in simple terms.

Until better documentation is available, you can also examine the Python library code alongside the Python struct module documentation as another option.

like image 28
Will Pierce Avatar answered Oct 19 '22 05:10

Will Pierce