Fast Binary Encoding (FBE) format specification

Fast Binary Encoding (FBE) format describes domain model with as a set of enums, flags and structs. Structs fields are described with base or complex types, optional fields and collections.

Example of the domain model:

// Package declaration
package proto

// Order side declaration
enum OrderSide
{
    buy;
    sell;
}

// Order type declaration
enum OrderType
{
    market;
    limit;
    stop;
}

// Order declaration
struct Order
{
    [key] int32 uid;
    string symbol;
    OrderSide side;
    OrderType type;
    double price = 0.0;
    double volume = 0.0;
}

// Account balance declaration
struct Balance
{
    [key] string currency;
    double amount = 0.0;
}

// Account state declaration
flags State
{
    unknown = 0x00;
    invalid = 0x01;
    initialized = 0x02;
    calculated = 0x04;
    broken = 0x08;
    good = initialized | calculated;
    bad = unknown | invalid | broken;
}

// Account declaration
struct Account
{
    [key] int32 uid;
    string name;
    State state = State.initialized | State.bad;
    Balance wallet;
    Balance? asset;
    Order[] orders;
}

Contents

Base types

bool

bool

Properties:

Examples:

bool field1;
bool field2 = false;
bool field3 = true;

byte

byte

Properties:

Examples:

byte field1;
byte field2 = 0;
byte field3 = 128;
byte field4 = 0xFF;

char, wchar

char

Properties:

Examples:

char field1;
char field2 = 48;
char field3 = 0x41;
char field4 = '!';

wchar

Properties:

Examples:

wchar field1;
wchar field2 = 1025;
wchar field3 = 0x0410;
wchar field4 = '!';

int8, uint8, int16, uint16, int32, uint32, int64, uint64

int8

int8 properties:

Examples:

int8 field1;
int8 field2 = 0;
int8 field3 = -128;
int8 field4 = 127;

uint8 properties:

Examples:

uint8 field1;
uint8 field2 = 0;
uint8 field3 = 128;
uint8 field4 = 0xFF;

int16

int16 properties:

Examples:

int16 field1;
int16 field2 = 0;
int16 field3 = -32768;
int16 field4 = 32767;

uint16 properties:

Examples:

uint16 field1;
uint16 field2 = 0;
uint16 field3 = 32768;
uint16 field4 = 0xFFFF;

int32

int32 properties:

Examples:

int32 field1;
int32 field2 = 0;
int32 field3 = -2147483648;
int32 field4 = 2147483647;

uint32 properties:

Examples:

uint32 field1;
uint32 field2 = 0;
uint32 field3 = 2147483648;
uint32 field4 = 0xFFFFFFFF;

int64

int64 properties:

Examples:

int64 field1;
int64 field2 = 0;
int64 field3 = -9223372036854775808;
int64 field4 = 9223372036854775807;

uint64 properties:

Examples:

uint64 field1;
uint64 field2 = 0;
uint64 field3 = 9223372036854775808;
uint64 field4 = 0xFFFFFFFFFFFFFFFF;

float, double

float

float properties:

Examples:

float field1;
float field2 = 0.0;
float field3 = -123.456;
float field4 = 123.456e+12;

double

double properties:

Examples:

double field1;
double field2 = 0.0;
double field3 = -123.456;
double field4 = 123.456e+123;

Complex types

bytes

Represents a byte array that can be changed in user code. Works like a BLOB that can be changed in random index.

*Implementation of the bytes type depends on equivalent type in each programming language (e.g. std::vector in C++, byte[] in C# and Java).*

bytes

bytes properties:

Examples:

bytes field1;

Final model is more compact:

bytes-final

Final model bytes properties:

decimal

A decimal number is a floating-point value that consists of a sign, a numeric value where each digit in the value ranges from 0 to 9, and a scaling factor that indicates the position of a floating decimal point that separates the integral and fractional parts of the numeric value.

The binary representation of a Decimal value consists of a 1-bit sign, a 96-bit integer number, and a scaling factor used to divide the 96-bit integer and specify what portion of it is a decimal fraction.

The scaling factor is implicitly the number 10, raised to an exponent ranging from 0 to 28. Therefore, the binary representation of a decimal value the form, ((-29^6 to 29^6) / 10^(0 to 28)), where -(2^96-1) is equal to min value, and 2^96-1 is equal to max value.

decimal

decimal properties:

Examples:

decimal field1;
decimal field2 = 0.0;
decimal field3 = 123456.123456;

References:

string

Represents a string as an array of bytes in UTF8 encoding.

Implementation of the string type depends on equivalent type in each programming language (e.g. std::string in C++, String class in C# and Java).

string

string properties:

Examples:

string field1;
string field2 = "";
string field3 = "Initial string";

Final model is more compact:

string-final

Final model string properties:

timestamp

Represents a timestamp as 64-bit count of nanoseconds from the Unix epoch. Timestamp range is enough to store any timestamp value from 1970 to 2554 year in nanoseconds (2^64 nanoseconds is ~584.554531 years).

Implementation of the timestamp type depends on equivalent type in each programming language (e.g. uint64_t in C++, DateTime struct in C#, Instant class in Java).

timestamp

timestamp properties:

Examples:

timestamp field1;
timestamp field2 = epoch;
timestamp field3 = now;

uuid

A universally unique identifier (uuid) is a 128-bit number used to identify information in computer systems.

Supported uuid types:

Implementation of the uuid type depends on equivalent type in each programming language (e.g. custom uuid_t in C++, Guid in C#, UUID in Java).

uuid

In its canonical textual representation, the sixteen octets of a uuid are represented as 32 hexadecimal (base 16) digits, displayed in five groups separated by hyphens, in the form 8-4-4-4-12 for a total of 36 characters (32 alphanumeric characters and four hyphens). For example:

123e4567-e89b-12d3-a456-426655440000
xxxxxxxx-xxxx-Mxxx-Nxxx-xxxxxxxxxxxx

The four bits of digit M indicate the uuid version, and the one to three most significant bits of digit N indicate the uuid variant. In the example, M is 1 and N is a (10xx), meaning that the uuid is a variant 1, version 1 uuid; that is, a time-based DCE/RFC 4122 uuid.

The canonical 8-4-4-4-12 format string is based on the “record layout” for the 16 bytes of the uuid:

Field Size Comments
Time low 4 bytes Integer giving the low 32 bits of the time
Time mid 2 bytes Integer giving the middle 16 bits of the time
Time hi and version 2 bytes 4-bit “version” in the most significant bits, followed by the high 12 bits of the time
Clock sequence 2 bytes 1-3 bit “variant” in the most significant bits, followed by the 13-15 bit clock sequence
Node 6 bytes 48-bit Node Id

All fields are stored in big-endian format!

uuid properties:

Examples:

uuid field1;
uuid field2 = uuid0;
uuid field3 = uuid1;
uuid field4 = uuid4;
uuid field5 = "123e4567-e89b-12d3-a456-426655440000";

References:

Collections

array

Represents a fixed size array of the given type.

Implementation of the array type depends on equivalent type in each programming language (e.g. std::array<T, N> in C++, T[] in C# and Java).

array

array properties:

Examples:

byte[2] field1;
string?[4] field2;
MyEnum[8] field3;
MyStruct?[16] field4;

vector

Represents a dynamic size vector of the given type.

*Implementation of the vector type depends on equivalent type in each programming language (e.g. std::vector in C++, List in C#, ArrayList in Java).*

vector

vector properties:

Examples:

byte[] field1;
string?[] field2;
MyEnum[] field3;
MyStruct?[] field4;

Final model is more compact:

vector-final

Final model vector properties:

list

Represents a linked list of the given type.

*Implementation of the list type depends on equivalent type in each programming language (e.g. std::list in C++, LinkedList in C# and Java).*

list

list properties:

Examples:

byte() field1;
string?() field2;
MyEnum() field3;
MyStruct?() field4;

Final model is more compact:

list-final

Final model list properties:

map

Represents an associative map of the given type.

Implementation of the map type depends on equivalent type in each programming language (e.g. std::map<K, V> in C++, SortedDictionary<K, V> in C#, TreeMap<K, V> in Java).

map

map properties:

Examples:

byte<int32> field1;
string?<string> field2;
MyEnum<uuid> field3;
MyStruct?<MyKey> field4;

Final model is more compact:

map-final

Final model map properties:

hash

Represents an associative hash table of the given type.

Implementation of the hash type depends on equivalent type in each programming language (e.g. std::unordered_map<K, V> in C++, Dictionary<K, V> in C#, HashMap<K, V> in Java).

hash

hash properties:

Examples:

byte{int32} field1;
string?{string} field2;
MyEnum{uuid} field3;
MyStruct?{MyKey} field4;

Final model is more compact:

hash-final

Final model hash properties:

Optional type

Represents an optional wrapper of the given type.

*Implementation of the optional type depends on equivalent type in each programming language (e.g. std::optional in C++, Nullable struct or any reference type in C#, reference type in Java).*

optional

optional properties:

Examples:

byte? field1;
int32? field2 = 0;
string? field3 = "Initial string";
uuid? field4 = null;
MyEnum? field5 = null;
MyStruct? field6 = null;

Final model is more compact:

optional-final

Final model optional properties:

Enums

Represents an enumerable data type. It contains several identificators with incremental values or given integer constants.

enum

By default the enum identificator starts from 0 and increments by 1. It is possible to provide a custom valid enum value for any identificator or even some existing enum identificator.

Examples:

// Default enum declaration (enum type is int32)
enum MyEnum
{
    value0;          // value0 ==  0
    value1;          // value1 ==  1
    value2;          // value2 ==  2
    value3 = -2;     // value3 == -2
    value4;          // value4 == -1
    value5;          // value5 ==  0
    value6 = 10;     // value6 == 10
    value7;          // value7 == 11
    value8;          // value8 == 12
    value9 = value1; // value9 ==  1
    value10;         // value10 == 2
    value11;         // value11 == 3
}

// byte enum declaration
enum MyByteEnum : byte
{
    value0;          // value0 == 0
    value1 = 0;      // value1 == 0
    value2;          // value2 == 1
    value3 = 254;    // value3 == 254
    value4;          // value4 == 255
    value5 = value3; // value5 == 254
}

// char enum declaration
enum MyCharEnum : char
{
    value0;          // value0 ==  0
    value1 = '1';    // value1 == '1'
    value2;          // value2 == '2'
    value3 = '3';    // value3 == '3'
    value4;          // value4 == '4'
    value5 = value3; // value5 == '3'
}

Flags

Represents a bit flags data type. It contains several identificators with their bits values.

flags

All flags identificators must have values!

Examples:

// Default flags declaration (flags type is int32)
enum MyFlags
{
    flag0 = 0x00;                    // flag0 == 0x00
    flag1 = 0x01;                    // flag1 == 0x01
    flag2 = 0x02;                    // flag2 == 0x02
    flag4 = 0x04;                    // flag4 == 0x04
    flagc = flag2;                   // flagc == 0x02
    flags = flag1 | flag4;           // flags == 0x05
}

// uint64 flags declaration
enum MyUInt64Flags : uint64
{
    flag0  = 0x00;                   // flag0  == 0x00
    flag1  = 0x01;                   // flag1  == 0x01
    flag2  = 0x02;                   // flag2  == 0x02
    flag4  = 0x04;                   // flag4  == 0x04
    flag8  = 0x08;                   // flag8  == 0x08
    flag16 = 0x10;                   // flag16 == 0x10
    flag32 = 0x20;                   // flag32 == 0x20
    flag64 = 0x40;                   // flag64 == 0x40
    flagc  = flag4;                  // flagc  == 0x04
    flags  = flag2 | flag8 | flag32; // flags  == 0x2A
}

Structs

Represents a struct data type. It aggregates several fields of different types into one logic structure. Moreover structs can be extended using inheritance.

Struct keys
Struct numeration
Struct inheritance

Implementation of the struct type depends on equivalent type in each programming language (e.g. struct in C++ and C#, class in Java).

Schema legend:

struct

struct properties:

Examples:

// Order declaration
struct Order
{
    [key] int32 uid;
    string symbol;
    OrderSide side;
    OrderType type;
    double price = 0.0;
    double volume = 0.0;
}

// Account balance declaration
struct Balance
{
    [key] string currency;
    double amount = 0.0;
}

// Account declaration
struct Account
{
    [key] int32 uid;
    string name;
    State state = State.initialized | State.bad;
    Balance wallet;
    Balance? asset;
    Order[] orders;
}

Final model is more compact, but you will loose versioning and the possibility to extend the struct with new fields:

struct-final

Final model struct properties: