Binary / wire format¶

A model is encoded without field names, type tags, alignment, or padding. The receiver must know the model class and its settings in advance.

Header and payload¶

The default header is Constructor(2), Flags(8), PayloadLength(4):

constructor (2 bytes) | flags (8 bytes) | body length (4 bytes) | fields

The header takes 14 bytes, even without optional fields. PayloadLength stores the byte length of all fields after the complete header. The entire header is excluded from this length. __header__ lets you change the order or encoding, or remove elements, including the whole header; see Headers and framing.

__constructor__ defaults to 1, and __byte_order__ to ByteOrder.BIG. The constructor is checked against the selected class; the library does not assign unique identifiers or select a model using this number.

An exact example¶

from bytespec import ProtoModel, field
from bytespec.types import UInt8

class Packet(ProtoModel):
    __constructor__ = 0x1234
    number: UInt8
    note: str | None = field(flag=0, prefix_length=1)

encoded = Packet(number=7, note="A").encode()
print(encoded.hex(" "))

12 34 00 00 00 00 00 00 00 01 00 00 00 03 07 01 41

34                    constructor = 0x1234
00 00 00 00 00 00 01   flags: bit 0 set
00 00 03              payload length = 3
                     number = 7
41                    note: byte length = 1, UTF-8 "A"

With note=None, the bit would be clear, the bytes 01 41 would be absent, and the payload length would be 1. Indices determine field order; flag numbers do not change it. Non-zero defaults are encoded as ordinary values; the format has no separate default markers.

Byte order¶

ByteOrder.BIG means big-endian, and ByteOrder.LITTLE means little-endian. The setting applies to fixed-width integers and floats, numeric header elements, and fixed-width length prefixes. Native alignment is not used.

from bytespec import ByteOrder
from bytespec.types import UInt16

class Point(ProtoModel):
    __byte_order__ = ByteOrder.LITTLE
    x: UInt16

print(Point(x=0x1234).encode()[-2:].hex(" "))

34 12

For varints, the algorithm fixes the byte order independently of the model. The contents of bytes and UUID.bytes are unchanged; text is determined by encoding. A nested model always uses its own __byte_order__ setting.

For UTF-16, the encoding itself determines the text’s byte order: choose utf-16-le or utf-16-be explicitly if the protocol requires a specific variant.

Values inside the payload¶

Fixed-width integers occupy 1, 2, 4, or 8 bytes. Signed integers use two’s complement. Floats are encoded as IEEE 754 binary32/binary64. A bool is one byte, 00 or 01.

str contains the length prefix of the encoded text, followed by the text. bytes contains the length and the original bytes. FixedBytesCodec(N) and UUID have no prefix. Datetime uses an ISO string representation, and a string enum uses the representation of its .value. IntEnum uses VarInt through EnumCodec; ordinary int also uses VarInt. date and time have no automatic codec.

Supported prefix widths and their limits are listed in Strings, lengths, and field settings. A field’s size in bytes is the prefix size plus the prefix’s value.

VarUInt and VarInt¶

VarUInt writes groups of 7 bits, starting with the least significant. The high bit of each byte indicates continuation. Zero is encoded as 00, 127 as 7f, 128 as 80 01, and 300 as ac 02. Only canonical encodings in the range 0 .. 2**64 - 1 are allowed: at most 10 bytes, with at most one significant bit and no continuation in the tenth byte. A redundant zero high-order group is forbidden.

VarInt first applies ZigZag: n >= 0 becomes 2*n, and negative n becomes 2*abs(n)-1. The result is then encoded with VarUInt. Thus, 0 → 00, -1 → 01, 1 → 02, and -2 → 03. The supported range is -2**63 .. 2**63 - 1.

Lists and nested messages¶

list[T]:
+------------------------+----------+----------+-----+
| byte length of items   | item 0   | item 1   | ... |
+------------------------+----------+----------+-----+

model field / model list item (default header):
+-------+----------------+-------------+
| flags | payload length | own fields  |
+-------+----------------+-------------+

A list does not store its element count. The decoder isolates a buffer of the specified length and reads values until it is exhausted. Strings and bytes within a list keep their own prefixes. A model keeps its own header, skipping Constructor, and uses its own byte order. The remaining header counts toward the total list length. With __header__ = (), the model consists only of fields; their codecs determine the boundaries. For an exact byte example, see How nested model headers are written.

Boundaries and unknown data¶

When decoding, the decoder processes only the declared header elements. If Constructor is present, it is checked during ordinary standalone decoding. If PayloadLength is present, the decoder checks that the declared body is available. In this case, fields receive a buffer bounded by the end of the payload: a built-in codec cannot finish reading a field using bytes from the next message. Nested models are bounded by the parent’s payload, and list elements by the list’s separate buffer. Malformed values are rejected as described in Handling errors.

There is an important distinction between two kinds of extra bytes:

Inside the declared payload: any bytes after all known fields are skipped. Unknown flag bits are also ignored. Skipped bytes and bits are not preserved in the instance and disappear after another encode().
After the declared payload: decode() raises DecodeError. decode_from() returns the end of the current message, leaving subsequent bytes to the caller.

Without PayloadLength, the message ends where its known fields end: an unknown tail is not skipped. decode_from() cannot distinguish available bytes from the next message from the current body, so external boundaries must be enforced by slicing.

This does not provide arbitrary compatibility between schemas. Adding data in the middle of the payload shifts subsequent known fields: they have no individual tags. A missing new required field is not restored from a default. When changing a schema, check decoding in both directions.

Boundary checks do not replace resource limits: the current API has no max_payload_size, max_string_size, max_list_elements, or max_depth. When handling external messages, input size and any application-specific limits must be enforced outside the library. The format has no checksum; a byte change that still represents a valid value is not detected on its own.

Exact signatures and class settings are collected in API Reference.