Strings, lengths, and field settings¶
So far, we have used the default format. Now let us look at cases where a field needs different settings.
Choose an encoding¶
For example, a message must contain only ASCII text:
from bytespec import ProtoModel, field
class Message(ProtoModel):
text: str = field(encoding="ascii")
message = Message(text="Hello!")
print(Message.decode(message.encode()).text) # Hello!
Without encoding, UTF-8 is used. ASCII cannot encode Cyrillic characters, so encode() raises EncodeError. An unknown encoding name raises SchemaError. The same settings apply to string enums and datetime.
Change the length prefix¶
Before a string, the library writes its length in bytes. This number is called a length prefix and takes 4 bytes by default. For short text, you can choose a narrower prefix:
class Message(ProtoModel):
text: str = field(prefix_length=1)
message = Message(text="Hello!")
print(Message.decode(message.encode()).text) # Hello!
prefix_length=1 reserves one byte for the length, not for the text itself. The field can now hold up to 255 encoded bytes. In UTF-8, that does not always mean 255 characters: for example, "я" takes 2 bytes.
|
Prefix size |
Maximum data length |
|---|---|---|
|
1 byte |
255 bytes |
|
2 bytes |
65,535 bytes |
|
4 bytes |
|
|
8 bytes |
|
|
1–10 bytes |
|
This is the limit imposed by the field prefix. The size of the entire model payload has a separate limit, set by PayloadLength in Headers and framing (2**32 - 1 bytes by default). The prefix itself is not included in the length it stores.
The same rules apply to bytes. Here is a variable-length prefix:
from bytespec.types import VarUInt
class Packet(ProtoModel):
data: bytes = field(prefix_length=VarUInt)
packet = Packet(data=b"ABC")
print(Packet.decode(packet.encode()).data) # b'ABC'
Pass VarUInt specifically. VarInt is not supported as a prefix. Setting prefix_length=None in field() keeps the default; it does not disable the prefix. For a fixed number of bytes without a prefix, there is a ready-made solution: Use a built-in codec.
Reuse a configuration¶
When several fields share a format, giving it a name is convenient. Annotated associates a Python type with bytespec settings:
from typing import Annotated
from bytespec.types import CodecSpec
ShortText = Annotated[str, CodecSpec(prefix_length=1, encoding="ascii")]
class User(ProtoModel):
name: ShortText
city: ShortText
user = User(name="Anna", city="Oslo")
print(User.decode(user.encode()).city) # Oslo
ShortText remains a string in Python code, but each such field is written in ASCII with a one-byte length prefix. CodecSpec groups the settings we previously passed to field().
Configure list elements¶
Now we can use ShortText inside a list:
class Message(ProtoModel):
tags: list[ShortText] = field(prefix_length=2)
message = Message(tags=["red", "blue"])
print(Message.decode(message.encode()).tags) # ['red', 'blue']
Here, the length of the entire list takes 2 bytes, and the length of each string takes 1 byte. The list length is the total size of its encoded elements in bytes, not their count.
The list’s field() settings configure only the list itself. Thus, list[str] = field(prefix_length=2, encoding="ascii") would leave the elements in UTF-8 with four-byte length prefixes. Element settings are defined through their type, as in the example above.
For numbers and UUIDs, prefix_length has no effect. Nested models also keep their own settings. If you combine field() and CodecSpec on one field, Combining settings applies.
Next: Putting a message together, where we will combine the features we have covered in one message.