A sleek Python library for binary data

   

---

tibs is a Rust-backed Python library for binary data that does not assume everything fits neatly into bytes. Use it for packets, registers, instruction formats, bitsets, compressed data and streams where fields can have many different interpretations and be any number of bits long.

It is used to power the popular bitstring library, which is by the same author.

Install

pip install tibs

Tibs works with Python 3.10 and later. There are pre-built wheels for most common platforms; if there are issues then please let me know.

The full documentation is available on Read the Docs.

What Tibs is

Tibs is a sequence of bits — like bytes, but the unit is the bit and the length can be anything. That sequence is the whole object: you slice it, concatenate it, search and replace inside it at bit granularity, or pin searches to byte boundaries for stream parsing. Immutable Tibs gives stable values and cheap slices; Mutibs gives in-place edits, just as bytes pairs with bytearray.

find · rfind · find_all · replace · count · starts_with · split_at · chunks · + · in

On top of those bits, the same object gives you two lenses — two ways of reading and writing the sequence without ever copying it into another type.

Read it as typed fields. Pull integers, floats, strings, hex or binary of any bit length straight out of the bits, without hand-rolling shifts and masks. A view handles little-endian ordering and LSB0 field labels so you don't reshuffle data yourself, and extract / deposit reach fields that are scattered across a word.

from_u · to_f · bin / hex · Dtype · pack / unpack · .le · .lsb0 · field() · extract / deposit · f-string formatting

Read it as a set of bits. Bitwise algebra, cardinalities and set predicates, with no intermediate object built along the way. Mutibs doubles as a large mutable bitset.

& | ^ ~ · count_and · count_xor · intersects · is_subset_of · set / unset · all / any

These aren't separate modes or separate types — it's one object, and the lenses are just different questions you ask of the same bits.

And it's fast — often significantly faster than similar libraries, with a Rust core and a performance regression suite in CI. It can also compress and serialize itself when you need to put the bits on disk or the wire.

encode / decode — Rice, Zstd, Raw

Quick taste

The sequence and its two lenses, in code.

The sequence of bits. Tibs is immutable, like bytes — good for parsing, slicing, hashing and holding stable values. Mutibs is its mutable counterpart, for patching in place.

>>> from tibs import Tibs
>>> # Four flag bits, a 12-bit integer, then two payload bytes.
>>> packet = Tibs.from_joined(["0b1010", Tibs.from_u(3200, 12), b"OK"])
>>> packet
Tibs('0xac804f4b')

>>> flags, size, payload = packet.split_at([4, 16])
>>> flags.bin, size.u, payload.bytes
('1010', 3200, b'OK')

>>> patched = packet.to_mutibs()
>>> patched[4:16] = Tibs.from_u(2047, 12)
>>> patched[-8:] = b"!"
>>> patched
Mutibs('0xa7ff4f21')
>>> packet  # the original is untouched
Tibs('0xac804f4b')

Read it as typed fields. Pull fields out as integers, floats or bytes, letting a view take care of byte order and bit numbering — the sort of job that gets awkward quickly with plain bytes and masks.

>>> # Linux eBPF: little-endian instruction, LSB0 field labels.
>>> instruction = Tibs.from_bytes(bytes.fromhex("07 01 00 00 44 33 22 11")).lsb0.le
>>> dst_reg = instruction.field(11, 8).u
>>> f"r{dst_reg} += {instruction.field(63, 32):#x}"
'r1 += 0x11223344'

Read it as a set of bits. Compare containers as sets — cardinalities and predicates that never build an intermediate object.

>>> # Are all the required capability bits granted?
>>> required, granted = Tibs('0b0010_1000'), Tibs('0b1010_1100')
>>> required.is_subset_of(granted)
True

>>> # Hamming distance between two 256-bit fingerprints.
>>> a, b = Tibs.from_random(256, seed=b'doc-a'), Tibs.from_random(256, seed=b'doc-b')
>>> a.count_xor(b)
138

And it's all one object. This log scanner searches a byte stream, slices out each header, and reads its fields as integers, in one pass:

>>> sync = Tibs("0xaa55")
>>> log = bytes.fromhex("00 ff aa 55 11 02 c0 01 7f aa 55 22 01 40 aa")
>>> bits = Tibs(log)
>>> records = []
>>> for start in bits.find_all_iter(sync, byte_aligned=True):
...     payload_len = bits[start + 24:start + 32].u
...     payload = bits[start + 32:start + 32 + payload_len * 8]
...     if len(payload) == payload_len * 8:
...         records.append((bits[start + 16:start + 24].u, payload.bytes))
>>> records
[(17, b'\xc0\x01'), (34, b'@')]

The full documentation covers construction from ints, floats, bytes and strings; endianness; searching and replacing; rotations; bit indexing; serialization; and worked examples.

Performance

Tibs is written in Rust with PyO3. The repository contains a dedicated performance regression suite and CI workflow that compare benchmark medians against the base commit.

The regression benchmarks cover operations such as:

• bit-pattern search, reverse search and byte-aligned search
• logical operations on large aligned and unaligned slices
• counting, inversion, shifting, reversing and concatenation
• bulk mutation, slice replacement, deletion, append and pop
• bytes conversion, random generation and bool-list construction
• typed packing and unpacking such as u16 value streams
• a sieve workload using Mutibs as a large mutable bitset

For local comparisons, tests/performance_comparison_new.py checks common operations against bitarray. With bitarray installed, run:

python tests/performance_comparison_new.py --bytes 250000 --values 20000 --repeats 5

Benchmarks are machine-dependent, but the mean speedup over bitarray will typically be > 2x.

Examples

The runnable examples in examples/ are small, but they are meant to look like real binary-data tasks. They're grouped by the view each one leans on most, though several use more than one.

The bits as a sequence — searching a stream and slicing fields out of it.

Example	Shows
log_scan.py	Find byte-aligned sync markers and pull records from a stream.
instruction_scan.py	Search for an opcode with mask, ignoring the register fields.
patch_config.py	Patch compact config fields in place with Mutibs.

Typed fields and views — numeric fields, with byte order and bit labels handled by a view.

Example	Shows
construct.py	Build and unpack a structured MPEG-style header.
sensor_samples.py	Pack and unpack 12-bit ADC samples.
little_endian_registers.py	Decode and rebuild little-endian register dumps with u16_le.
ebpf_instruction.py	Decode LSB0, little-endian instruction fields.
scattered_field.py	Read and write a register field split around status bits with extract/deposit.

Sets of bits — bitwise algebra and comparison.

Example	Shows
sieve.py	Use a large mutable bitset for a prime sieve.
fingerprints.py	Compare items as sets of bits with count_and, count_xor and is_subset_of.

Project status

Tibs has reached the 1.0 stable API milestone. Documented public behavior will remain compatible across future 1.x releases. It is used to power the bitstring library and gets several million downloads per month.

There are over 1000 unit tests, including Hypothesis tests and performance benchmarks.

For the full API reference, see the documentation.

Credits

The tibs library was created by Scott Griffiths and is released under the MIT License.

The Tibs cat artwork was created by Ada Griffiths and is not covered by the software license. All rights reserved.