September in TigerLand
Dear friends,
Hope your September was spectacular! Last month, we improved logical availability, prepared for Zig 0.15.1 and landed performance improvements that, together, broke the +600K TPS record! We also released the final talk from SD25 (1000x), announced our connector for Redpanda, and spoke about TigerBeetle at the New York Stock Exchange.
Let’s go!
True art is creation, and creation is beyond all theories. C.G. Jung
- The design phase is the cradle of performance, where high-level architecture shapes interfaces that enable improvements by multiple orders of magnitude. It’s rewarding to see our toddler beetle growing and learning to use the inherent power within.
Last month, we landed a set of performance-related changes, the result of meticulous work that put the data plane under the microscope, which reduced tail latency and broke the +600K TPS record!
We introduced an optimized implementation of the Least Significant Digit (LSD) Radix sort, a stable, linear-time sorting algorithm.
Sorting the memory tables with LSD-Radix delivered a ~23% throughput increase over the standard library sort, as it uses fewer CPU cycles, executes fewer instructions, and incurs fewer branch misses, resulting in higher instruction-level parallelism (IPC).
The k-way merge iterator, responsible for joining streams during compaction and queries, also received performance improvements. It now uses a tournament tree (aka tree of losers) instead of a heap, skips unnecessary sorts, and avoids binary search where possible by leveraging known key ranges.
To keep P100 latency predictably low, TigerBeetle employs an incremental sort strategy, amortizing the cost of sorting the entire mutable table by doing only small portions at a time and performing the final full sort at the end. Although this leverages the fact that the sort algorithm was optimized for cases where the data being sorted is a sequence of pre-sorted sub-arrays, this redundant final step was still sub-optimal.
Now, we use the new k-way merge iterator to join all sorted runs together, removing the need for a full sort at the end, improving latency and query times.
We removed one unnecessary memory copy from the state machine logic! The benefits compound exponentially over time.
We improved the cluster’s logical availability by allowing a client partitioned from the primary to submit requests and receive replies from backup replicas. Additionally, the client now always sends the same request through an alternative backup route alongside the primary, not relying on timeouts and retries to react to network partitions.
We also spotted a duplication of state between the MessageBuffer and the Connection, where both had to keep track of what kind of peer they were dealing with (e.g., a client or a replica). The refactored code is now safer and simpler, as the MessageBuffer is solely responsible for iterating over the available messages.
We redesigned the interface between Replica and Multiversion binaries, simplifying the code responsible for coordinating the upgrade of the tigerbeetle binary across the cluster.
Multiversioning support for Windows is now significantly more robust.
It was also time for some spring cleaning, especially to prepare for the Zig 0.15.1 upgrade, by anticipating changes that could already be implemented before upgrading. By the way, we love the velocity going on in Zig right now—Andrew making the right calls for the long term.
The Change Data Capture (CDC) job also received a new command-line argument, --requests-per-second-limit, to throttle the maximum number of requests per second.
When streaming changes from TigerBeetle, we usually want them sent as soon as possible, but this can sometimes overflow downstream systems with too many messages. Users can now fine tune for low latency streams while keeping a safe ceiling limit. Please see our documentation for more details.
- We are very excited to have contributed to the Redpanda Connect repository with a new Open Source (Apache 2.0) CDC connector for TigerBeetle!
Redpanda Connect is a powerful and flexible stream processor that users can leverage to connect TigerBeetle with a wide range of targets, including popular message brokers, analytical databases, and data lakes.
- Raui Ghazaleh helped clarify our documentation about the behavior of expired pending transfers on closed accounts. Thank you, Raui!
Last month in IronBeetle, we started the refactor to reduce our reliance on BoundedArray, computed next hop for routing and tracking replication latency and answered: which is easier to write: a function to generate a random permutation, or a function to enumerate all permutations…
Join us live every week on Twitch or catch up on the TigerTube!
1000x: The Power of an Interface for Performance (Sep 1) In the final talk from Systems Distributed ‘25, Joran deconstructed and reconstructed one of the world’s most popular interfaces, to show the power of an interface for performance. Lights, camera, photo finish!
The Making of Systems Distributed (Sep 3) “ The best software is more than a program, it’s a relationship between the makers and the users ”. Take a walk behind the scenes of SD25, an annual benefit for Zig Software Foundation. We already can’t wait for SD26, spending time with friends, together paying it forward.
Look Out for Bugs (Sep 4) Matklad wrote about finding bugs: “ Bottom line: reading the code is surprisingly efficient at proactively revealing problems. Create space for calm reading. When reading, find ways to build mental models quickly, this is not entirely trivial. ”
Redpanda and TigerBeetle: Real-time Financial Transaction Streaming (Sep 24-25) We announced the TigerBeetle connector for Redpanda—” real-time transaction streaming, now correct and fast, by default ”—and spoke about the partnership at a panel and evening event hosted by NYSE Wired, with thanks to Brian J Baumann, as well as a closed studio interview with John Furrier of theCUBE, along with fellow #AnimalDatabaseAlliance member, Alex Gallego. Special times, strong friendships, steaming subways in NYC!
TigerBeetle at the New York Stock Exchange (Sep 25) Bright and early, Joran was (back) at the NYSE with John Furrier to talk about TigerBeetle: the world moving to real-time, transaction volumes increasing, and why “the world doesn’t need another string database “, but rather—one that can count.
P99 CONF: The Tale of Taming TigerBeetle’s Tale (Oct 22-23) It’s nearly time for P99 CONF! In just two weeks, hear about performance wins via modern superscalar CPUs, patterns CPUs love, and speed in production—with TigerBeetle’s Tobi!
‘Till next time… New York, New York!
The TigerBeetle Team
RSS
