June in TigerLand

    Dear friends,

    We hope your June was giant. Last month, we rewrote the REPL parser, changed our Python client packaging, and published our Ruby client. We also went to Wales, Georg and Tobi’s paper on Bi-Channel Networking got accepted at EDBT, and we announced the next TigerHoops at MIT’s Rockwell Cage.

    The 4th Systems Distributed in Boston is three weeks away! Take a break from your regular programming with speakers including Andrew Kelley, Gerard Holzmann, Jens Axboe, James Cowling, Joran Dirk Greef, and Zooko.

    Let’s go!

    “For me, the purpose of life is, at least partly, to have joy.
    Programmers often feel joy when they can concentrate
    on the creative side of programming.”
    — Yukihiro “Matz” Matsumoto

    Ruby is now an official language for TigerBeetle clients! Polished with joy, it includes a u128 bounds check, more idiomatic status return type, and a migration guide; with thanks to @citizen428! 💎

    We improved the TigerBeetle REPL, often the ‘first touch’ for a new user: The REPL parser was rewritten to add a fuzzer, and snapshot testing, to assert the exact error message users see, instead of just error codes. We fixed latent bugs uncovered in the old parser: dropped parameters, broken semicolon handling, mixed-case flags being ignored, duplicate or unknown flags slipping through, and an outright infinite loop on malformed input.

    We also took control of our Python client packaging: hatchling was replaced with our own wheel builder written in Zig, giving fully reproducible builds with no external dependencies. This allows us to check whether the published client packages can still be built from source.

    Finally, we introduced—and then fixed—an upgrade-related bug:

    • TigerBeetle stores its data in LSM trees, organized into a forest. Alongside primary lookups, the forest maintains secondary indexes so objects can also be found by other fields. A secondary index like Transfer.code can refer to many transfers that share the same value, whereas a unique key like Transfer.pending_id refers to exactly one transfer.
    • In May, we generalized TigerBeetle’s LSM forest so objects can have multiple unique keys. We also recategorized Transfer.pending_id from CompositeKey to UniqueKey. Under the hood, both key types store the same value: a { field, timestamp } tuple. However, the key changed from { field, timestamp } → { field }, since the latter is sufficient to guarantee uniqueness. This was intended to be a behavior-only change, but we overlooked that this new key format is materialized to disk during compaction, turning it into an on-disk format change in the LSM’s IndexBlocks and ManifestLog. Out in the field, that tripped an assertion during upgrade.
    • The fix was to revert unique keys to ordinary secondary indexes, while we work on binary-compatibility for IndexBlocks. We also extended inspect constants and added runtime assertions to catch unintended schema changes in future.

    Last month on IronBeetle, we continued the Pragmatics of Consensus mini-series where matklad and Tobi dive into our implementation of Viewstamped Replication. This time, we tackled State Sync: how a replica that’s fallen behind catches back up. We reviewed how we go from the abstraction of an infinite log to a physically realizable abstraction of a finite log at a checkpoint, one of the most beautiful but also challenging parts of TigerBeetle!

    Join us live every Thursday at 5pm UTC on Twitch, YouTube, and X!

    For Straddle, a Deterministic System of Record, Jun 5 Instead of a SaaS-style ledger with “all the bells and whistles”, Straddle wanted a database with explicit guarantees: strict serializability, atomicity, and no lost writes. For CEO Keith Raphael, with a 15-year career in payments, “discovering TigerBeetle really got the minds flowing”, and from first discussions to TigerBeetle Cloud contract signature took a week. And last month, we joined Straddle’s engineering offsite in Wales!

    Scalar Replacement of Aggregates: How “Copy to Locals” Unlocks the Compiler, Jun 15
    According to Debasish Gosh, some performance fixes “look almost suspiciously small in the diff”—including PR #3201! We enjoyed his post exploring this performance work, including an explanation of scalar replacement of aggregates. Thanks, Debasish!

    The Bi-Channel Networking Paradigm for Database Systems in the Cloud, Jun 19
    “Database systems should no longer treat networking as a black box but co-design it with database operations”, according to Georg Kreuzmayr and Tobias Ziegler of TigerBeetle, Muhammad El-Hindi, Benjamin Wagner, and Viktor Leis, in their paper accepted for EDBT/ICDT, 2027 in Lille. Congratulations to the authors!

    Jonas Axboe Extends his Pro 1500 Championship Lead! Jun 20-21
    Round 4 of the Radical Cup proved to be another exhilarating weekend of racing: Jonas took pole in 2 of 3 races at Road America, extending his Pro 1500 class championship lead with 391 points! Congrats, Jonas!

    Systems Distributed, Jul 27–28
    We’re finalizing designs, details, and boating schedules, as we prepare to welcome this year’s participants for Systems Distributed. If you’re on the fence, jump on that plane/train to Boston; because the best software is more than a program and the future deserves good programming! See who’s speaking.

    Zig Day, Jul 26
    The day before Systems Distributed, we’ll be taking part in Boston’s first Zig Day, organized by VP of Community at Zig Software Foundation, Loris Cro! Although this special event, taking place at MIT Museum, is now full, you can join the waiting list and if you’re lucky to get a spot, we’ll see you there!

    TigerHoops the 4th in Boston, Jul 29
    New York, San Diego, the Bay Area… Boston! The fourth TigerHoops will be at MIT’s Rockwell Cage, at 5pm on Wednesday, July 29th. Book your spot for friendly pickups, Gatorade, and beats. Beginners welcome and vests provided.

    P99 CONF: The Fastest Object Storage Client in The World, Oct 21-22 Last year at P99 CONF, Dr. Tobias Ziegler shared the tale of taming TigerBeetle’s tail: how we reduced tail latency through algorithm engineering. This year, Georg Kreuzmayr will take TigerBeetle on tour, and speak about the fastest object storage client in the world (that we know of)… We can’t wait!

    Tweet Tweet Tweet Tweet Tweet Tweet Tweet

    ’Till next time… Ruby, Ruby, Ruby!

    The TigerBeetle Team

    An idling tiger beetle Speech bubble says hi