January in TigerLand

    Dear friends,

    Hope your January was jazzy! Last month, we launched a brand-new docs site, optimized TigerBeetle for workloads that submit just a few operations per request, welcomed Marina and Martijn to the team, and added new videos to the TigerTube (aka Database TV!).

    Let’s go!

    Benjamin Franklin

    • TigerBeetle’s design goals are safety , performance , and developer experience. Irrevocably in that order! But it’s not a zero-sum game. For example, take the case of replication.

    In a TigerBeetle cluster, the primary replica is responsible for replicating a client request throughout the cluster. Once a quorum of replicas backup the request, the primary can continue processing. Not all replicas will always be in sync with the primary and machines can respond at different speeds, so it’s expected that a replica may lag behind the cluster temporarily.

    However, we noticed a case where TigerBeetle processes a high volume of tiny requests so quickly, that a minority of the cluster struggle to keep up. Over time, these lagging replicas may then fall too far behind to catch up, triggering a repair mode called state sync and slowing down the cluster!

    • Executing a request too quickly should never be a problem. It highlights we weren’t replicating messages fast enough, especially the small ones! When sending a message to another replica, instead of letting it sit in the send queue, we can take advantage of a non-blocking write to the socket, which, for small messages, likely copies straight to the kernel’s send buffer. If this kernel buffer would be exceeded(EWOULDBLOCK), we fall back to asynchronous sending via io_uring on Linux.

    • TigerBeetle distributes network load across the cluster by sharing replication responsibilities across replicas in a ring topology. This effectively prevents the primary replica’s network bandwidth from becoming a bottleneck. However, for tiny messages, it can actually slow down replication!

    To address this, we introduced an experimental star-replication topology as a workaround while improving replication to switch between topologies depending on the workload.

    • Replication wasn’t the only issue: once a lagging replica entered repair mode, it became overly chatty, requesting a large number of data blocks that were already outdated by the time the request arrived!

    While TigerBeetle’s repair protocol performance was already close to rsync, understanding this issue nevertheless allowed us to make the repair protocol more efficient in all scenarios, not just for small messages.

    • We took this opportunity to tackle another common issue with workloads that process only a few operations per request: space and write amplification! In a previous newsletter, we discussed how there’s always a trade-off between space efficiency (smaller file sizes) and write efficiency (fewer write operations) when compacting partially filled tables in an LSM Tree. We can’t have both… unless we design the problem away!

    TigerBeetle’s design follows NASA’s Power of Ten Rules for Safety-Critical Code, to enforce explicit limits on everything! By knowing the limits (in advance), we can actually know if the memory table still has room for one more request—and if it does, simply not compact it! Instead, we coalesce and deduplicate elements in memory, achieving both space and write efficiency by avoiding the compaction of nearly empty tables!

    While the TigerBeetle server is currently production-ready only on Linux, this is a great quality-of-life improvement for developers running on Windows!

    • Finally, our documentation has been completely rewritten as a static website! Handcrafted with care and designed in every detail, we set out to create the experience we prefer when browsing documentation: simple, clear, and fast. Just don’t head straight to the 404 error page… or you might find the *not found* beetle! 😉

    In a small step for Database Television (DTV), the number of live TigerBeetle channels doubled with both matklad and brson taking to the airwaves. IronBeetle continued to look at how TigerBeetle works from the inside, while Brian’s TigerBeetle Stream (new) kicked off with building a TigerBeetle Rust client (brson was the Rust project lead at Mozilla).

    Join us every week on Twitch or catch up on the TigerTube.

    Have you tried rubbing a database on it? 😆 Join us this month (remotely) at Jamie Brandon’s HYTRADOI for matklad’s talk on the Rocket Science of Simulation Testing! The conference is hosted in a chat room and all talks are recorded and captioned in advance. You can join from a different time-zone, watch on the jog, and hang out at one of the coolest conferences in (art nouveau!) DBMS design.

    A rendezvous for Developers, Data Scientists and DevOps, this year’s J On The Beach takes place in Malaga on 14-16th May. Federico will take to the stage in person (marking the very first time that TigerBeetle sets foot in Spain!).

    Riccardo Binetti’s talk “TigerBeetlex: An Elixir and Zig Love Story” got accepted to ElixConf, taking place in Poland (and virtually) in May. A member of the Zig and TigerBeetle communities, Riccardo will share how he built a TigerBeetle client in Elixir. This story has been incredible. We first met Riccardo in person in Milan in April ‘22 at the very first European Zig meetup, and since then he went on to create and maintain TigerBeetlex. We’re rooting for you, Riccardo!

    Friends of TigerBeetle around the world are asking when and where the third edition of Systems Distributed will take place. We’re excited to share the launch announcement with you any day now!

    Tweet Tweet Tweet Tweet Tweet Tweet Tweet Tweet

    A warm welcome to Marina in Marketing and Martijn in Support!

    Till next time… country roads!

    The TigerBeetle Team

    RSS iconRSS
    An idling tiger beetle Speech bubble says hi