Authored by Sophia, Platform Engineer at Stelia
In a previous article, we discussed how Rust has earned its place in our system design and some of the reasons why we choose to use it extensively at Stelia. One of those reasons is “fearless concurrency”, which enables developers to write responsive, high-performance code knowing Rust makes certain types of bugs impossible. This allows us at Stelia to respond to API requests quickly, transfer significant amounts of data efficiently, and ultimately provide resources to customers as fast as possible. In this second piece, we explore how “fearless concurrency” empowers development, but also consider situations where Rust guarantees have their limits.
Why concurrent programming matters
With modern server CPUs having over 100 cores, and GPUs having tens of thousands, any program with performance requirements needs to be executing code on as many of those cores as possible – only utilising a single core sacrifices far too much time. But writing code that runs on multiple cores can introduce bugs that are impossible in sequential programs. Mainstream programming languages have given programmers the capability to write concurrent programs for decades, but Rust is the first mainstream language that prevents many of those bugs by itself. Developers can rely on Rust to point out many concurrency mistakes at compile time, allowing them to write concurrent code “fearlessly”.
Concurrency and parallelism
Two oft-confused terms are concurrency and parallelism – a concurrent program can perform multiple tasks non-sequentially (i.e., task 2 does not have to wait for task 1 to finish), and a parallel program executes multiple tasks at the exact same time. If Alice is writing an email when her phone rings, she’s operating non-concurrently if she stops typing the entire time she’s on the phone; concurrently if she alternates between typing and talking; and both concurrently and in parallel if she’s typing and talking at the exact same time. Operating concurrently doesn’t by itself let Alice work faster, she can just alternate between the two tasks. She can only finish sooner by doing both at once (i.e. in parallel), and being able to work concurrently generally makes that easier – if Alice can’t alternate between typing and talking, she definitely can’t do both at the same time.
Rust offers two ways to write programs that execute non-sequentially, widely called “sync” and “async” Rust. “Sync” Rust achieves concurrency using operating system threads, which may also execute in parallel by running on different cores. “Async” Rust executes a task until it gets stuck waiting for something, then immediately switches to another task, etc, and can execute on one core or multiple.
How Rust makes concurrency “fearless”
Rust describes itself as offering “fearless concurrency” because it prevents common concurrent programming bugs at compile time, in particular a specific class of race conditions. A race condition refers to a program unintentionally depending on the order of computations. Typically, the code executes in an order where the program works as expected, but occasionally, some code executes earlier or later than the developers anticipated, and the program breaks.
Imagine Alice and Bob share a shopping list in a notebook (they’re old-school) – when one of them remembers something they need, they write it on the next empty line. If both of them ever tried to add to it at the exact same time, they’d write over each other and produce an incomprehensible mess.
We could describe this scenario as a data race (when Alice adds to it, she is, effectively, racing to finish before Bob has to add something), which is a specific type of race condition that happens due to shared mutable state (the line of the notebook). The solution for Alice and Bob is obvious – only one of them should write at once (which we’d describe as “synchronising”). But in software, with potentially dozens of cores accessing data and hundreds of thousands of lines of code, it’s often much harder to avoid this scenario.
Even if Alice and Bob stop trying to add things at the same time, they might still encounter race conditions simply by viewing the shared mutable state at the wrong time. If Bob is ordering groceries and glances at the list while Alice is halfway through writing something, she’ll be quite confused when their delivery contains straw instead of strawberries. Bob misunderstood because he read the list while Alice was still modifying it. It isn’t enough just to synchronise when they write to the list, but also when they read from it.
How Rust solves for this
“Safe” Rust has rules that prevent the modification of shared data unless all accesses to it are synchronised, which eliminates this entire class of software bugs. This is verified by the compiler when building the program – instead of programmers trying to ensure that all their code interacts with shared mutable state in a correct way, the compiler forces them to. Rust does provide a mechanism to opt out of some of its rules by marking specific sections of code as “unsafe”, meaning the developer is responsible for ensuring the code is correct (i.e., free of data races, among other things).
Rust also provides an alternative to shared mutable state through message passing; instead of sharing state between threads, data can be transferred through channels, ensuring only one thread has a reference to it at a time.
This makes it trivial to write concurrent programs without the risk of data races – in Rust, you’re doing that by default. If performance requirements change and introducing parallelism would improve the program’s performance, Rust programs can usually be parallelised in less time, and with a smaller chance of introducing concurrency bugs, than any other mainstream programming language. If a Rust program does have a data race, it can only be a bug in the small percentage of “unsafe” code, so developers have to debug a much smaller surface area than they would if the program was written in another language.
Fearful concurrency? Where Rust’s guarantees have their limits
Despite Rust’s concurrency guarantees, certain classes of bugs remain outside its scope.
Deadlocks
Code is still susceptible to deadlocks, where parts of a system are stuck mutually waiting for each other to proceed.
If Alice and Bob have agreed not to write on their to-do list until the other has finished with it, there’d be an issue if Alice picks up the notebook the same time Bob picks up the pen – Alice will hold the notebook until she can get the pen to write her task down, and Bob will hold the pen until he’s got the notebook to write his task down. The pen and notebook are resources that are ‘locked’, and Alice and Bob are deadlocked.
While it’s impossible for Rust to prevent this, drop guards mean developers can’t simply forget to write the code that unlocks the resources – a surprisingly common occurrence. Testing tools, such as loom and shuttle, can be used to test for deadlocks (and in some cases, loom can prove the program doesn’t contain them).
Cancellation safety in async Rust
Unfortunately, async Rust includes the pitfalls present in sync Rust, while also introducing a less obvious one – cancellation safety.
Async Rust generally looks very similar to sync Rust code, except that calls to async functions require adding the .await operator. These ‘await’ points are where the task may identify that its progress is currently blocked on something, and it will be paused so another task may be executed instead.
Unlike sync Rust code, however, the caller of an async function may cancel it – command it to stop progressing, permanently. Imagine a function that polls data from a queue, awaits a network request, and then reinserts it if the request fails; if the function is cancelled while awaiting, the data will simply be discarded.
This is particularly relevant when writing libraries, especially ones that involve FFI or other unsafe code, as developers must remember that their function may stop running after any .await and ensure their code is valid regardless.
Rust is valuable, but not a silver bullet
Rust is an excellent programming language with many benefits beyond the way it enables concurrent programming, but it isn’t magic. Data races are generally some of the hardest bugs to reproduce and address, and eliminating them at compile time is a huge benefit, but other concurrency bugs are still possible. Concurrent Rust programs should make use of the excellent tooling available to test and verify correctness, and developers must remember that “no data races” does not mean “no concurrency bugs”.
For teams like ours building platforms where performance, reliability, and correctness are non-negotiable, we benefit significantly from Rust’s guarantees. Language selection is ultimately a strategic decision, and fearless concurrency remains one of the more compelling reasons that Rust earns its place in systems where the cost of getting concurrency wrong is high. While it doesn’t eliminate the need for rigorous engineering, it allows for that rigour to be directed where it matters most.
