Why did we choose Rust over Golang or C/C++ to develop TiKV?

What is Rust

Rust is a systems programming language sponsored by Mozilla Research. It moves fast and steady with a 6-week release cycle ever since its 1.0 version in May 2015.

See the following list for some of the features that most attract us:

The Rust Ecosystem

Because of the excellent package management tool, Cargo, Rust has many types of libraries, such as Hyper for HTTP, Tokio and mio for asynchronous I/O, basically all the libraries that are required to construct a backend application.

Generally speaking, Rust is mainly used to develop server-side applications with high performance at this stage. In addition, its innovation in the type system and syntax gives it a unique edge in developing Domain-Specific Libraries (DSL).

The Rust adoption

As a new programming language, Rust is unique. To name just a few projects that are using Rust,

As one of the listed Friends of Rust, TiKV has been one of the top projects in Rust according to the Github trending.

TiKV is a distributed key-value database. It is the core component of the TiDB project and is the open source implementation of Google Spanner. We chose Rust to build such a large distributed storage project from scratch. In this blog, I will uncover the rationale.

In the past long period of time, C or C++ has dominated the development of infrastructure software such as databases. Java or Golang has problems such as GC jitter especially in case of high read/write pressure. On the one hand, Goroutine, the light-weight thread and the fascinating feature of Golang, has significantly reduced the complexity of developing concurrent applications at the cost of the extra overhead in context switching in the Goroutine runtime. For an infrastructure software like a database, the importance of performance goes without saying. On the other hand, the system needs to remain its “Certainty” which makes it convenient for performance tuning. But introducing GC and another runtime contributes to the opposite. So for quite a long time, C/C++ seems to be the only choice.

TiKV originates from the end of 2015. Our team was struggling among different language choices such as Pure Go, Go + Cgo, C++11, or Rust.

   2. There are too many and too different programming paradigms in C++ as well as too many tricks. It demands extra costs to unify the coding style especially when there are more and more new members who might not be familiar with C++. After years of using languages with GC, it is very hard to go back time for manually managing the memory.

3.	The lack of package management and CI tools. It appears not to be trivial, but the automated tools are very important for a large project because it is directly related to the development efficiency and the speed of iterating. What’s more, the C++ libraries are far from enough and some of them need to be created by ourselves.

To sum up, Rust, as an emerging programming language, seems to be new to most of the developers in China, but it has become the most promising challenger to C/C++. Rust was also crowned the “most loved” technology in StackOverflow’s 2016 developer survey. So from a long term, Rust will shine in scenarios where memory safety and performance matter the most.