TiDB and TiKV Blog

See TiDB Deployment Tutorials for:

chaos-mesh-action: Integrate Chaos Engineering into Your CI

Chaos Mesh is a cloud-native chaos testing platform that orchestrates chaos in Kubernetes environments. This article shares how to use chaos-mesh-action, a GitHub action to integrate Chaos Mesh into the CI process.

Chaos Mesh 1.0: Chaos Engineering on Kubernetes Made Easier

Today, we are proud to announce the general availability of Chaos Mesh® 1.0, following its entry into CNCF as a sandbox project in July, 2020. After 10 months of effort within the open-source community, Chaos Mesh is now ready in terms of functionality, scalability, and ease of use. Here are some highlights.

TiDB Operator: Your TiDB Operations Expert in Kubernetes

This post analyzes the pros and cons of running databases in Kubernetes. It also introduces TiDB Operator, a tool for managing TiDB clusters in Kubernetes, and describes how large companies are using it in their production environments as well as their best practices.

A Scale-Out Database Powers China's Letgo with Reduced Maintenance Costs

Zhuan Zhuan embraced TiDB 4.0's new features with great results. It's a truly HTAP database with one-minute deployment, visual troubleshooting, and reduced maintenance costs.

ZaloPay: Using a Scale-Out MySQL Alternative to Serve Millions of Users

A DevOps engineer in ZaloPay, Vietnam's most popular mobile payment application, shared why his team chose TiDB as ZaloPay's merchant platform core database. He talked about their pain points, how they use TiDB, and what he likes about it the most.

How We Build an HTAP Database That Simplifies Your Data Platform

This post talks about why HTAP matters in a database platform, how TiDB implements HTAP, and how you can apply TiDB in different scenarios.

Why We Chose an HTAP Database over MySQL for Horizontal Scaling and Complex Queries

After trying out different database solutions, BIGO chose TiDB. In this post, a BIGO DBA talks about TiDB 4.0's new features, how they benefit BIGO's business, and BIGO's plan to use TiDB 4.0 even more in the future.

Announcing HTAP support in TiDB Cloud

Today, we're glad to announce that TiDB Cloud, our fully managed database service powered by TiDB, now supports Hybrid Transactional/Analytical Processing (HTAP) workloads.

How to Migrate Data from Amazon Aurora MySQL to TiDB Cloud

This post introduces how to smoothly migrate data from Amazon Aurora MySQL to TiDB Cloud using Dumpling and TiDB Lightning.

Celebrate TiKV Graduation within CNCF

The Cloud Native Computing Foundation (CNCF) today just announced that TiKV has become the twelfth project to graduate, joining other graduated projects such as Kubernetes, Jaeger, Harbor, etc.

PingCAPers Make Their Debut at VLDB

The 46th VLDB conference will be broadcast online from August 31st to September 4th. PingCAP is honored to attend this conference, and we will share two papers with you TiDB: A Raft-based HTAP Database and Interleaved Multi-Vectorizing.

PingCAP Successfully Completes SOC 2 Type 1 Examination for TiDB Cloud

PingCAP, provider of the cloud-native hybrid transactional/analytical processing (HTAP) TiDB database, successfully completes an SOC 2 Type 1 examination for TiDB Cloud.

Why We Chose a Distributed SQL Database to Complement MySQL

VIPKid chose TiDB to manage its high data volume, highly concurrent write application. Learn how TiDB excels in that scenario, along with multidimensional queries, data life cycle management, and real-time analytics.

TiDB Cloud: Managed SQL at Scale on AWS and GCP

This post introduces in detail the features of TiDB Cloud - the fully managed, horizontally scaling, cloud-native, distributed SQL database in the cloud.

Building an Automated Testing Framework Based on Chaos Mesh® and Argo

This article describes how we use TiPocket, an automated testing framework, to build a full Chaos Engineering testing loop for TiDB.

Announcing PD Transfer to TiKV Project

We decide to move the Placement Driver library entirely to TiKV org, happening at 11 AM, UTC+8, August 17, 2020.

What's New and Improved in TiDB Docs

Along with the rapid releases of TiDB products and projects, we also constantly update and improve our TiDB documentation. This post covers recent big changes, new content, and improvements in TiDB documentation.

Japan's Largest Mobile Payment Company Migrates from Aurora to a Scale-Out Database

As PayPay's business grew, their payment management database became a bottleneck. Learn why PayPay chose TiDB over Aurora to handle their most crucial data and how efficiently they migrated to TiDB.

TiDB 4.0: An Elastic, Real-Time HTAP Database Ready for the Cloud

At TiDB DevCon 2020, Max Liu, CEO at PingCAP, gave a keynote speech. He believes that today's database should be more flexible, more real-time, and easier to use, and TiDB, an elastic, cloud-native, real-time HTAP database, is exactly that kind of database.

Create a Scale-Out Hive Cluster with a Distributed, MySQL-Compatible Database

This post shows how to deploy a Hive cluster with TiDB to achieve horizontal scalability of Hive Metastore.

Heterogeneous Database Replication to TiDB

PingCAP is experienced in heterogeneous database replication. This post gives best practice tips for database migration to TiDB.

Announcing Chaos Mesh® as a CNCF Sandbox Project

We're thrilled to announce that Chaos Mesh®, our open-sourced Chaos Engineering platform for Kubernetes, is now officially accepted as a CNCF Sandbox project.

Horizontally Scaling the Hive Metastore Database by Migrating from MySQL to TiDB

This post shows how TiDB helps Zhihu eliminate their database bottleneck and horizontally scale their Hive Metastore database to meet the growing business needs.

How TiDB's HTAP Makes Truly Hybrid Workloads Possible

This post introduces the design details of the HTAP architecture of TiDB, including the real-time updatable columnar engine, the multi-Raft replication strategy, and smart selection.

Cluster Diagnostics: Troubleshoot Cluster Issues Using Only SQL Queries

TiDB 4.0 introduces cluster diagnostics, a built-in widget in TiDB Dashboard, which lets you diagnose cluster problems within a specified time range. You can summarize the diagnostic results and cluster-related load monitoring information in a diagnostic report.

A Few More Reasons Rust Compiles Slowly

This is the fourth episode of the Rust Compile Time series. It discusses some factors that cause Rust to build slow, including LLVM, compiler architecture, and linking.

Announcing TiDB as a Service, Fully-Managed TiDB Offering

TiDB Cloud Beta is released. It's the easiest, most economical, and most resilient way to unlock the full power of TiDB in the cloud, allowing the users to deploy and run TiDB clusters with just a few clicks.

Rust's Huge Compilation Units

The third episode of the Rust Compile Time series discusses why Rust's compilation units are so big and how that affects compile times.

VLDB 2020: TiDB, A Raft-based HTAP Database

PingCAP will publish a research paper "TiDB: A Raft-based HTAP Database" in the August issue of the Proceedings of the VLDB Endowment.

TiDB 4.0 GA, Gearing You Up for an Unpredictable World with a Real-Time HTAP Database

TiDB 4.0 has reached general availability. It's an elastic, real-time Hybrid Transactional/Analytical Processing (HTAP) database, and, best of all, it's now ready for the cloud.

Generics and Compile-Time in Rust

This is the second episode of the Rust Compile Time series. Brian Anderson, one of Rust's original authors, talks about monomorphization, using the TiKV project as a case study.

How We Improved TPC-C Performance by 50% and TPC-H Performance by 100%

TiDB 4.0 release greatly outperforms TiDB 3.0. Our TPC-C benchmark improved by about 50% and our TPC-H benchmark improved by about 100%.

TiDB Dashboard: Easier Troubleshooting for Distributed Databases

TiDB 4.0 introduces TiDB Dashboard, which provides various built-in widgets in a graphical interface that let you easily diagnose, monitor, and manage your clusters in one place. Now you can troubleshoot TiDB clusters more easily.

Pessimistic Locking: Better MySQL Compatibility, Fewer Rollbacks Under High Load

With improvements in stability and functionality in TiDB 4.0, we finally remove the experimental label for pessimistic locking, making it a generally available feature. See how pessimistic locking behaves in TiDB.

TiCDC: Replication Latency in Milliseconds for 100+ TB Clusters

TiDB 4.0 introduces TiCDC, an open-source feature that replicates TiDB's incremental changes to downstream platforms. It shows only millisecond replication latency for 100+ TB clusters.

Large Transactions in TiDB

This post describes how we implemented support for large transactions in TiDB 4.0.

TiDB at ZaloPay Infrastructure & Lesson Learned

As VNG's business developed, their infrastructure became an obstacle to their growth. Then they chose TiDB as the core database for their ZaloPay app and used it to store their critical data. Now, VNG has 20+ TiDB nodes in their production environment.

SQL Plan Management: Never Worry About Slow Queries Again

This post introduces how SQL Plan Management helps the optimizer automatically select efficient execution plans to prevent performance regressions caused by unexpected plan changes.

Building, Running, and Benchmarking TiKV and TiDB

This post introduces how to build and run your own TiDB or TiKV, and how to run some benchmarks on those databases.

Get a TiDB Cluster Up in Only One Minute

TiUP is a component manager that streamlines installing and configuring a TiDB cluster into a few easy commands. It helps get your cluster up and running quickly with a minimal learning curve.

Key Visualizer: Observe Distributed Databases to Discover the Unknowns

Key Visualizer is a visual diagnostic tool that makes it easier to troubleshoot distributed SQL databases. Users can observe system health, quickly find hotspots in the cluster, and gain deep insights into applications.

Simulating Clock Skew in K8s Without Affecting Other Containers on the Node

As a cloud-native chaos engineering platform, Chaos Mesh supports TimeChaos, which simulates clock skew in containers or K8s for distributed systems to test system robustness, without affecting other containers on the node.

How to Back Up and Restore a 10-TB Cluster at 1+ GB/s

Backing up or restoring a large-scale distributed SQL database is a time-consuming task. Our upcoming TiDB 4.0 release will support Backup & Restore (BR), a distributed backup and restore tool, which lets you back up and restore data at 1+ GB/s for 10-TB data.

TiDB 4.0 Preview: An Easier-to-Use, Production-Ready HTAP Database

Get a preview of TiDB 4.0. Our landmark release has production-ready HTAP capabilities, continues to solve issues for large-scale, distributed SQL databases, and is easier to use than ever.

A Peek into the Future of Database: A Unified Infrastructure to Adapt Intelligently

Today is PingCAP's 5-year birthday. In this post, Ed Huang, PingCAP's CTO, discusses about what he thinks the future holds for databases. The future of the database is about unification, adaptiveness, and intelligence.

Quickly Find Rust Program Bottlenecks Online Using a Go Tool

It can be hard to find Rust programs' performance bottlenecks online. By integrating pprof-rs in TiKV, we can use the Go tool pprof to visualize TiKV's profiling data. This helps analyze the program's performance online.

Run Your First Chaos Experiment in 10 Minutes

In this 10-minute tutorial, we will help you to quickly get started with Chaos Engineering and run your first chaos experiment with Chaos Mesh.

Choosing the Right Database for Your Applications

With so many databases on the market, choosing the right fit for your app is not an easy task. As a leading e-commerce platform in Southeast Asia and Taiwan, Shopee has an effective strategy for database selection. Learn from their experience.

TiDB Usability Challenge - Dare to Dream Bigger

TiDB Challenge Program is a series community program to bring TiDB to a new level in terms of stability, performance, and usability. With amazing improvements in performance in season 1 last year, we are now pleased to announce that season 2 is ready for you.

TiDB Celebrates 400 Contributors

The TiDB repository reaches the 400th contributor milestone. All our honors and achievements belong to the TiDB community, our beloved contributors, and distinguished committers.

Early Impressions of Go from a Rust Programmer

Nick Cameron is a long-time Rust programmer who has recently started using Go. In this post, he talks about his early impressions of Go. Read this post to learn more.

How We Reduced Multi-region Read Latency and Network Traffic by 50%

High read latency and network traffic are common issues for a multi-region architecture. At TiDB Hackathon 2019, a team won 2nd place by reducing multi-region read latency and network traffic by 50%. Read this post to learn how they did it.

Remote Work - Part 3

It's the third post in a series on remote work. In this post, Nick Cameron talks a bit about some of the practicalities, specifically around work and life balance.

Remote Work - Part 2

Nick Cameron has nearly ten-year experience of remote work. In this post, he discusses communication in remote work.

Remote Work - Part 1

Nick has been working remotely for nearly ten years. In this post, he introduces why you might benefit from remote work and some of the things that make remote work successful.

Doubling System Read Throughput with Only 26 Lines of Code

The Follower Read feature lets any follower replica in a Region serve a read request under the premise of strongly consistent reads. It reduces the load on the Raft leader and improves the read throughput of the TiDB cluster. Read this post to learn more.

The Rust Compilation Model Calamity

In this first episode of the Rust Compile time series, Brian Anderson, one of Rust's original authors, shares with you his researches and experiences with Rust compile times, using the TiKV project as a case study.

Chaos Mesh - Your Chaos Engineering Solution for System Resiliency on Kubernetes

Recently, PingCAP open-sourced Chaos Mesh, a Chaos Engineering platform that features all-around fault injection methods for complex Kubernetes systems. Find out how this testing tool can make your applications more reliable.

How TiKV Reads and Writes

This post details how TiKV, as a distributed database, stores the data contained in a write request and how it retrieves the corresponding data with consistency guaranteed.

AutoTiKV: TiKV Tuning Made Easy by Machine Learning

AutoTiKV is a machine-learning-based tuning tool that helps decrease tuning costs and make life easier for DBAs. This post shows AutoTiKV's design, its machine learning model, and the automatic tuning workflow.

Squashed Bugs, Served Hot and Fresh with Failure Rate Heatmaps

At TiDB Hackathon 2019, a team won the third prize by building a bot that helps quickly locate bugs in the code. Read this post to get more details.

How We Achieved 40 K QPS and 20+ Billion Rows of Data in a Single Scale-out Cluster

As the business developed, standalone MySQL couldn't meet JD Cloud & AI's OSS metadata storage requirements. This post introduces how TiKV empowered JD Cloud & AI to manage huge amounts of OSS metadata with a simple and horizontally scalable architecture.

NetEase Games: Why We Chose TiDB over Other MySQL-based and NewSQL Storage Solutions

As business boomed, NetEase Games met bottlenecks in database scalability and data isolation. This post introduces why they chose TiDB over other MySQL-based and NewSQL storage solutions to solve their pain points.

New Structure, New Roles - TiDB Community Upgrade!

PingCAP has upgraded and adjusted the TiDB community organization structure, with more community roles and community organizations introduced to better stimulate the vitality of the community and maintain a positive and healthy community environment.

10x Performance Improvement for Expression Evaluation Made Possible by Vectorized Execution and the Community

We vectorized 360+ built-in functions along with the TiDB community. Vectorized execution has greatly improved the expression evaluation performance. Some functions even achieved 10x performance. Here is our vectorization story.

Building a Large-scale Distributed Storage System Based on Raft

This post introduces the PingCAP team's firsthand experience in designing a large-scale distributed storage system based on the Raft consensus algorithm.

How We Compiled a Golang Database in the Browser Using WebAssembly

We compiled a Golang database (TiDB) into an in-browser database using WebAssembly (Wasm). This post introduces why and how we built an in-browser database.

TiDB in the Browser: Running a Golang Database on WebAssembly

This post introduces how you can run TiDB directly in a web browser, how it is possible, and what the limitations are.

Delivering Real-time Analytics and True HTAP by Combining Columnstore and Rowstore

TiDB is an HTAP database that targets both OLTP and OLAP scenarios. TiFlash is its extended analytical engine. This post introduces how TiFlash fuels TiDB to become a true HTAP database that lets users perform real-time analytics.

INSERT INTO tidb.hackathon_2019 VALUES ("Hack", "Fun", "TiDB Ecosystem")

This post introduces the top 3 winners at TiDB Hackathon 2019 and the cozy and friendly environment of this meeting.

Safety First! Common Safety Pitfalls in Distributed Databases Found by Jepsen Tests

This post introduces why Jepsen is important to the distributed database industry, how it works, common issues found, how we use Jepsen with TiDB, and a short summary of all problems Jepsen has found.

How TSQ Becomes a Performance Bottleneck for TiKV in AWS ARM Environment

Explore into Linux kernel to find out how TSQ becomes a performance bottleneck for TiKV in an AWS ARM environment.

Lesson Learned from Queries over 1.3 Trillion Rows of Data Within Milliseconds of Response Time at Zhihu.com

As the business boomed and massive data in applications accrued each month, Zhihu faced severe challenges in scaling the backend system. This post introduces how Zhihu managed to keep milliseconds of query response time over a large amount of data and how TiDB, an open source MySQL-compatible HTAP database, empowered Zhihu to get real-time insights into data.

Porting TiDB to ARM64 for Greater Flexibility

This article describes how PingCAP compiled and benchmarked TiDB on the Amazon Web Services (AWS) ARM64 platform.

Migrating the TiKV Rust Client from Futures 0.1 to 0.3

This post introduces Nick's experience in migrating the TiKV Rust client from Futures 0.1 to 0.3.

TiDB Operator 1.0 GA: Database Cluster Deployment and Management Made Easy with Kubernetes

Deploying transactional databases like TiDB in Kubernetes for production has always been challenging. Now, with TiDB Operator 1.0, running TiDB database clusters has never been easier.

Why Benchmarking Distributed Databases Is So Hard

Benchmarks are hard to get right, and many articles touting benchmarks are actually benchmarketing, showcasing skewed outcomes to sell products. This post introduces some of the motivations for benchmarking and the common tools, and discusses a few things to keep in mind when benchmarking.

The KOST Stack: An Open-Source Cloud-Native Hybrid Database Stack

A new infrastructure pattern is emerging called the KOST stack (Kubernetes, Operator, Spark, TiDB). This blog post introduces each component of the KOST stack and cloud-native HTAP in the wild.

TiDB 3.0 Reaches GA, Delivers Stability at Scale and Performance Boost

TiDB 3.0 is released! This blog post introduces some highlights of TiDB 3.0, including major features focused on stability, significant performance improvements in Sysbench and TPC-C benchmarks, a newly introduced component, and important features and improvements.

Distributed Systems Training in Go and Rust

PingCAP has made a talent plan for training and/or evaluating students, new employees, and new contributors to TiDB and TiKV. This blog post gives a brief overview of training courses for this plan on writing distributed systems in Go and Rust.

Best Practices for Developing Applications with TiDB

This article shows how to efficiently develop high-quality applications with TiDB, an open-source NewSQL database that supports Hybrid Transactional and Analytical Processing (HTAP) workloads and can serve as a scale-out MySQL database without manual sharding.

TiDB Passes Jepsen Test for Snapshot Isolation and Single-Key Linearizability

TiDB's first official Jepsen Test report is published. This post introduces some additional context to the test results and PingCAP's thoughts on what's next.

Percona Live Austin -- Summary and Reflection

In this post, Kevin Xu shares the PingCAP team's experience and activities at Percona Live in Austin, including several technical sharing sessions and boosting friendship with people inside and outside PingCAP.

Tutorial: Use TiDB DM to Migrate and Replicate Data from MySQL, MariaDB & Amazon Aurora

Learn to migrate and replicate data from MySQL, MariaDB and Amazon Aurora using TiDB Data Migration.

Design and Implementation of Golang Failpoints

This post introduces design and implementation of Golang Failpoint.

TiDB Binlog Tutorial

Learn to deploy TiDB Binlog with a simple TiDB cluster.

PingCAP Open-Sources TiDB Binlog to Reduce Migration Cost

Our team at PingCAP is excited to announce that we are open-sourcing our own binlog implementation, TiDB Binlog. TiDB Binlog is a tool we've been developing in-house (until now) to collect binary log data from TiDB server and provide real-time data backup and replication.

Try to Fix Two Linux Kernel Bugs While Testing TiDB Operator in K8s

This post introduces how the EE team at PingCAP tackled two Linux kernel bugs while testing TiDB Operator in K8s. PingCAP engineers also hope the K8s community, RHEL and CentOS can help fix these bugs thoroughly in the near future.

PingCAPers at the First RustCon Asia!

PingCAP has been an active member in the Rust community and one of the first production users of Rust in building TiKV. As part of being a co-organizer for the first RustCon Asia, PingCAP also contributed several topics in both keynote sessions and workshops.

Titan: A RocksDB Plugin to Reduce Write Amplification

As a RocksDB plugin for key-value separation, inspired by WiscKey, Titan is available for preview in TiDB 3.0. This post introduces the design and implementation of Titan.

Powering the Xiaomi Mobile Lifestyle with TiDB

As sales of Xiaomi smartphones continue to climb and the MIUI user base continues to grow, the Database Administration team at Xiaomi was having an increasingly hard time managing their MySQL database infrastructure until they adopted TiDB, an open source distributed hybrid transactional and analytical processing database created and supported by PingCAP. Now they also have plans to migrate additional workloads to TiDB in the future.

Introduction to Analytics Queries for the MySQL DBA

This post introduces some simple use cases of analytics queries where a MySQL DBA can expand their repertoire and answer some basic business questions by writing SQL queries with window functions.

BookMyShow.com: More Uptime, 30% Less Operational Cost with TiDB

With TiDB, BookMyShow has experienced increased uptime and availability, since TiDB automatically partitions and distributes data across the cluster. Meanwhile, operational and maintenance cost has been reduced by 30%. No engineer needs to be fully dedicated to database operations anymore. This post will introduce why BookMyShow chose TiDB over Greenplum by illustrating how TiDB solved their pain points.

TiDB Certification (Beta) is Ready

The official TiDB Certification is now available in beta and an online exam is ready for you to sign up.

Shopping on Shopee, the TiDB Way

As our business boom, our team faced severe challenges in scaling our backend system to meet the demand until we found TiDB, a MySQL-compatible NewSQL hybrid transactional and analytical processing (HTAP) database, built and supported by PingCAP. Now we can provide better service and experience for our users without worrying about our database capacity.

FOSDEM 2019 Recap: Global Technology, Local Community

In this post, Kevin Xu shares the PingCAP team's experience and activities at FOSDEM 2019, including three talks they delivered at the conference -- the first one about TiDB and its architecture, horizontal scalability and MySQL compatibility, the second about improving development posture in Rust based on the work building TiKV, and the last about Rust implementation of Prometheus.

TiDB Tools (III): TiDB Data Migration Architecture Design and Implementation Principles

TiDB Data Migration is an integrated data transfer and replication management platform that supports full data migration or incremental data replication from MySQL or MariaDB instances into a TiDB cluster. This post introduces its architecture design and implementation principles.

TiDB Tools (II): Introducing TiDB Lightning

TiDB Lightning is an open source TiDB ecosystem tool that supports high speed full-import of a large SQL dump into a TiDB cluster. This post introduces its architecture and future improvements on the roadmap.

TiDB Tools (I): TiDB Binlog Architecture Evolution and Implementation Principles

TiDB Binlog is a tool used to collect the logical changes made to a TiDB cluster and provide incremental backup and replication. This post introduces its architecture evolution and implementation principles.

TiDB 3.0 Beta: Stability at Scale

This post introduces the key new features of TiDB 3.0 Beta.

PingCAP 2018 Year in Review

Thank you for a great 2018!

5 Key Differences Between MySQL and TiDB for Scaling in the Cloud

This post introduces the top five key differences between TiDB and MySQL.

TiDB, the Key to a Better Life for Meituan-Dianping's 290 Million Monthly Users

As our business grew quickly, we were overwhelmed trying to tackle the mounting data until we found TiDB, a MySQL-compatible NewSQL hybrid transactional and analytical processing (HTAP) database, built and supported by PingCAP. Now we can harness our data with more confidence than ever before and provide better services for our users to enjoy a better life.

TiDB 2.1 GA: Battle-Tested to Handle an Unpredictable World

PingCAP, a leading distributed database company that created the popular MySQL-compatible cloud-native NewSQL database TiDB, announces that TiDB 2.1 is ready for General Availability.

How TiKV Uses "Lease Read" to Guarantee High Performances, Strong Consistency and Linearizability

This post discusses Raft Log Read, `ReadIndex` Read, and Lease Read, and why TiKV adopts the Lease Read approach.

TiDB: Architecture and Use Cases of A Cloud-Native NewSQL Database

This post provides a macro-level overview of TiDB and is the main reference content for readers to orient and dig deeper into other TiDB subjects.

Always Fun, Always On: How TiDB Helps iQiyi Deliver Streaming Videos

As our business grew exponentially, we were overwhelmed trying to handle the mounting data until we found TiDB, a MySQL-compatible NewSQL hybrid transactional and analytical processing (HTAP) database, built and supported by PingCAP. Now we no longer worry about data volume and can bring high-quality entertainment services to our users with more confidence than before.

Launching TiDB Academy, First Course-“Distributed Database with TiDB for MySQL DBAs”

Today, we are excited to launch TiDB Academy, a series of technical training courses and certifications on TiDB and distributed databases in general, taught by our senior technical team.

PingCAP Raises $50 Million in Series C Round, Sets Eyes on Global Expansion, Cross-Cloud Offering, and More Core Technology Investment

PingCAP, a leading distributed database company that created the popular cloud-native NewSQL database TiDB, announces a $50 million Series C funding round led by FOSUN and Morningside Venture Capital. All previous investors—China Growth Capital, Yunqi Partners, Matrix Partners China, and others—have also participated in this round. PingCAP plans to use this new capital to expand the TiDB ecosystem globally, build cross-cloud product offering, and invest in innovation of its core technology.

TiDB Reaches the 200 Contributors Milestone

TiDB recently added its 200th contributor. As CEO and co-founder of PingCAP who began building TiDB three years ago, I would like to thank the entire TiDB community for helping us reach this important milestone!

9 Why's to Ask When Evaluating a Distributed Database

Since PingCAP started building TiDB, it has encountered countless challenges, pitfalls, and critical design choices. Over the past three years, the PingCAP team has answered many questions from the users. Ed Huang, the CTO of PingCAP, has summarized these questions into “9 why's” that every engineer may ask when looking at a distributed database and gives his answers to these questions in this post to help make decision-making of engineers a bit easier.

Managing the Surging Data Volume of a Fast-Growing Marketplace with TiDB

With our fast-growing business, the surging data volume posed a serious challenge to our backend system and put the operations team under great pressure. How to tackle these challenges became a thorny problem until we found TiDB, a MySQL compatible distributed hybrid transactional and analytical processing (HTAP) database, built and supported by PingCAP. Finally, we do not have to worry about scaling databases and can focus on building better applications for our users.

Landing Your First Rust Pull Request in TiKV

This guide is intended to show how you can land your first Pull Request (PR) in Rust to contribute to TiKV in less than 30 minutes. But before we do that, here's some helpful background.

TiSpark: More Data Insights, Less ETL

The motivation behind building TiSpark was to enable real-time analytics on TiDB without the delay and challenges of ETL. Extract, transform, and load (ETL)--a process to extract data from operational databases, transform that data, then load it into a database designed to supporting analytics--has been one of the most complex, tedious, error-prone, and therefore disliked tasks for many data engineers. However, it was a necessary evil to make data useful, because there hasn't been good solutions on the market to render ETL obsolete--until now.

The Hybrid Database Capturing Perishable Insights at Yiguo

As the largest B2C fresh produce online marketplace in China, serving close to 5 million users and 1,000+ enterprise customers, Yiguo.com uses TiDB to capture real-time insights from fresh transactional data to make timely and accurate business decisions.

How To Spin Up an HTAP Database in 5 Minutes with TiDB + TiSpark

In this 5-minute tutorial for beginners, we will show you how to spin up a standard TiDB cluster using Docker Compose on your local computer, so you can get a taste of its hybrid power, before using it for work or your own project in production.

Implement Raft in Rust

As an open-source distributed scalable HTAP database, TiDB uses the Raft Consensus Algorithm in its distributed transactional key-value storage engine, TiKV, to ensure data consistency, auto-failover, and fault tolerance. TiDB has thus far been used by more than 200 companies in their production environments in a wide range of industries, from e-commerce and food delivery, to fintech, media, gaming, and travel.

TiDB 2.0 is Ready - Faster, Smarter, and Battle-Tested

TiDB 2.0 is released! We absorbed insights and feedbacks from our customers, listened to requests and issues from our community, and reflected internally on our ultimate vision of building a distributed hybrid transactional and analytical processing database that scales itself, heals itself, and lives in the cloud.

From Chaos to Order -- Tools and Techniques for Testing TiDB, A Distributed NewSQL Database

As an open source distributed NewSQL Hybrid Transactional/Analytical Processing (HTAP) database, TiDB contains the most important asset of our customers--their data. One of the fundamental and foremost requirements of our system is to be fault-tolerant. But how do you ensure fault tolerance in a distributed database? This article covers the top fault injection tools and techniques in Chaos Engineering, as well as how to execute Chaos practices in TiDB.

Blitzscaling the Largest Dockless Bikesharing Platform with TiDB's Help

Mobike has been using the TiDB database in the production environment since early 2017. Now they have deployed TiDB in multiple clusters with close to 100 nodes, handling dozens of TBs of data for different application scenarios. This post will provide a deep dive on why Mobike chose TiDB over MySQL and its sharding solutions by illustrating how TiDB solves their pain points.

How to do Performance Tuning on TiDB, A Distributed NewSQL Database

Doing performance tuning on distributed systems is no joking matter. It's much more complicated than on a single node server, and bottlenecks can pop up anywhere, from system resources in a single node or subcomponent, to cooperation between nodes, to even network bandwidth. Performance tuning is a practice that aims to find these bottlenecks and address them, in order to reveal more bottlenecks and address them as well, until the system reaches an optimal performance level. In this article, I will share some best practices on how to tune "write" operations in TiDB to achieve maximum performance.

Bringing TiKV to Rust Devroom at FOSDEM 2018

At the crack of dawn on February 1, I landed in Brussels, Belgium, for the first time in my life. The goal of my trip wasn't to taste the local cuisine, tour world-famous museums, or grab a pint of the local brew. It was to deliver a talk three days later at FOSDEM 2018 Rust Devroom about our experience at PingCAP using Rust to build TiKV, a distributed transactional Key-Value storage engine.

TiDB DevCon 2018 Recap - News, Latest Development, and Roadmap

On January 20th, 2018, more than 200 coders, hackers, and techies streamed into Garage Café, a chic coffee shop in the heart of Beijing's techhub, Zhongguancun. They were there to be part of TiDB DevCon 2018, a technology party for the developers, by the developers!

2017 Reflection and Gratitude

Thank you all, our beloved contributors, customers, and partners, for an amazing 2017! Hello, 2018!

Tick or Tock? Keeping Time and Order in Distributed Databases

At re:Invent 2017, Amazon Web Services (AWS) announced Amazon Time Sync Service which is a highly accurate and reliable time reference that is natively accessible from Amazon EC2 instances. It is much like the Google TrueTime which was published in 2012. Why do Google and AWS both want to make efforts to provide global time service? Is there any inspiration for building distributed database? This topic is important to think about.

PingCAP Plants its Seed in Silicon Valley

PingCAP, a cutting-edge distributed Hybrid Transactional/Analytical Processing (HTAP) database company, is excited to announce the opening of its Silicon Valley office, located at the GSV Labs in Redwood City, California.

A TiKV Source Code Walkthrough – Raft Optimization

Paxos or Raft is frequently used to ensure data consistency in the distributed computing area. But Paxos is known for its complexity and is rather difficult to understand while Raft is very simple. Therefore, a lot of emerging databases tend to use Raft as the consensus algorithm at its bottom layer. TiKV is no exception.

PingCAP Launches TiDB 1.0

TiDB is compatible with MySQL, strong consistent and highly available.

Scale the Relational Database with NewSQL

This is the speech Li SHEN gave at the 3rd NEXTCON.

Why did we choose Rust over Golang or C/C++ to develop TiKV?

Every developer has his/her favorite programming language. For the TiKV team members, it's Rust.

RocksDB in TiKV

This is the speech Siddon Tang gave at the RocksDB meetup on August 28, 2017.

Futures and gRPC in Rust

This is the speech Siddon Tang gave at Bay Area Rust Meetup August 2017.

How We Found a Data Corruption Bug in RocksDB

Data was corrupted. A cluster panicked. The crime scene was compromised. What happened? Detective Huang went all lengths to locate the criminal and solved it once and for all.

When TiDB Meets Jepsen

What happens when TiDB meets Jepsen?

The Design and Implementation of Multi-raft

The goal of TiKV is to support 100 TB+ data and it is impossible for one Raft group to make it, we need to use multiple Raft groups, which is called Multi-raft.

How TiDB tackles fast data growth and complex queries for yuanfudao.com

This document is a case study that details the reasons why yuanfudao.com chose TiDB as its backend database solution to tackle their fast data growth and complex queries.

A TiKV Source Code Walkthrough - Raft in TiKV

TiKV uses the Raft algorithm to implement the strong consistency of data in a distributed environment. This blog introduces the details how Raft is implemented.

TiDB Best Practices

This article summarizes some best practices in using TiDB, mainly including SQL usage, OLAP/OLTP optimization techniques and especially TiDB's exclusive optimization switches.

TiDB Internal (III) - Scheduling

This is the third one of three blogs to introduce TiDB internal.

TiDB Internal (II) - Computing

This is the second one of three blogs to introduce TiDB internal.

TiDB Internal (I) - Data Storage

This is the first one of three blogs to introduce TiDB internal.

Refactoring the Built-in Functions in TiDB

In order to accelerate expression evaluation, we recently refactored its framework. This tutorial will show you how to use the new computational framework to rewrite or add a built-in function in TiDB.

Rust in TiKV

This is the speech Siddon Tang gave at the 1st Rust Meetup in Beijing on April 16, 2017.

A Brief Introduction of TiDB

This is the speech Edward Huang gave at Percona Live Open Source Database Conference 2017.

Migration from MySQL to TiDB to handle tens of millions of rows of data per day

This document is a use case that details the performance of MySQL and TiDB with tens of millions of rows of data per day.

About the TiDB Source Code

The target audience of this document is the contributors in the TiDB community. The document aims to help them understand the TiDB project. It covers the system architecture, the code structure, and the execution process.

Adding Built-in Functions

TiDB code is updated and the procedure of adding built-in functions is greatly simplified. This document describes how to add built-in functions to TiDB.

Subquery Optimization in TiDB

Subquery optimization, especially rewriting the correlated subquery, is a very difficult part in SQL query optimization. To be compatible with MySQL, TiDB enables users to write subqueries anywhere they want. For those subqueries that are not correlated, which are also called uncorrelated subqueries, TiDB evaluates in advance; for those correlated subqueries, TiDB removes the correlations as much as possible. For example, TiDB can rewrite a correlated subquery to `SemiJoin`. This article is focused on introducing the correlated subquery optimization methods in TiDB.

MVCC in TiKV

This document gives an overview of MVCC implementation in TiKV.

Travelling Back in Time and Reclaiming the Lost Treasures

This document introduces the History Read feature in TiDB.

A Deep Dive into TiKV

This document introduces how TiKV works as a Key-Value database.

How we build TiDB

This is the speech Max Liu gave at Percona Live Open Source Database Conference 2016.

- Show More Articles -