Weekly update (August 13 ~ August 21, 2016)

Last week, we landed 26 PRs in the TiDB repositories and 15 PRs in the TiKV repositories.

Notable changes to TiDB

• Upgrade the query optimizer.
• Upgrade the lexer.
• Replace golang protobuf with gogo protobuf.
• Optimize the distributed executor.
• Repair the Time and Decimal types to improve the compatibility with MySQL.
• Support the Set names binary statement.
• Support Covering Index.
• Optimize the table scanning when the condition is false constant.
• Fix several bugs.

Notable changes to TiKV

• Add the leader lease read support for better performance, see benchmark.
• Use delete_file_in_range from RocksDB to destroy Regions quickly to avoid blocking Raft storage threads.
• Support GC for obsolete data versions.
• Coprocessor supports covering index for Select Where and the aggregation operations.
• Check whether the key is already rolled back when prewrite to fix a transaction bug.
• Support the --log-file flag to redirect log to the log file.

Notable changes to Placement Driver

• Refine the join flag to support multiple join scenarios.
• Use unix socket in test to avoid the “Address Already in Use” error.
• Redirect requests to the Leader if the current member is a Follower.

Benchmark

Use sysbench to benchmark leader lease read and previous Raft quorum read in 3-node TiKV.

Insert

# Prepare data
sysbench --test=./lua-tests/db/oltp.lua --mysql-host=${host} --mysql-port=${port} \
 --mysql-user=${user} --mysql-password=${password} --oltp-tables-count=$1 \
 --oltp-table-size=5120000 --rand-init=on prepare

# Run benchmark
sysbench --test=./lua-tests/db/insert.lua --mysql-host=${host} --mysql-port=${port} \
 --mysql-user=${user} --mysql-password=${password} --oltp-tables-count=1 \
 --oltp-table-size= 5120000 --num-threads=${threads} --report-interval=60 \
 --max-requests=1280000 --percentile=99 run

As we can see, the qps is increased by about 70%, and the latency is decreased by about 40%.

Select

# Prepare data
sysbench --test=./lua-tests/db/oltp.lua --mysql-host=${host} --mysql-port=${port} \
 --mysql-user=${user} --mysql-password=${password} --oltp-tables-count=1 \
 --oltp-table-size=5120000 --rand-init=on prepare

# Run benchmark
sysbench --test=./lua-tests/db/select.lua --mysql-host=${host} --mysql-port=${port} \
 --mysql-user=${user} --mysql-password=${password} --oltp-tables-count=1 \
 --oltp-table-size=5120000 --num-threads=${threads} --report-interval=60 \
 --max-requests=1280000 --percentile=99 run

As we can see, the qps is increased by about 130%, and the latency is decreased by about 50%.

New contributors

• hhkbp2