Weekly update in TiDB

Last two weeks, we landed 74 PRs in the TiDB repositories.

Added

• Support Index Nest Lookup Join operator.#2834, #2945

• Add many builtin functions: quote, is_ipv4, compress, inet_aton, format, bin, random-bytes, sin, inet_ntoa, cos, from_base64, tan, cot, to_days, timestampadd

• Check privileges for show databases/tables statement.

• Calculate distinct information in statistic module.#2947, #2966

• Add a switcher to split large inset transaction into multiple small transactions automatically.

• Add memory table INFORMATION_SCHEMA.USER_PRIVILEGES.

• Add memory table INFORMATION_SCHEMA.ENGINES.

• Support Super_priv.

• Support Top-N operator push down in optimizer.

Fixed

• Fix case-when expression and coalesce expression type inference.

• Fix a goroutine leak in tikv client.

• Fix a bug in the date_format builtin function.

• Fix goroutine leak in tikv client.

• Reset affected rows counter when retrying SQL statement.

• Recognize number literal as decimal when meet int64 overflow error.

Improved

• Refactor optimizer: Introduce TaskProfile to represent a group of physical plans. Add base phyiscal plans.

• Refactor statistic module.#2913, #2952, #2956, #2993

• Refactor coprocessor architecture: table scan and index scan operator, aggregation operator, limit operator, top-n operator

• Add sorted information for SortMergeJoin operator.

• Refactor distsql package.

• Reduce memory usage in HashJoin operator.

New Contributors

Thank you guys!

• Blame cosmos

• Zhao Yi Jun

• Bin Liu

• Michael Belenchenko

• Van

• jinhelin

Weekly update in TiKV

Last two weeks, We landed 46 PRs in the TiKV repositories.

Added

• Add rate limiter for RocksDB compaction.

• Add gRPC support for PD, see 493, 590, 1591, 1712, 1725.

• Add RocksDB SST format for snapshot.

• Show replication configuration in pd-ctl.

• Make RocksDB info log path configurable.

• Slow down balance interval increasing speed.

• Report disk space usage to PD.

• Report region write-flow rate to PD.

• Show more information for region in HTTP API.

• Output statistics regularly for pd-tso-bench.

• Avoid selecting on the same context in different Goroutines.

Fixed

• Avoid removing the leader peer directly.

• Avoid open RocksDB many times.

• Parse ALREADY_BOOTSTRAPPED error correctly, see 593, 1720.

• Use pool to store TSO request to reduce memory allocation and GC pressure.

Improved

• Abort Prewrite command if the transaction is rolled back before.

• Make PD scheduler more reliable.

• Make Raft ReadIndex more stable.

• Add many unit tests to improve the system stability.