3.0. Tuning Contest Guide – System Software Lab.

1. Workload (Benchmark): YCSB-cpp

The goal of the Yahoo Cloud Serving Benchmark (YCSB) project is to develop a framework and common set of workloads for evaluating the performance of different “key-value” and “cloud” serving stores.

Contest Workload

YCSB Workload
- Record Count = 2,000,000
- Operation Count = 2,000,000
- Workload file
Load A -> Run A -> Run B -> Run D

2. Tuning Options

Team	write_buffer_size	max_file_size	compression	block_cache	filter_policy	block_size	block_restart_interval
WAL/Manifest	128MB	64MB	snappy	128MB	10	8KB	16
Memtable	32MB	16MB	snappy	32MB	10	8KB	16
Compaction	32MB	4MB	snappy	8MB	10	2KB	16
SSTable	64MB	32MB	snappy	13.75MB	10	2MB	4
Bloom Filter	128MB	64MB	snappy	64MB	9	8KB	4
Cache	47.68MB	4MB	snappy	40MB	10	8KB	32

3. Result

Avg. Throughput (ops/sec)	WAL/Manifest	Memtable	Compaction	SSTable	Bloom Filter	Cache
Load A	71065	34800	23336	57734	71972	35522
Run A	73928	58610	40784	63753	72253	47830
Run B	131709	130014	114509	147205	136455	125638
Run D	191939	200841	177168	207546	194346	193820

4. Ranking & Report

Total Rank	Team	Load A	Run A	Run B	Run D	Report
1	SSTable	3	3	1	1	Link (KOR)
1	Bloom Filter	1	2	2	3	Link (KOR)
3	WAL/Manifest	2	1	3	5	Link (KOR)
4	Memtable	5	4	4	2	Link (KOR)
5	Cache	4	5	5	4	Link (KOR)
6	Compaction	6	6	6	6	Link (ENG)

5. Notice

Study the db options and their relationships.
Analyze workloads such as key/value size and key/operations distribution.
Hypothesize the best option set and verify it by experiment.
Submit your best option set to the assistant by e-mail.
- Send your YCSB-cpp/leveldb/leveldb.properties to koreachoi96@gmail.com until Monday, 15 August 2022, 9:00 AM
Write a report about your hypothesis, experiment, and final decision in markdown format.
- Pull request at tuning directory until Tuesday, 16 August 2022, 2:00 PM
- Title: [Tuning]team__report.md
Check your ranking at Tuesday, 16 August 2022, 4:00 PM

Please refer to LevelDB options.h and RocksDB tuning guide before you start!

6. LevelDB options and restrictions

Modify YCSB-cpp/leveldb/leveldb.properties like below.

# YCSB-cpp/leveldb/leveldb.properties
# ---------------------------Restriction----------------------------
#  max_memory size (= write_buffer_size + block_cache_size) <= 1024 * 1024 * 1024 (1GB)
#  max_file_size <= 1024 * 1024 * 1024 (1GB)
leveldb.max_open_files=10000

# -------------LevelDB Options, Tune them!------------------
leveldb.dbname=/tmp/ycsb-leveldb
leveldb.format=single
leveldb.destroy=false

leveldb.write_buffer_size=2097152
leveldb.max_file_size=4194304
leveldb.compression=snappy 
leveldb.cache_size=4194304
leveldb.filter_bits=10
leveldb.block_size=4096
leveldb.block_restart_interval=16

7. Install YCSB-cpp

Install and build leveldb in release mode
git clone https://github.com/ls4154/YCSB-cpp.git

Modify config section in Makefile

#---------------------build config-------------------------
DEBUG_BUILD ?= 0
# put your leveldb directory
EXTRA_CXXFLAGS ?= -I/example/leveldb/include
EXTRA_LDFLAGS ?= -L/example/leveldb/build -lsnappy

BIND_LEVELDB ?= 1
BIND_ROCKSDB ?= 0 
BIND_LMDB ?= 0

make
Modify YCSB-cpp/leveldb/leveldb.properties

Run benchmarks

# Command
./ycsb -load -db leveldb -P workloads/workloada -P leveldb/leveldb.properties -s
./ycsb -run -db leveldb -P workloads/workloada -P leveldb/leveldb.properties -s
./ycsb -run -db leveldb -P workloads/workloadb -P leveldb/leveldb.properties -s
./ycsb -run -db leveldb -P workloads/workloadd -P leveldb/leveldb.properties -s

You can make copies of leveldb.properties and use them for benchmarks.

# Command
./ycsb -load -db leveldb -P workloads/workloada -P leveldb/leveldb.properties1 -s
./ycsb -load -db leveldb -P workloads/workloada -P leveldb/leveldb.properties2 -s

8. Tuning Enviornment

DKU Linux Server

System	Specification
CPU
Memory
Storage
Linux
Ubuntu