Pages

Friday, October 27, 2017

Database benchmarks

Standard OLTP Benchmarks

  • TPC-C/E
  • SPECjEnterprise2010
Open source / Free OLTP Benchmarks
  • HammerDB - Workload provides TPC-C/H like transactions, not to be directl compared with TPC-C/H. Use New Orders Per Min (NOPM) metric to compare different DB vendors.. It supports Oracle, DB2, SQL server, MySQL/MariaDB, Aurora, Redshift, etc. Requires GUI.
  • YCSB - Workloads are mostly SELECT/UPD/INS combinations for variious data stores:
    A—Update heavy Read: 50% Update: 50% - Zipfian - Session store recording recent actions in a user session
    B—Read heavy Read: 95% Update: 5% - Zipfian - Photo tagging; add a tag is an update, but most operations are to read tags
    C—Read only Read: 100% - Zipfian - User profile cache, where profiles are constructed elsewhere (e.g., Hadoop)
    D—Read latest Read: 95% Insert: 5% -  Latest - User status updates; people want to read the latest statuses 
    E—Short ranges Scan: 95% Insert: 5% - Zipfian/Uniform (Workload E uses the Zipfian distribution to choose the first key in the range, and the Uniform distribution to choose the number of records to scan) - Threaded conversations, where each scan is for the posts in a given thread (assumed to be clustered by thread id)

    JDBC version is available that can be used for RDBMS like Oracle, MySQL, etc.
  • Sysbench - Includes oltp (read/writes), fileio, cpu, memory, threads, mutex workloads. 
  • Swingbench - Includes 6 workloads - OrderEntry (JDBC or PL/SQL variants), SalesHistory, TPC-DS Like, JSON, CallingCircle and StressTest
  • Orion Calibration - Tool for Oracle DB, simulating I/O workloads using same I/O software stack as Oracle. 
  • iobench - Java benchmark targeting EC2 and EBS