tencent cloud

Test Tool

The performance test in this document is conducted with the TPC-C benchmark load built in HammerDB. TPC-C is a typical OLTP workload that simulates a scenario where a wholesaler with multiple warehouses ships goods to a large number of customers. The adjustment of the number of warehouses can reflect the data size that the database can sustain in the test.

• HammerDB download address
• HammerDB user guide
• Introduction to OLTP Testing (TPROC-C derived from TPC-C)

Test Environment and Parameters

Test instance editions

The test instances are of 2008 R2 Enterprise Edition, 2012 Enterprise Edition, 2014 Enterprise Edition, 2016 Enterprise Edition, 2017 Enterprise Edition, and 2019 Enterprise Edition.

Test instance specifications

High-availability edition

The test instances of high availability edition cover all purchasable specifications, including 1-core 2 GB MEM, 1-core 4 GB MEM, 1-core 8 GB MEM, 2-core 16 GB MEM, 4-core 32 GB MEM, 8-core 64 GB MEM, 12-core 96 GB MEM, 16-core 128 GB MEM, 24-core 192 GB MEM, 32-core 256 GB MEM, 48-core 384 GB MEM, 64-core 512 GB MEM, and 90-core 720 GB MEM.

Basic edition

The test instances of basic edition cover all purchasable specifications, including 1-core 2 GB MEM, 1-core 4 GB MEM, 2-core 4 GB MEM, 2-core 8 GB MEM, 4-core 8 GB MEM, 4-core 16 GB MEM, 8-core 16 GB MEM, 8-core 32 GB MEM, 16-core 32 GB MEM, 16-core 64 GB MEM, 24-core 48 GB MEM, and 24-core 96 GB MEM.

Load generation environment

The machines on which HammerDB is installed are of the same models as the SQL Server instances, ensuring that the performance of the instances can be fully measured in the stress test.

TPC-C benchmark parameters

• Number of Warehouses = 100: Sets the number of warehouses to 100.
• Minutes of Rampup Time = 2: Sets the rampup time before the test to 2 minutes.
• Minutes Test Duration = 5: Sets the test duration to 5 minutes.

Number of virtual users

The number of virtual users is the number of concurrent connections. In this document, different numbers of concurrent connections are tested on instances of different editions with different specifications.

High-availability edition

Basic edition

Test Method

1. Prepare the TPC-C workload.

• Number of Warehouses: The number of warehouses, which will affect the size of the test database generated.
• Virtual Users to Build Schema: The number of concurrent connections when generating the load data, which cannot be larger than the number of warehouses. This value affects the efficiency of load data generation, so we recommend you make it the same as the number of CPU cores of the load generating device.
  

2. Set the test script.

• Total Transactions per User: The total number of transactions per user. We recommend you set this parameter to a higher value so as to ensure that the user will not exit due to the completion of transactions during the stress test.
• Minutes of Rampup Time: Rampup time for the stress test.
• Minutes for Test Duration: Duration of the stress test.
  

3. Set the automated test script.

• Minutes per Test in Virtual User Sequence: The interval between two automated test sessions during which the program completes various tasks such as creating virtual users, ramping up, running the test, and stopping the test. This value should be greater than the sum of "Minutes of Rampup Time" and "Minutes for Test Duration".
• Active Virtual User Sequence (Space Separated): The number of virtual users generated by each iteration of the automated test. It can be understood as the number of concurrent connections.
  

4. Select Autopilot > Autopilot in the left pane to start the test.
   
5. The test result will be output in the hammerdb.log file.
   

Test Results

Note：

• The TPM in HammerDB is obtained through the SQL Server performance counter "batch requests/sec", so the TPM actually refers to the batch requests per minute.
• The size of test data set for a instance specification is larger than the memory size of the specification.

High availability edition (local SSD)

Performance comparison of different high availability editions (local SSD)

TPM comparison of different high availability editions (local SSD)

Basic edition (premium cloud disk)

Performance comparison of different basic editions (premium cloud disk)

TPM comparison of different basic editions (premium cloud disk)

Basic editions (SSD cloud disk)

Performance comparison of different basic editions (SSD cloud disk)

TPM comparison of different basic editions (SSD cloud disk)