Performance Test Report

Last updated: 2019-11-06 16:26:18

    Testing Tool

    The performance test in this document is conducted with the TPC-C benchmark load built in HammerDB. TPC-C is a typical OLTP workload that simulates a scenario where a wholesaler with multiple warehouses ships goods to a large number of customers. The adjustment of the number of warehouses can reflect the data size that the database can sustain in the test.

    Test Environment and Parameters

    Physical machines where instances are located
    High-performance physical machines with a specification of 80 cores, 767 GB memory, and 30 TB storage capacity.

    Test instance specifications
    The test instances are of 2008 Enterprise Edition, 2012 Enterprise Edition, 2016 Enterprise Edition, and 2017 Enterprise Edition. They cover almost all purchasable specifications, including 1-core 2 GB, 1-core 4 GB, 1-core 8 GB, 2-core 16 GB, 4-core 32 GB, 8-core 64 GB, 12-core 96 GB, 16-core 128 GB, and 32-core 256 GB.

    Load generation environment
    The machines on which HammerDB is installed are of the same models as the database instances, ensuring that the performance of the SQL Server instances can be fully measured in the stress test.

    TPC-C benchmark parameters

    • Number of Warehouses = 100: Sets the number of warehouses to 100, which will generate a test database of approximately 55 GB in size.
    • Minutes of Rampup Time = 2: Sets the warm-up time before the test to 2 minutes.
    • Minutes Test Duration = 5: Sets the test duration to 5 minutes.

    Number of virtual users
    The number of virtual users is the number of concurrent connections. In this document, different numbers of concurrent connections are tested on instances with different specifications.

    Concurrent Connections 2 4 8 16 32 64 128 256 512 1,024
    1-core 2 GB - -
    1-core 4 GB - -
    1-core 8 GB - -
    2-core 16 GB -
    4-core 32 GB -
    8-core 64 GB -
    12-core 96 GB
    16-core 128 GB
    32-core 256 GB

    Testing Method

    1. Prepare the TPC-C workload.
      • Number of Warehouses: The number of warehouses, which will affect the size of the test database generated.
      • Virtual Users to Build Schema: The number of concurrent connections when generating the load data, which cannot be larger than the number of warehouses. This value affects the efficiency of load data generation, so it is recommended to be the same as the number of CPU cores of the load generating device.
    2. Set the test script.
      • Total Transactions per User: The total number of transactions per user. You are recommended to set this parameter to a higher value so as to ensure that the user will not exit due to the completion of transactions during the stress test.
      • Minutes of Rampup Time: Warm-up time for the stress test.
      • Minutes for Test Duration: Duration of the stress test.
    3. Set the automated test script.
      • Minutes per Test in Virtual User Sequence: The interval between two automated test sessions during which the program completes various tasks such as creating virtual users, warming up, running the test, and stopping the test. This value should be greater than that of “Minutes of Rampup Time and Minutes for Test Duration”.
      • Active Virtual User Sequence (Space Separated): The number of virtual users generated by each iteration of the automated test. It can be understood as the number of concurrent connections.
    4. Select Autopilot > Autopilot in the left pane to start the test.
    5. The test result will be output in the hammerdb.log file.

    Test Results

    The TPM in HammerDB is obtained through the SQL Server performance counter "batch requests/sec", so the TPM actually refers to the batch requests per minute.

    • SQL Server 2008 Enterprise Edition
    Instance Specification Storage Capacity Data Set Concurrent Connections TPM
    1-core 2 GB 300 GB 55 GB 256 279,798
    1-core 4 GB 300 GB 55 GB 256 284,680
    1-core 8 GB 300 GB 55 GB 256 269,039
    2-core 16 GB 800 GB 55 GB 256 368,366
    4-core 32 GB 800 GB 55 GB 256 657,641
    8-core 64 GB 1,500 GB 55 GB 256 1,164,062
    12-core 96 GB 1,500 GB 55 GB 1024 1,348,121
    16-core 128 GB 2,000 GB 55 GB 1024 1,357,678
    32-core 256 GB 3,000 GB 55 GB 1024 1,401,600
    • SQL Server 2012 Enterprise Edition
    Instance Specification Storage Capacity Data Set Concurrent Connections TPM
    1-core 2 GB 300 GB 55 GB 256 229,854
    1-core 4 GB 300 GB 55 GB 256 234,401
    1-core 8 GB 300 GB 55 GB 256 236,773
    2-core 16 GB 800 GB 55 GB 256 333,797
    4-core 32 GB 800 GB 55 GB 256 608,801
    8-core 64 GB 1,500 GB 55 GB 256 1,020,500
    12-core 96 GB 1,500 GB 55 GB 1024 1,266,868
    16-core 128 GB 2,000 GB 55 GB 1024 1,385,158
    32-core 256 GB 3,000 GB 55 GB 1024 1,526,762
    • SQL Server 2016 Enterprise Edition
    Instance Specification Storage Capacity Data Set Concurrent Connections TPM
    1-core 2 GB 300 GB 55 GB 256 219,142
    1-core 4 GB 300 GB 55 GB 256 222,796
    1-core 8 GB 300 GB 55 GB 256 219,676
    2-core 16 GB 800 GB 55 GB 256 336,843
    4-core 32 GB 800 GB 55 GB 256 665,065
    8-core 64 GB 1,500 GB 55 GB 256 1,070,826
    12-core 96 GB 1,500 GB 55 GB 1024 1,337,473
    16-core 128 GB 2,000 GB 55 GB 1024 1,705,660
    32-core 256 GB 3,000 GB 55 GB 1024 2,280,252
    • SQL Server 2017 Enterprise Edition
    Instance Specification Storage Capacity Data Set Concurrent Connections TPM
    1-core 2 GB 300 GB 55 GB 256 201,851
    1-core 4 GB 300 GB 55 GB 256 202,510
    1-core 8 GB 300 GB 55 GB 256 208,685
    2-core 16 GB 800 GB 55 GB 256 331,650
    4-core 32 GB 800 GB 55 GB 256 625,370
    8-core 64 GB 1,500 GB 55 GB 256 1,102,296
    12-core 96 GB 1,500 GB 55 GB 1024 1,325,010
    16-core 128 GB 2,000 GB 55 GB 1024 1,716,818
    32-core 256 GB 3,000 GB 55 GB 1024 2,520,856

    Was this page helpful?

    Was this page helpful?

    • Not at all
    • Not very helpful
    • Somewhat helpful
    • Very helpful
    • Extremely helpful
    Send Feedback
    Help