dnf install -y unixbench lmbench sysbench iperf3dnf install -y tuned && systemctl enable --now tuned # Use tuned-adm to switch profiles in the scenarios described later.
unixbench # Standard run: 1 copy + nproc copiesunixbench -c 1 -i 3 execl spawn shell8 # Run a single subtest onlyunixbench --print-workdir # View the directory where results are stored
lmbench <program name>.lmbench lat_ctx -s 0 2 # Context switch latency (in microseconds)lmbench lat_tcp -s; lmbench lat_tcp localhost; lmbench lat_tcp -S localhost # TCP loopback latency (in microseconds)lmbench lat_proc exec # Process creation latency (in microseconds)lmbench bw_mem 1g rd # Memory read bandwidth (MB/s)
--db-driver=pgsql).sysbench cpu --threads=$(nproc) runsysbench memory --memory-block-size=1M --memory-total-size=10G runsysbench fileio --file-total-size=4G --file-test-mode=rndrw preparesysbench fileio --file-total-size=4G --file-test-mode=rndrw --time=60 runsysbench fileio --file-total-size=4G cleanup
iperf3 -s # Serveriperf3 -c <server IP> -t 60 # Client
always mode of THP may increase memory compaction overhead. TencentOS Server 4 already defaults to madvise. Verify this first:cat /sys/kernel/mm/transparent_hugepage/enabled
never. For persistence, see the "Persisting Tuning Results" section. Writing it in rc.local is not recommended.cat > /etc/sysctl.d/99-database-perf.conf << 'EOF'vm.swappiness = 5vm.dirty_ratio = 10vm.dirty_background_ratio = 3EOFsysctl -p /etc/sysctl.d/99-database-perf.conf
dnf install -y numactlnumactl --hardwarenumactl --cpunodebind=0 --membind=0 /usr/sbin/mysqld
numactl --interleave=all /usr/sbin/mysqld
shared_buffers = 16GB, approximately 8192 huge pages are required when 2MB huge pages are used. Leave a small margin:grep Hugepagesize /proc/meminfoecho "vm.nr_hugepages = 8400" > /etc/sysctl.d/99-hugepages.confsysctl -p /etc/sysctl.d/99-hugepages.confgrep HugePages_ /proc/meminfo # Verify that HugePages_Total reaches the expected value
huge_pages = on for PostgreSQL and USE_LARGE_PAGES = ONLY for Oracle.none for NVMe/SSD and mq-deadline for HDDs.# Reclaim memory from other processes first when memory is tight, and preserve the database cachesystemctl set-property mysqld.service MemoryLow=32G# Use the lowest I/O priority for background tasks such as backup and exportionice -c3 <backup command>
# MySQL OLTP mixed read/write (the script is located in /usr/share/sysbench/, and you need to create the test database first)mysql -u<user> -p -e "CREATE DATABASE sbtest"sysbench oltp_read_write --mysql-user=<user> --mysql-password=<password> \\--tables=10 --table-size=100000 preparesysbench oltp_read_write --mysql-user=<user> --mysql-password=<password> \\--tables=10 --table-size=100000 --threads=$(nproc) --time=60 run# Redis (redis-benchmark is installed with the redis package)redis-benchmark -n 100000 -c 50 -t set,get# PostgreSQL (pgbench is provided by the postgresql-contrib package)pgbench -i -s 100 <database name>pgbench -c $(nproc) -j $(nproc) -T 60 <database name>
dirty_ratio has limited effect.ionice takes effect only with the bfq and mq-deadline schedulers, and has no effect with none.vm.zone_reclaim_mode = 1. When this parameter is enabled and the local node runs out of memory, the kernel reclaims page cache from the local node first instead of allocating from other nodes, causing the cache to be cleared frequently, which is typically detrimental to databases.cat > /etc/sysctl.d/99-web-perf.conf << 'EOF'net.core.rmem_max = 16777216net.core.wmem_max = 16777216net.ipv4.tcp_rmem = 4096 131072 16777216net.ipv4.tcp_wmem = 4096 65536 16777216net.core.somaxconn = 65535net.ipv4.tcp_max_syn_backlog = 8192net.ipv4.tcp_tw_reuse = 1net.ipv4.tcp_fin_timeout = 30EOFsysctl -p /etc/sysctl.d/99-web-perf.conf
somaxconn must be used together with the listen backlog of Nginx/Tomcat, and the smaller of the two values takes effect.fs.file-max is already large enough by default on TencentOS Server 4 and does not need to be adjusted. What needs to be adjusted is the process-level limit. Services managed by systemd are configured through unit files, and /etc/security/limits.conf does not take effect for systemd services:mkdir -p /etc/systemd/system/nginx.service.dcat > /etc/systemd/system/nginx.service.d/limits.conf << 'EOF'[Service]LimitNOFILE=1048576EOFsystemctl daemon-reload && systemctl restart nginx
dnf install -y irqbalancesystemctl enable --now irqbalance
systemctl stop irqbalance # Otherwise, irqbalance will overwrite manual settings.grep eth0 /proc/interruptsecho 0-3 > /proc/irq/<IRQ number>/smp_affinity_list
# Increase the NIC send and receive ring buffer size (do not exceed the maximum value shown by ethtool -g)ethtool -g eth0ethtool -G eth0 rx 4096 tx 4096# Each line in /proc/net/softnet_stat corresponds to a CPU. If the second column (hexadecimal) keeps increasing, packets are dropped because softirqs cannot be processed in time. In this case, increase the receive queue size.cat /proc/net/softnet_statsysctl -w net.core.netdev_max_backlog=16384
ethtool -l eth0 # View the number of queuesecho ffff > /sys/class/net/eth0/queues/rx-0/rps_cpus # Set this for every rx queue, and calculate the mask based on the actual number of CPUs.sysctl -w net.core.rps_sock_flow_entries=32768echo 4096 > /sys/class/net/eth0/queues/rx-0/rps_flow_cnt # rps_sock_flow_entries / number of queues
cat > /etc/sysctl.d/99-web-ports.conf << 'EOF'net.ipv4.ip_local_port_range = 10000 65000# Reserve the local listening service ports to prevent them from being occupied by outbound connections (replace with the actual ports).net.ipv4.ip_local_reserved_ports = 8080,9000-9100EOFsysctl -p /etc/sysctl.d/99-web-ports.conf
cat > /etc/sysctl.d/99-web-keepalive.conf << 'EOF'net.ipv4.tcp_keepalive_time = 600net.ipv4.tcp_keepalive_intvl = 30net.ipv4.tcp_keepalive_probes = 3EOFsysctl -p /etc/sysctl.d/99-web-keepalive.conf
SO_KEEPALIVE set.tuned-adm profile throughput-performance.# HTTP stress test (run on another machine)dnf install -y httpd-toolsab -n 1000000 -c 1000 -k http://<server IP>/# Single-node multi-core throughput (focus on multi-copy Index)unixbench -c $(nproc) -i 3 syscall context1 pipe
tcp_tw_reuse = 1 is effective only for reusing outbound connections and does not affect inbound connections.setsockopt(SO_RCVBUF/SO_SNDBUF) or rely on automatic system tuning.tcp_tw_recycle was removed in Linux 4.12 because it can cause packet loss in NAT environments, and this parameter does not exist on TencentOS Server 4.BDP = bandwidth (bps) × RTT (s)Required window size = BDP / 8
cat > /etc/sysctl.d/99-wan-perf.conf << 'EOF'# Set the buffer upper limit to more than twice the BDP.net.core.rmem_max = 67108864net.core.wmem_max = 67108864net.ipv4.tcp_rmem = 4096 131072 67108864net.ipv4.tcp_wmem = 4096 65536 67108864net.ipv4.tcp_window_scaling = 1# After a long-lived connection becomes idle, slow start is not re-initiated, allowing faster recovery to full speed during intermittent transmission.net.ipv4.tcp_slow_start_after_idle = 0EOFsysctl -p /etc/sysctl.d/99-wan-perf.conf
cat > /etc/sysctl.d/99-bbr.conf << 'EOF'net.core.default_qdisc = fqnet.ipv4.tcp_congestion_control = bbrEOFsysctl -p /etc/sysctl.d/99-bbr.confsysctl net.ipv4.tcp_congestion_control # Confirm that the algorithm has been switched to bbr.
tcp_bbr module is not loaded by default, so it is normal that bbr is not listed in tcp_available_congestion_control. The kernel automatically loads this module when it is configured.sysctl -w net.ipv4.tcp_mtu_probing=1
iperf3 -c <peer IP> -t 60 # Single-stream bandwidthiperf3 -c <peer IP> -t 60 -R # Reverse directionss -ti dst <peer IP> # View the proportion of rwnd_limited / sndbuf_limited (not displayed when not limited)
rwnd_limited and sndbuf_limited should decrease significantly. Otherwise, the bottleneck is not related to the window.net.ipv4.tcp_mem (in pages). You can check whether the limit is nearly reached by viewing the TCP mem value in cat /proc/net/sockstat.dnf install -y tuned-profiles-cpu-partitioningvi /etc/tuned/cpu-partitioning-variables.conf# isolated_cores=2-15tuned-adm profile cpu-partitioningreboot
cpu-partitioning automatically sets the nohz_full and rcu_nocbs boot parameters and prevents irqbalance from assigning interrupts to isolated cores. If you do not use this profile and manually isolate CPUs, you need to exclude the isolated cores in /etc/sysconfig/irqbalance:# /etc/sysconfig/irqbalanceIRQBALANCE_BANNED_CPULIST=2-15
systemctl restart irqbalance.-a). Otherwise, only the main thread takes effect:taskset -a -cp 2-15 <PID>
cat /sys/class/net/eth0/device/numa_node # The node where the NIC resides. -1 indicates no NUMA information.numactl --cpunodebind=<node> --membind=<node> <business command>
ethtool -c eth0 # View current settingsethtool -C eth0 adaptive-rx off rx-usecs 0
ethtool -c.echo never > /sys/kernel/mm/transparent_hugepage/enabled (For persistence, see the "Persisting Tuning Results" section).latency-performance or network-latency profile. In KVM virtual machines, the cpuidle driver is typically none, which causes slow wake-up after vCPUs become idle. You can enable haltpoll, which is built into the TencentOS Server 4 kernel. This increases CPU usage on the host and is suitable for instances with dedicated physical cores:cat /sys/devices/system/cpu/cpuidle/current_driver # none indicates that it is not enabledgrubby --update-kernel=ALL --args="cpuidle_haltpoll.force=1"reboot
chrt -f -p 50 <PID> # SCHED_FIFO priority 50
kernel.sched_rt_runtime_us to -1. By default, 5% of CPU time is reserved for normal tasks to prevent real-time tasks from freezing the system.taskset -c 2,3 lmbench lat_ctx -s 0 2 # Context switch latencylmbench lat_tcp -s; taskset -c 2 lmbench lat_tcp localhost; lmbench lat_tcp -S localhost
tuned-adm profile throughput-performance.mkdir -p /buildmount -t tmpfs -o size=32G tmpfs /build
# CPUWeight defaults to 100. Lower it to yield resources during CPU contention. TasksMax prevents uncontrolled process growth. MemoryHigh starts throttling after the limit is exceeded.systemd-run --scope -p CPUWeight=20 -p TasksMax=8192 -p MemoryHigh=16G -- make -j$(nproc)# Or use nice/ionice to lower CPU and I/O priority.nice -n 10 ionice -c3 make -j$(nproc)
CPUQuota/CPUWeight limits for other services to reserve resources for build tasks.unixbench -c 1 -c $(nproc) -i 3 execl spawn shell1 shell8lmbench lat_proc exec
tuned-adm profile virtual-host
dnf install -y libvirt numactlnumactl --hardware # View the CPU and memory of each nodevirsh vcpupin <virtual_machine> <vCPU_ID> <physical_CPU> --configvirsh emulatorpin <virtual_machine> <physical_CPU_list> --config # QEMU's own threadsvirsh numatune <virtual_machine> --mode strict --nodeset 0 --config
<memoryBacking><hugepages/></memoryBacking>
<interface type='network'><model type='virtio'/><driver name='vhost' queues='4'/></interface>
ethtool -L eth0 combined 4 in the VM to enable all queues.cat /sys/kernel/mm/ksm/run # 0 indicates that it is not running, and no action is requiredecho 2 > /sys/kernel/mm/ksm/run # Stop KSM and split the merged pages
unixbench -c 1 -c $(nproc) -i 3sysbench cpu --threads=$(nproc) run
cat /proc/pressure/cpu /proc/pressure/memory /proc/pressure/io
some indicates the percentage of time that at least one task is stalled while waiting for the resource. full indicates the percentage of time that all non-idle tasks are stalled simultaneously. Note that the CPU full value is always 0 at the system level, so only some is relevant. avg10, avg60, and avg300 are the average values over the last 10 seconds, 60 seconds, and 300 seconds, respectively. If a metric remains consistently high, the bottleneck lies in the corresponding type of resource.dnf install -y tunedsystemctl enable --now tunedtuned-adm recommendtuned-adm profile $(tuned-adm recommend)tuned-adm activetuned-adm verify
Profile | Main Settings | Scenario |
throughput-performance | governor=performance,readahead≥4096,dirty_ratio=40,swappiness=10 | General high throughput for physical machines |
virtual-guest | Inherits throughput-performance, dirty_ratio=30, swappiness=30. | General-purpose for virtual machines |
latency-performance | governor=performance, limit C-state, dirty_ratio=10, swappiness=10. | Low latency |
network-latency | Inherits latency-performance, disables THP and NUMA auto-balancing, busy_poll=50. | Low-latency network |
hpc-compute | Inherits latency-performance, THP=always, disables NUMA auto-balancing. | HPC computing |
Parameter | TencentOS Server 4 Default Value | Recommendation |
vm.swappiness | 60 | Keep the default value unless swap usage needs to be reduced. |
vm.dirty_ratio | 20 | Keep the default value. |
net.ipv4.tcp_congestion_control | cubic | Keep the default value unless cross-region transfer is required. |
THP | madvise | Keep the default value unless never is required for database or low-latency scenarios. |
sysctl <parameter_name> to check the current values.unixbench # Compare the single-copy and multi-copy Index values before and after switching profiles
mkdir -p /etc/tuned/my-dbcat > /etc/tuned/my-db/tuned.conf << 'EOF'[main]summary=Database workload tuninginclude=throughput-performance[vm]transparent_hugepages=never[sysctl]vm.swappiness=5vm.dirty_ratio=10vm.dirty_background_ratio=3EOFtuned-adm profile my-dbtuned-adm verify
include are overwritten by the child profile.reapply_sysctl by default (see /etc/tuned/tuned-main.conf). After a profile is applied, the configurations in /etc/sysctl.d/ are reloaded, and for parameters with the same name, the values in sysctl.d take precedence. After switching to tuned for management, delete the sysctl.d files with the same names that were created for the preceding scenarios to avoid configuration inconsistencies.Common Recommendations | Status on TencentOS Server 4 | Recommended Practice |
Adjust kernel.sched_latency_ns and kernel.sched_min_granularity_ns through sysctl. | Since 5.13, these parameters have been moved from sysctl to debugfs and are for debugging purposes only. The 6.6 kernel uses the EEVDF scheduler, and min_granularity has been replaced by base_slice_ns. | No adjustment is required during service tuning. |
Set net.ipv4.tcp_tw_recycle | Removed in Linux 4.12. | Use only tcp_tw_reuse. |
Set the I/O scheduler to noop, or use the boot parameter elevator= | Under blk-mq, the corresponding scheduler is none; the elevator= boot parameter is no longer effective. | Configure through udev rules. |
vm.dirty_ratio=40 | On machines with large memory, excessive dirty page accumulation can easily cause I/O stalls during writeback. | Configure based on the scenario. |
Modify /etc/pam.d/common-session to enable pam_limits | Debian-specific path. In TencentOS Server 4, system-auth is already enabled. | Use LimitNOFILE in the systemd service. |
intel_idle.max_cstate=1 | Ineffective on AMD machines. | Use the tuned latency-performance profile. |
vm.panic_on_oom=1 | Directly restarts the entire machine upon OOM. | Use only when a high availability failover mechanism is available. |
Disk Type | Recommended Scheduler | Reason |
NVMe | none | NVMe has built-in multi-queue capabilities and does not require additional scheduling. |
SSD | none or mq-deadline | Fast random access, scheduling overhead yields no benefit. |
Mechanical disk (database/throughput-oriented) | mq-deadline | Ensures request latency upper limit with low overhead. |
Mechanical disk (multi-user mixed workload) | bfq | Allocate bandwidth fairly to prevent starvation. |
cat /sys/block/sda/queue/schedulercat > /etc/udev/rules.d/60-scheduler.rules << 'EOF'ACTION=="add|change", KERNEL=="sd[a-z]*|nvme[0-9]*n[0-9]*", ATTR{queue/rotational}=="0", ATTR{queue/scheduler}="none"ACTION=="add|change", KERNEL=="sd[a-z]*", ATTR{queue/rotational}=="1", ATTR{queue/scheduler}="mq-deadline"EOFudevadm control --reload && udevadm trigger --type=devices --action=change
vda, vdb, and so on) are usually reported as mechanical disks (rotational is 1), but the backend is mostly cloud storage. Keep the default none and do not treat them as mechanical disks.blockdev --getra /dev/sda # Unit: 512-byte sectorsblockdev --setra 8192 /dev/sda
discard mount option, which performs TRIM synchronously on every file deletion and slows down both deletion and write operations:systemctl enable --now fstrim.timer # Runs once a week by default
# su is the stripe unit size, and sw is the number of data disks (for example, 4 for a 4+1 RAID 5 configuration)mkfs.xfs -d su=256k,sw=4 /dev/md0
/dev/md*) usually detects stripe parameters automatically, while hardware RAID requires manual configuration.Tool | Purpose | Installation Command |
unixbench | Comprehensive performance benchmark test | dnf install -y unixbench |
lmbench | Microbenchmark testing (latency/bandwidth) | dnf install -y lmbench |
sysbench | CPU/memory/I/O/database stress testing | dnf install -y sysbench |
iperf3 | Network bandwidth test | dnf install -y iperf3 |
fio | Fine-grained storage I/O testing | dnf install -y fio |
perf | CPU performance analysis (function-level) | dnf install -y perf |
Esta página foi útil?
Você também pode entrar em contato com a Equipe de vendas ou Enviar um tíquete em caso de ajuda.
comentários