Stress Testing Servers with stress-ng, sysbench, fio

After upgrading a server from Intel Xeon to AMD EPYC, we discovered that OLTP benchmark performance dropped by 15%. The cause was thermal throttling: the old cooler couldn't handle the new TDP, so the processor reduced frequency. Stress testing helped identify the problem before going into production. Such scenarios are not uncommon: an underloaded system can hide cooling defects, memory instability, or suboptimal disk settings. In this article, we'll show how to perform comprehensive server validation using stress-ng, sysbench, and fio to obtain objective metrics for decision-making.

Which tools and for what?

What is bogo-ops/s and should we rely on it?

bogo-ops/s is an internal metric of stress-ng showing the number of operations performed per second. It is useful for relative comparison of two configurations (e.g., before and after replacing thermal paste), but does not provide an absolute standard. For precise measurements, use sysbench.

How does stress-ng detect thermal throttling?

stress-ng is a Swiss Army knife of stress testing: over 300 stressor types. According to stress-ng author Colin Ian King, the tool has over 300 stressors. Unlike burnP6, it can load specific CPU instructions, cause memory contention, and check thermal throttling. In fact, stress-ng can load the CPU up to 3x more aggressively than synthetics that don't strain the memory controller.

# Stress all CPU cores for 60 seconds
stress-ng --cpu 0 --cpu-method matrixprod --timeout 60s --metrics-brief

# CPU + memory simultaneously
stress-ng --cpu 4 --vm 2 --vm-bytes 2G --timeout 120s

# Thermal throttling test (+ temperature monitoring)
stress-ng --cpu 0 --timeout 300s & watch -n1 "sensors | grep 'Core|Package'"

# Stressor to test OS scheduler
stress-ng --sched 4 --timeout 60s --metrics-brief

--cpu 0 uses all available cores. --cpu-method matrixprod performs matrix operations, heavily loading FPU and caches. The --metrics-brief flag outputs bogo-ops/s at the end—not an absolute metric, but convenient for configuration comparison.

sysbench: OLTP workload and memory benchmark

sysbench has historically been used for MySQL/PostgreSQL benchmarking, but its CPU and memory tests are also useful:

# CPU test (prime number calculation)
sysbench cpu --cpu-max-prime=20000 --threads=8 run

# Memory bandwidth test
sysbench memory --memory-block-size=1K --memory-total-size=10G \
  --memory-oper=write --threads=8 run

# File system random read/write test
sysbench fileio --file-total-size=10G prepare
sysbench fileio --file-total-size=10G --file-test-mode=rndrw \
  --time=120 --max-requests=0 --threads=4 run
sysbench fileio --file-total-size=10G cleanup

sysbench memory output shows memory throughput in MiB/s. Typical values: DDR4-3200 dual-channel theoretically gives ~40-50 GB/s, real sysbench shows 20-30 GB/s depending on access pattern.

fio: serious disk subsystem testing

fio is the standard for storage benchmarking. There are many parameters, but the key ones:

# nvme-test.fio
[global]
ioengine=libaio
direct=1
numjobs=4
runtime=60
group_reporting

[seq-read]
rw=read
bs=1M
size=10G
filename=/dev/nvme0n1

[rand-4k-read]
rw=randread
bs=4k
size=4G
iodepth=32
filename=/dev/nvme0n1

[rand-4k-write]
rw=randwrite
bs=4k
size=4G
iodepth=32
filename=/dev/nvme0n1

fio nvme-test.fio 

Critical metrics in fio output:

• IOPS: for random 4K, good NVMe SSD gives 500k+ IOPS, SATA SSD ~100k, HDD 100-200
• BW (bandwidth): for sequential 1M, NVMe 3-7 GB/s, SATA SSD ~500 MB/s
• lat (latency): avg and p99.9, for NVMe random read p99.9 should be <200µs, for SATA SSD <1ms

iodepth=32 sets queue depth. For SSD/NVMe increase to 64-128 for maximum throughput. direct=1 bypasses page cache, testing hardware rather than OS cache.

Additional fio parameters for deep analysis: add --lat_percentiles=1 and --percentile_list=50:90:99:99.9 to measure high-percentile latencies. Also useful: --write_lat_log for detailed logging.

How to perform comprehensive stress testing before production?

A typical acceptance scenario involves three phases:

#!/bin/bash
# Phase 1: warm-up for 10 minutes under full load
stress-ng --cpu 0 --vm 4 --vm-bytes 1G --io 4 --timeout 600s \
  --metrics-brief 2>&1 | tee /tmp/stress-report.txt

# Phase 2: disk I/O benchmark
fio --name=acceptance-test --ioengine=libaio --direct=1 \
  --rw=randrw --rwmixread=70 --bs=4k --numjobs=4 \
  --size=8G --runtime=120 --time_based \
  --filename=/var/lib/test-fio.dat \
  --group_reporting 2>&1 | tee /tmp/fio-report.txt

# Phase 3: memory bandwidth
sysbench memory --memory-block-size=1M --memory-total-size=50G \
  --threads=$(nproc) run 2>&1 | tee /tmp/sysbench-report.txt

echo "Acceptance test complete. Reports in /tmp/"

During Phase 1, monitor CPU temperature (sensors), throttling (dmesg | grep -i throttl), and core frequency (cpupower frequency-info). If the server throttles at 80% load, a cooling issue exists—better found now.

What's included in the stress testing service

• Selecting scenarios tailored to your hardware: CPU, RAM, disk, network
• Preparing and executing tests with automatic metric collection
• Analyzing results: identifying bottlenecks, comparing to baseline
• Report with graphs (temperature, frequencies, IOPS, latency)
• Recommendations for configuration improvements (if needed)
• Training your team to run tests independently

Typical metrics for different storage types

Timelines and cost

One-time testing of a single server with a ready scenario takes 2-4 hours, with costs starting from $500. Developing test suites and automating acceptance testing for a fleet of servers takes 1 to 2 days. Clients typically save up to 40% on hardware replacement costs by identifying issues early. With over 5 years of experience and 30+ infrastructure projects, we have performed stress testing for high-load systems under Kubernetes and DBMS. Contact us to discuss your scenario—we'll assess the project and propose a plan.

Why order stress testing from us?

Experience with different hardware types: NVMe, SATA SSD, HDD, NUMA, ARM. We automate the process so that regression tests can be repeated. We guarantee you'll get an objective performance picture without OS cache masking. We provide a detailed report and consultation on results. Get a consultation—write to us, and we'll select the optimal set of tests for your task.