Why Does an NVMe SSD Throttle During Small-File Backups but Not One Large Local Copy?

An NVMe SSD can slow during small-file backups because metadata, random writes, sync barriers, and CPU work create a different bottleneck than one large copy.

The word “throttle” is often used for any speed drop, but thermal throttling is only one possibility. Small-file backup jobs repeatedly create directory entries, update timestamps and permissions, open and close files, calculate checksums, compress or encrypt data, and commit metadata. The NVMe may show lower MB/s while processing more operations per second and higher latency. Compare equal-size local workloads before changing cooling or replacing the drive.

Prove the Difference With Equal-Size Local Tests

Create one large file and a directory tree of small files with the same total payload. Copy each locally to the same NVMe destination while recording elapsed time, IOPS, latency, CPU use, and temperature.

Fio defines separate sequential and random I/O modes, allowing the test to isolate workload shape instead of comparing unrelated backup runs.

If the large local copy stays fast while small-file creation slows, the PCIe link and sequential NAND path are probably available. Continue with metadata, queue latency, cache, CPU, and heat.

Measure Metadata Operations Instead of Only MB/s

Track files per second, creates, closes, renames, directory updates, ACL writes, extended attributes, and sync operations. Compare these with payload throughput.

Red Hat describes small-file storage as metadata-intensive, explaining why thousands of small files can produce low MB/s without reaching the NVMe’s sequential bandwidth limit.

A backup can therefore be busy and healthy while displaying a disappointing byte rate. Judge it by completed files, latency, and restore requirements as well as throughput.

Check Whether Cache Cleanup Increases Write Latency

Record free space, written bytes, sustained latency, and whether performance recovers after an idle period. Compare a freshly trimmed test area with the normal backup destination only when safe.

Crucial explains that active garbage collection can affect SSD speed, especially when the device has little idle time or reusable space.

Small random writes can require more internal movement than one sequential stream. Maintain free-space headroom and avoid assuming the advertised sequential rate applies to steady-state small writes.

-15% OFF

ZimaBoard 2 - Mini Home Server for Your Big Idea

Single board computer zimaboard2

$339.00 - $519.00

Buy Now

Inspect Queue Latency and Device Saturation

Capture average request latency, queue depth, utilization, read/write rates, and CPU I/O wait during both tests. Look for high latency with modest MB/s.

The iostat report helps show when the device is saturated by many small requests even though aggregate throughput remains far below sequential specifications.

Increasing backup concurrency can improve a lightly loaded device but worsen an already saturated queue. Change one worker or queue setting at a time.

Separate Filesystem Records From File Payload

Identify the source and destination filesystems and the metadata each backup preserves. On NTFS, record MFT growth and whether security descriptors, alternate streams, and timestamps are copied.

Microsoft states that the MFT contains at least one record per file, illustrating why many small files create far more filesystem-record work than one file of the same total size.

If the backup preserves ACLs, xattrs, sparse ranges, or checksums, include that work in the comparison. Turning off integrity features solely to improve a benchmark can weaken the backup.

Verify Real Thermal Throttling Instead of Inferring It

Log NVMe composite temperature, thermal warning time, controller busy time, and performance before and after cooling. Repeat with stable airflow and the same small-file set.

Kingston explains that thermal throttling reduces performance until the SSD returns to a safe temperature.

If throughput falls without a temperature threshold or thermal warning, call it a workload bottleneck rather than thermal throttling. Cooling will not solve metadata serialization or CPU-bound hashing.

Optimize the Backup Layer That Actually Saturates

Test a modest change such as fewer checksum workers, one additional copy worker, batched metadata, a local manifest, or excluding regenerable caches. Preserve restore-critical metadata.

The ZimaSpace article on small-file SMB copy testing covers network copies; this article isolates the local NVMe and backup pipeline when the large local copy remains fast.

The diagnosis is complete when the slowdown maps to metadata rate, random-write latency, cache cleanup, CPU work, or verified thermal throttling, and a targeted change improves the same controlled small-file test.