How to Benchmark Immich With a Repeatable Home-Server Workload

Eva Wong

IceWhale author

Eva Wong is the Technical Writer and resident tinkerer at ZimaSpace. A lifelong geek with a passion for homelabs and open-source software, she specializes in translating complex technical concepts into accessible, hands-on guides. Eva believes that self-hosting should be fun, not intimidating. Through her tutorials, she empowers the community to demystify hardware setups, from building their first NAS to mastering Docker containers.

A repeatable Immich benchmark fixes the media cohort, client path, cache state, background overlap, and measured endpoint before changing one variable.

Without those controls, a faster second run may reflect warm data rather than better hardware, while an idle test may hide import-time contention. A useful home-server benchmark reproduces the household’s real browsing, upload, search, and recovery demands in the same sequence.

Define User Outcomes Before Collecting Metrics

Start with observable outcomes such as upload acceptance, time until a new photo becomes searchable, timeline thumbnail completion, original opening, video start, or recovery to a usable library. CPU utilization and disk throughput explain these outcomes, but they are not substitutes for the user-facing result.

The ZimaSpace Immich data-path article separates upload acceptance, processing readiness, search selection, and delivered media. That structure is useful because a benchmark must time one endpoint rather than blending several dependent stages into a misleading total.

Select two interactive endpoints and one background endpoint. Give each a pass target and record median plus slow-tail latency or completion rate. A benchmark with many unrelated scores becomes difficult to interpret; a small set tied to a specific decision keeps the result actionable.

Freeze the Dataset, Client Path, and Starting State

Use the same representative photos and videos in every run, including formats and sizes that match the family library. Fix the account, client device, network route, Immich version, and derivative settings. Even a changed browser cache or Wi-Fi path can overwhelm the configuration difference being tested.

A vector-search benchmarking article emphasizes repeatable embeddings, insertion, and retrieval workloads when studying PostgreSQL search performance. Immich is a broader application, but the experimental principle transfers: controlled inputs and defined retrieval operations are required before timing differences support a conclusion.

Create a dataset manifest containing file checksums, counts, byte totals, photo formats, video durations, and expected search results. Add a start checklist for restarted services, warm-up actions, queued jobs, and competing applications. If the checklist differs, label the run noncomparable instead of averaging it.

Run Cold, Warm, and Sustained Phases

The cold phase reveals initialization and first-access cost. The warm phase reveals reuse under immediate repetition. The sustained phase mixes repeated interactions with a representative import or background queue long enough to expose thermal throttling, memory pressure, storage queues, and resource competition.

A community report describes a first smart search taking about five seconds and an immediate repeat around half a second while model memory changes. Those numbers are not a benchmark standard; they demonstrate why averaging cold and warm requests conceals the transition the test needs to explain.

Run each phase at least three times, preserving raw timestamps and resource traces. Keep median and a slow-tail measure for interactive work, plus queue items per minute for background work. Stop the run if errors, swap storms, or thermal limits invalidate the intended steady condition.

Featured

ZimaCube 2 Personal Cloud Home NAS

ZimaCube2

Use a One-Variable Run Sheet

Write the hypothesis before the run: changing database placement, worker concurrency, memory limit, network path, or accelerator should improve one named endpoint through one stated mechanism. Keep everything else fixed. This prevents a multi-change upgrade from producing a faster result with no defensible cause.

A general bottleneck analysis explains that CPU, RAM, storage, and network limits produce different utilization patterns and user effects. Applied to Immich, supporting metrics should move consistently with the endpoint; the busiest chart is not automatically the limiting component.

Record baseline and changed results for cold, warm, and sustained phases, then mark pass, fail, or inconclusive. Reject improvements that disappear on repetition or cause errors, unstable temperatures, lost queue progress, or longer recovery. Preserve the manifest and run sheet so a future release can be compared honestly.