Benchmarks
Every speed number on this page comes from one run of the
bench workflow on a GitHub-hosted
runner, and the agreement table from the committed parity baseline. Both are generated: the first line of each block
names the skif commit, the tool versions, the runner image and the machine, and the speed block links the
workflow run the numbers came from. How they are
measured is under Methodology.
Full skif runs are timed against knip. skif --no-js, which skips the JS analysis, is timed against sherif and
syncpack. All of them run on the pinned public repositories with their
dependencies installed and on generated workspaces. Absolute times depend on the machine; the ratios are the result.
On npm-cli knip runs with --no-gitignore, since npm’s .gitignore otherwise hides every workspace from it. On the
large repositories (angular, vscode, n8n and next-js) both tools run with the configuration of the
agreement comparison rather than knip with the repository’s own, so they read the same files:
n8n’s own knip configuration checks dependencies only, and vscode’s workbench is reached only from the entry points its
bundler script names.
Medians of 15 runs after 3 warmup runs (5 runs after 1 warmup run on angular, vscode, n8n and next-js) (hyperfine 1.20.0), skif 0.7.0 at b2a21d3adb74 built with rustc 1.100.0 (0ef1bc386 2026-11-09) (measured by the harness at 27ef55dd0372) against knip 6.45.1, sherif 1.13.0, syncpack 15.3.3, measured on 2026-11-17 on a GitHub-hosted runner, image ubuntu24 20261111.370.1, Ubuntu 24.04.5 LTS, Linux 7.0.0-1017-azure, x86_64, AMD EPYC 7763 64-Core Processor, 4 logical CPUs (2 physical cores), 15.6 GiB memory, hypervisor Microsoft, load average 5.76 / 3.53 / 2.07 before and 3.24 / 3.01 / 2.82 after the run (workflow run, attempt 1).
Dead-code analysis against knip (full skif run)
| Repository | skif | knip | knip / skif |
|---|---|---|---|
| create-t3-turbo | 42 ms | fails: its Expo Metro config needs metro-cache, which pnpm’s strict install does not provide | - |
| vue-core | 93 ms | fails: its SFC playground Vite config reads a build-time global that only the repository’s own build defines | - |
| tanstack-query | 512 ms | 4.3 s | 8x |
| babel | 1.1 s | 6.6 s | 6x |
| npm-cli | 121 ms | 823 ms | 7x |
| angular | 1.2 s | 4.9 s | 4x |
| vscode | 3.2 s | 16.3 s | 5x |
| n8n | 6.3 s | 29.7 s | 5x |
| next-js | 1.1 s | 7.5 s | 7x |
knip exits with an error after its report on n8n (two Vite configs do not load outside the repository’s own build (@n8n/mcp-apps needs an MCP app mode, @n8n/nodes-langchain the unbuilt n8n-workflow); knip reports without their entries) and next-js (its root Jest config requires the built next/jest, which an install without scripts leaves out; knip reports without that config’s entries); its time there is that of a run without the entries of the configurations it could not load.
Type awareness (full skif run, types off by default and on with --type-aware)
| Target | types on | types off (default) | on / off | peak RSS, types on | peak RSS, types off (default) |
|---|---|---|---|---|---|
| graph: 10000 files in 100 packages | 437 ms | 336 ms | 1.3x | 119 MiB | 111 MiB |
| create-t3-turbo | 656 ms | 42 ms | 15.6x | 250 MiB | 28 MiB |
| vue-core | 422 ms | 93 ms | 4.6x | 147 MiB | 36 MiB |
| tanstack-query | 3.9 s | 512 ms | 7.6x | 403 MiB | 109 MiB |
| babel | 13.0 s | 1.1 s | 11.6x | 462 MiB | 158 MiB |
| npm-cli | 156 ms | 121 ms | 1.3x | 50 MiB | 45 MiB |
| angular | 5.1 s | 1.2 s | 4.4x | 619 MiB | 218 MiB |
| vscode | 13.9 s | 3.2 s | 4.3x | 2,964 MiB | 439 MiB |
| n8n | 27.6 s | 6.3 s | 4.4x | 3,661 MiB | 829 MiB |
| next-js | 3.3 s | 1.1 s | 3.1x | 554 MiB | 193 MiB |
Manifest and policy checks (skif --no-js)
| Target | skif –no-js | sherif | syncpack |
|---|---|---|---|
| create-t3-turbo | 4.4 ms | 33 ms | 40 ms |
| vue-core | 4.6 ms | 35 ms | 43 ms |
| tanstack-query | 12 ms | 41 ms | 122 ms |
| babel | 12 ms | 37 ms | 342 ms |
| npm-cli | 7.0 ms | 33 ms | 77 ms |
| angular | 6.9 ms | 33 ms | 104 ms |
| vscode | 61 ms | 34 ms | 163 ms |
| n8n | 130 ms | 36 ms | 231 ms |
| next-js | 96 ms | 34 ms | 436 ms |
| Generated, 50 packages | 5.6 ms | 35 ms | 46 ms |
| Generated, 500 packages | 29 ms | 59 ms | 178 ms |
| Generated, 2000 packages | 121 ms | 144 ms | 786 ms |
All measurements
Section titled “All measurements”Mean and median of every timed command on every target of the same run, with the ratio of each tool’s median to
skif’s, and the speed targets that ADR 0001
sets: manifest and policy rules on 500 packages in under 50 ms, the JS analysis at least 5x faster than knip on the same
tree, and parsing and building the module graph of 10,000 files in under 2 s. The graph target times a whole skif
run, which also runs every JS rule, so it is an upper bound.
generated: 50 packages (pnpm)
| Tool | Mean (ms) | Median (ms) | vs skif |
|---|---|---|---|
| skif –no-js | 5.6 | 5.6 | - |
| sherif | 35.3 | 35.4 | 6.3x |
| syncpack lint | 46.1 | 45.9 | 8.2x |
generated: 500 packages (pnpm)
| Tool | Mean (ms) | Median (ms) | vs skif |
|---|---|---|---|
| skif –no-js | 29.3 | 29.3 | - |
| sherif | 59.4 | 59.5 | 2.0x |
| syncpack lint | 178 | 178 | 6.1x |
generated: 2000 packages (pnpm)
| Tool | Mean (ms) | Median (ms) | vs skif |
|---|---|---|---|
| skif –no-js | 122 | 121 | - |
| sherif | 144 | 144 | 1.2x |
| syncpack lint | 792 | 786 | 6.5x |
graph: 10000 files in 100 packages
| Tool | Mean (ms) | Median (ms) | vs skif |
|---|---|---|---|
| skif | 347 | 336 | - |
| skif –type-aware | 437 | 437 | - |
corpus: create-t3-turbo (pnpm)
Installed with pnpm 10.19.0.
| Tool | Mean (ms) | Median (ms) | vs skif |
|---|---|---|---|
| skif | 41.7 | 42.0 | - |
| skif –type-aware | 672 | 656 | - |
| skif –no-js | 4.4 | 4.4 | - |
| sherif | 33.7 | 32.9 | 7.6x |
| syncpack lint | 40.6 | 39.9 | 9.2x |
| knip –reporter json | fails: its Expo Metro config needs metro-cache, which pnpm’s strict install does not provide | - | - |
corpus: vue-core (pnpm)
Installed with pnpm 12.11.2.
| Tool | Mean (ms) | Median (ms) | vs skif |
|---|---|---|---|
| skif | 92.7 | 92.7 | - |
| skif –type-aware | 421 | 422 | - |
| skif –no-js | 4.7 | 4.6 | - |
| sherif | 34.5 | 34.5 | 7.4x |
| syncpack lint | 43.2 | 43.4 | 9.3x |
| knip –reporter json | fails: its SFC playground Vite config reads a build-time global that only the repository’s own build defines | - | - |
corpus: tanstack-query (pnpm)
Installed with pnpm 12.4.2.
| Tool | Mean (ms) | Median (ms) | vs skif |
|---|---|---|---|
| skif | 513 | 512 | - |
| skif –type-aware | 3904 | 3893 | - |
| skif –no-js | 12.2 | 12.2 | - |
| sherif | 40.9 | 41.3 | 3.4x |
| syncpack lint | 123 | 122 | 10.1x |
| knip –reporter json | 4327 | 4329 | 8.4x |
corpus: babel (yarn)
Installed with yarn 4.17.0.
| Tool | Mean (ms) | Median (ms) | vs skif |
|---|---|---|---|
| skif | 1117 | 1118 | - |
| skif –type-aware | 12969 | 13004 | - |
| skif –no-js | 12.2 | 12.2 | - |
| skif –recursive | 1103 | 1101 | - |
| sherif | 36.6 | 36.8 | 3.0x |
| syncpack lint | 343 | 342 | 28.0x |
| knip –reporter json | 6552 | 6551 | 5.9x |
skif --recursive also lints the nested roots that are not workspace members. No comparison tool reads them, so the row has no ratio and is part of no claim.
corpus: npm-cli (npm)
Installed with npx --yes npm@11.6.2 ci --ignore-scripts.
| Tool | Mean (ms) | Median (ms) | vs skif |
|---|---|---|---|
| skif | 121 | 121 | - |
| skif –type-aware | 156 | 156 | - |
| skif –no-js | 7.0 | 7.0 | - |
| sherif | 32.4 | 32.5 | 4.6x |
| syncpack lint | 77.0 | 76.8 | 11.0x |
| knip –reporter json | 824 | 823 | 6.8x |
corpus: angular (pnpm)
Installed with pnpm 12.16.1.
5 runs after 1 warmup run, as corpus.toml caps them here.
| Tool | Mean (ms) | Median (ms) | vs skif |
|---|---|---|---|
| skif | 1159 | 1161 | - |
| skif –type-aware | 5087 | 5089 | - |
| skif –no-js | 6.9 | 6.9 | - |
| sherif | 33.0 | 32.9 | 4.7x |
| syncpack lint | 103 | 104 | 14.8x |
| knip –reporter json | 4862 | 4858 | 4.2x |
corpus: vscode (pnpm)
Installed with pnpm 12.16.0.
5 runs after 1 warmup run, as corpus.toml caps them here.
| Tool | Mean (ms) | Median (ms) | vs skif |
|---|---|---|---|
| skif | 3249 | 3246 | - |
| skif –type-aware | 13932 | 13940 | - |
| skif –no-js | 61.6 | 60.9 | - |
| skif –recursive | 5137 | 5137 | - |
| sherif | 34.6 | 34.4 | 0.6x |
| syncpack lint | 163 | 163 | 2.7x |
| knip –reporter json | 16369 | 16323 | 5.0x |
skif --recursive also lints the nested roots that are not workspace members. No comparison tool reads them, so the row has no ratio and is part of no claim.
corpus: n8n (pnpm)
Installed with pnpm 12.4.2.
5 runs after 1 warmup run, as corpus.toml caps them here.
| Tool | Mean (ms) | Median (ms) | vs skif |
|---|---|---|---|
| skif | 6276 | 6284 | - |
| skif –type-aware | 27676 | 27577 | - |
| skif –no-js | 130 | 130 | - |
| sherif | 36.7 | 36.0 | 0.3x |
| syncpack lint | 232 | 231 | 1.8x |
| knip –reporter json | 29725 | 29735 | 4.7x |
knip –reporter json exits with an error after its report here, as corpus.toml declares: two Vite configs do not load outside the repository’s own build (@n8n/mcp-apps needs an MCP app mode, @n8n/nodes-langchain the unbuilt n8n-workflow); knip reports without their entries.
corpus: next-js (pnpm)
Installed with pnpm 12.9.1.
5 runs after 1 warmup run, as corpus.toml caps them here.
| Tool | Mean (ms) | Median (ms) | vs skif |
|---|---|---|---|
| skif | 1071 | 1070 | - |
| skif –type-aware | 3277 | 3279 | - |
| skif –no-js | 95.4 | 95.5 | - |
| skif –recursive | 3326 | 3324 | - |
| sherif | 34.7 | 34.3 | 0.4x |
| syncpack lint | 436 | 436 | 4.6x |
| knip –reporter json | 7524 | 7531 | 7.0x |
skif --recursive also lints the nested roots that are not workspace members. No comparison tool reads them, so the row has no ratio and is part of no claim.
knip –reporter json exits with an error after its report here, as corpus.toml declares: its root Jest config requires the built next/jest, which an install without scripts leaves out; knip reports without that config’s entries.
| ADR 0001 target | Check | Result |
|---|---|---|
| generated: 500 packages (pnpm) | manifest and policy rules < 50 ms (measured 29.3 ms) | pass |
| graph: 10000 files in 100 packages | parse and module graph of 10000 files < 2000 ms, whole run including every JS rule (measured 347.3 ms) | pass |
| corpus: create-t3-turbo (pnpm) | JS analysis >= 5x faster than knip (not measured: knip fails here: its Expo Metro config needs metro-cache, which pnpm’s strict install does not provide) | not measured |
| corpus: vue-core (pnpm) | JS analysis >= 5x faster than knip (not measured: knip fails here: its SFC playground Vite config reads a build-time global that only the repository’s own build defines) | not measured |
| corpus: tanstack-query (pnpm) | JS analysis >= 5x faster than knip (measured 8.4x) | pass |
| corpus: babel (yarn) | JS analysis >= 5x faster than knip (measured 5.9x) | pass |
| corpus: npm-cli (npm) | JS analysis >= 5x faster than knip (measured 6.8x) | pass |
| corpus: angular (pnpm) | JS analysis >= 5x faster than knip (measured 4.2x) | FAIL |
| corpus: vscode (pnpm) | JS analysis >= 5x faster than knip (measured 5.0x) | pass |
| corpus: n8n (pnpm) | JS analysis >= 5x faster than knip (measured 4.7x; knip –reporter json exited with an error in 5 runs, as corpus.toml declares, after its report: two Vite configs do not load outside the repository’s own build (@n8n/mcp-apps needs an MCP app mode, @n8n/nodes-langchain the unbuilt n8n-workflow); knip reports without their entries) | FAIL |
| corpus: next-js (pnpm) | JS analysis >= 5x faster than knip (measured 7.0x; knip –reporter json exited with an error in 5 runs, as corpus.toml declares, after its report: its root Jest config requires the built next/jest, which an install without scripts leaves out; knip reports without that config’s entries) | pass |
| Type-aware budget (ADR 0010) | Check | Result |
|---|---|---|
| graph: 10000 files in 100 packages | peak RSS with types <= 4096 MiB (measured 119 MiB) | pass |
| graph: 10000 files in 100 packages | whole run with types < 5000 ms (measured 437.4 ms) | pass |
| corpus: create-t3-turbo (pnpm) | peak RSS with types <= 4096 MiB (measured 250 MiB) | pass |
| corpus: create-t3-turbo (pnpm) | JS analysis >= 5x faster than knip (not measured: knip fails here: its Expo Metro config needs metro-cache, which pnpm’s strict install does not provide) | not measured |
| corpus: vue-core (pnpm) | peak RSS with types <= 4096 MiB (measured 147 MiB) | pass |
| corpus: vue-core (pnpm) | JS analysis >= 5x faster than knip (not measured: knip fails here: its SFC playground Vite config reads a build-time global that only the repository’s own build defines) | not measured |
| corpus: tanstack-query (pnpm) | peak RSS with types <= 4096 MiB (measured 403 MiB) | pass |
| corpus: tanstack-query (pnpm) | JS analysis >= 5x faster than knip (measured 1.1x) | FAIL |
| corpus: babel (yarn) | peak RSS with types <= 4096 MiB (measured 462 MiB) | pass |
| corpus: babel (yarn) | JS analysis >= 5x faster than knip (measured 0.5x) | FAIL |
| corpus: npm-cli (npm) | peak RSS with types <= 4096 MiB (measured 50 MiB) | pass |
| corpus: npm-cli (npm) | JS analysis >= 5x faster than knip (measured 5.3x) | pass |
| corpus: angular (pnpm) | peak RSS with types <= 4096 MiB (measured 619 MiB) | pass |
| corpus: angular (pnpm) | JS analysis >= 5x faster than knip (measured 1.0x) | FAIL |
| corpus: vscode (pnpm) | peak RSS with types <= 4096 MiB (measured 2964 MiB) | pass |
| corpus: vscode (pnpm) | JS analysis >= 5x faster than knip (measured 1.2x) | FAIL |
| corpus: n8n (pnpm) | peak RSS with types <= 4096 MiB (measured 3661 MiB) | pass |
| corpus: n8n (pnpm) | JS analysis >= 5x faster than knip (measured 1.1x; knip –reporter json exited with an error in 5 runs, as corpus.toml declares, after its report: two Vite configs do not load outside the repository’s own build (@n8n/mcp-apps needs an MCP app mode, @n8n/nodes-langchain the unbuilt n8n-workflow); knip reports without their entries) | FAIL |
| corpus: next-js (pnpm) | peak RSS with types <= 4096 MiB (measured 554 MiB) | pass |
| corpus: next-js (pnpm) | JS analysis >= 5x faster than knip (measured 2.3x; knip –reporter json exited with an error in 5 runs, as corpus.toml declares, after its report: its root Jest config requires the built next/jest, which an install without scripts leaves out; knip reports without that config’s entries) | FAIL |
| corpus: n8n (pnpm) | types on <= 3x types off on the largest repository, the one whose untyped run takes longest (measured 4.4x: 27676 ms against 6276 ms) | FAIL |
| corpus | median types on / types off across 9 repositories <= 2x (measured 4.4x) | FAIL |
Accuracy on seeded defects
Section titled “Accuracy on seeded defects”A generated 500-package workspace with defects injected on purpose. Actual is counted from the files, so the right
answer is known. Counted by hand on 2026-11-06 with skif 0.1.0 and not re-counted since; the method is in the
benchmark harness:
| Defect | Actual | skif | sherif | syncpack |
|---|---|---|---|---|
| Dependencies with mismatched versions | 200 | 200 | 200 | 200 |
| Unsorted dependency maps | 25 | 25 | 25 | n/a |
Type packages in dependencies |
24 | 24 | 24 | n/a |
| Empty dependency objects | 31 | 31 | 31 | n/a |
skif and sherif name the same 200 dependencies. skif and syncpack report the same 4,568 mismatched instances, with
nothing unique to either. n/a means the tool does not check for that defect.
Precision on seeded dead code
Section titled “Precision on seeded dead code”A generated 5-package workspace with seeded unused files, exports, types, dependencies and class, enum and namespace
members, next to live code that is reached only through patterns that caused false findings on real repositories
(namespace reads, typeof import(), import.meta.glob, tsconfig paths, # subpath imports, manual mocks, fixtures
and more) and live members that matching by name can get wrong (one member name on two classes, reads through an
interface, a generic constraint, destructuring, a key of literal type, inheritance and overrides, enum and namespace
aliases). The answer key lists 73 seeded findings: the 33 of earlier measurements
and 40 members. skif 0.8.0 (commit 4d3bdd7f) and knip 6.45.1, measured on 2026-11-17; reproduce with
mise run bench:deadcode -- --knip:
| Tool | Precision | Recall | Precision, 33 non-member findings | Recall, 33 non-member findings |
|---|---|---|---|---|
| skif | 100% | 79% | 100% | 100% |
| knip | 74% | 59% | 69% | 100% |
On members alone (20 class, 15 enum and 5 namespace members seeded):
| Tool | Class members | Enum members | Namespace members |
|---|---|---|---|
| skif | 15 of 20 found, no false finding | 5 of 15 found, no false finding | 5 of 5 found, no false finding |
| knip | no check (dropped in knip 6) | 5 of 15 found, no false finding | 5 of 5 found, no false finding |
skif matches member reads by name, without a type checker
(ADR 0010). The class members it
misses are unused members whose name another class’s member shares: a read of area on one class also credits the
unused area of another. It reports no live member, including those read through an interface the class satisfies
without implements, through a generic constraint (T extends HasWeight) and through a key of literal type
(frame[axis], axis: "height"). skif and knip both miss the unused members of an enum that is assigned to a
variable or indexed by a keyof typeof key, which counts as using the whole enum. knip’s false findings are exports read only through
typeof import("./m")["name"], files in fixtures directories and a devDependency whose binary only vercel.json
runs. Per issue type and pattern: the
benchmark harness.
Agreement with knip
Section titled “Agreement with knip”Real repositories have no answer key for dead code, so this measures agreement, not correctness. A weekly job runs both tools on pinned public repositories and compares the findings per issue type; the table is generated from the committed baseline of that comparison:
Findings knip@6.45.1 and skif share on 18 pinned repositories (angular, babel, cal-com, create-t3-turbo, jest, mastra, n8n, next-js, npm-cli, nx-examples-integrated, opencode, rules-js, rushstack, tanstack-query, turborepo-examples, vercel-examples, vscode, vue-core; 17 of them installed), from tools/bench/baselines/parity.json, recorded at skif 4b75e64ede5d, 633f7df720bf.
| Finding | Shared | Of skif’s findings | Of knip’s findings |
|---|---|---|---|
| Unused files | 9,223 | 100% | 97% |
| Unused exports | 6,842 | 100% | 100% |
| Unused types | 3,644 | 100% | 100% |
| Unused dev dependencies | 1,098 | 99% | 98% |
| Unlisted dependencies | 831 | 100% | 99% |
| Unused dependencies | 474 | 100% | 96% |
| Unused namespace exports | 425 | 100% | 100% |
| Unused namespace types | 354 | 100% | 100% |
| Duplicate exports | 164 | 99% | 99% |
| Unused enum members | 123 | 100% | 100% |
| Unused namespace members | 105 | 99% | 95% |
| Unlisted binaries | 88 | 99% | 97% |
| Unresolved imports | 25 | 100% | 100% |
| Unused optional peer dependencies | 1 | 100% | 33% |
A low share means the tools differ, and either one could be right. Some differences are intended and counted apart:
skif makes test files entries, does not report files in fixtures directories
(ADR 0007), reads a
namespace import differently. Findings in packages skif does not judge, in packages where knip could not load a config,
inside a file the other tool reports as unused (the file is the one disagreement), and exports used only by such
files are counted apart too. So are declaration files, which skif never reports as unused, types named in the
signature of an export skif keeps
(ADR 0009),
duplicate exports of entry modules, which skif’s rule leaves alone, and dependencies a Bazel BUILD file names, which
knip does not read. It also sets apart what follows from where each tool looks, decided from paths and manifests:
projects with their own package.json that are not knip workspaces, on both sides (knip neither judges their
dependencies nor uses their entry files, while skif lints them as roots of their own), directories skif’s file
discovery skips, and TypeScript sources whose built path a package publishes, also through a tsconfig outDir (and
the node and credential files an n8n node package lists), which skif makes entries, together with their exports and
the files they import that only knip reports as unused. An import one tool reports as unlisted and the other as
unresolved counts as agreement. A repository whose knip run is not a valid comparison is named and left out of the
sums. Reading the disagreements found false findings on both sides
(ADR 0007).
The remaining differences, and what each tool reports differently, are in the
comparison guide.
Methodology
Section titled “Methodology”What is measured
Section titled “What is measured”skif-bench run times whole
command-line invocations, process start-up included, on three kinds of targets:
| Target | Contents | Tools timed |
|---|---|---|
| Generated workspaces | 50, 500 and 2000 packages, pnpm layout, 5% issue rate, seed 1 | skif --no-js, sherif, syncpack lint |
| Module graph | 10,000 TypeScript files in 100 packages | skif |
| Pinned corpus | the bench repositories of corpus.toml |
skif, skif --no-js, sherif, syncpack lint, knip --reporter json, and skif --recursive where a repository holds nested roots (informational) |
skif runs as skif . --format json --no-install, with --no-js for the manifest and policy rows and, where a
repository sets bench-knip-config = "parity", with a --config that adds the entry files the agreement comparison gives both tools,
while knip gets that comparison’s configuration in place of the repository’s own; the working directory is the
target. Each compared row is timed on the scope its comparison tool reads, so none runs with --recursive
(ADR 0014):
a recursive run also lints the nested projects that are not workspace members (403 in next-js), which sherif, syncpack
and knip do not read. Where a repository holds such projects (babel, vscode, next-js), a recursive run is timed as an
informational row that is part of no ratio, claim or check. skif then runs once more under /usr/bin/time for its peak resident set size. Non-zero exits are
ignored, since a linter exits 1 when it finds something; a run that exits above 1 or by a signal is an error, since its
time may be that of a partial run.
Runs and statistics
Section titled “Runs and statistics”hyperfine runs each command without a shell (-N), with 3 warmup runs and 15
measured runs by default (the workflow’s warmup and runs inputs). On the large repositories, where one knip run
takes tens of seconds, corpus.toml caps them at 1 warmup run and 5 measured runs (bench-max-warmup,
bench-max-runs), so the whole run fits the workflow’s time limit; the tables name the counts of a capped target. The
tables show medians, which one slow run does not move. The warmup means the numbers are warm file cache, not cold
start.
Pinned corpus and tools
Section titled “Pinned corpus and tools”tools/bench/corpus.toml pins every repository
to a full commit SHA and declares the sherif, syncpack and knip versions under [tools]. The harness installs those
versions with npm ci --ignore-scripts from tools/bench/tools/package-lock.json, refuses to run when the lock and
[tools] differ, and times the executables in node_modules/.bin directly, so no npx resolution is timed and
nothing on PATH is used. The harness fails before timing anything when a repository is not pinned to a full commit
SHA or a tool not to an exact version.
Installed dependencies
Section titled “Installed dependencies”knip needs installed dependencies to load the repositories’ tool configurations, so the workflow installs each corpus
repository first (run --install), through a pinned corepack whose pnpm and yarn shims follow the repository’s
packageManager. Installing changes the timings of every tool, not only knip’s, so compare only runs made the same
way. Where knip fails even installed, for reasons that are the repository’s own, corpus.toml declares the failure
(bench-knip-failure); a run whose error matches is published as fails: <reason>, without a ratio, and any other
error stops the run from being published. Where knip prints its whole report before the error, because a configuration
it cannot load only takes the entries it names away (next-js’s root Jest config, two n8n Vite configs), the
declaration says so (timed = true) and the run is published with its time and ratio and the reason beside them.
The runner
Section titled “The runner”The workflow runs only when dispatched, on a pinned GitHub-hosted runner image (ubuntu-24.04 or
ubuntu-24.04-arm), and builds skif and the harness with --locked and the rust-version of their Cargo.toml
(ADR 0014).
GitHub-hosted runners are shared virtual machines: a neighbour’s load moves the times of a run by several percent, and
two runs of the same image may land on different CPU models (the summary names the one it got). Read the ratios
against knip, sherif and syncpack, which share the machine and its noise, rather than the absolute times.
Publishing a run
Section titled “Publishing a run”mise run bench:import -- <run-id> imports only a successful, dispatched run of bench whose commit is on main,
puts its summary and raw hyperfine exports into
tools/bench/results, and regenerates the blocks of
this page and the headline claims of the README and the comparison guide. It writes
nothing when a comparison tool was not timed, the run was narrowed to some repositories, a tool exited with an error
that is not a declared knip failure, an ADR 0001 check is invalid, or the run did not come from clean checkouts on a
GitHub-hosted runner. The headline claims give, for each tool, the range of the ratios of its median to skif’s over
the targets it completed on, with the median ratio from three targets on, each rounded down to one decimal.
Reproducing
Section titled “Reproducing”gh workflow run bench.yml -f ref=main -f runner=ubuntu-24.04 # also: -f runs=15 -f warmup=3 -f only=babelgh run list --workflow bench.yml --limit 1 # the run idmise run bench:import -- <run-id>On your own machine, mise run bench builds skif in release mode and times it the same way (it needs hyperfine and
npm); mise run bench:local also installs the corpus and the tools first, as the workflow does. Local numbers are
not published. mise run bench:deadcode -- --knip reproduces the precision table, and
mise run parity -- --install --write-baseline followed by mise run parity:publish the agreement table. The
benchmark harness documents every command and
option.