Performance

This page presents performance benchmarks for Panache, comparing its formatting and linting speed against popular alternatives like Prettier, Pandoc, rumdl, mdformat, Yamark, mado, markdownlint, and markdownlint-cli2 on real Quarto and Markdown documents, and its language-server latency, runtime, and memory use against Marksman and q2. The benchmarks highlight Panache’s efficiency and suitability for editor and repository-wide work.

Overview

Panache is designed for speed without compromising on correctness. Built in Rust and compiled to native code, it delivers fast formatting with low startup overhead. In this document, we present benchmarks comparing Panache’s formatting and linting performance against popular alternatives like Prettier, Pandoc, rumdl, mdformat, Yamark, mado, markdownlint, and markdownlint-cli2 on a realistic corpus of Quarto and Markdown documents. R Markdown is outside the current benchmark corpus.

The formatting and linting benchmarks have two suites: per-document benchmarks that run each formatter on each document in the corpus individually, and repository-wide benchmarks that run formatters on entire repositories of tracked documents. The former highlights raw formatting speed on a variety of real-world documents; the latter captures the overhead of processing multiple files in a single run, which is more representative of repository-wide formatting or linting. CLI caches are disabled for these comparisons. A separate language-server suite measures startup, warm request latency, edit runtime, and memory use during an editor session.

Yamark runs with an empty configuration and external code formatters disabled (--config /dev/null --skip-embedded-formatters). Its default wrapping and style settings apply. Tools differ in supported syntax and formatting policy, so these timings compare their respective workloads. Failed repository runs have null timings in the data and are excluded from the plots.

Yamark’s formatting path uses source spans and formatting plans and preserves display math as opaque content. Panache builds the lossless syntax tree also used by its linter and language server, and parses and formats TeX math, which explains part of the gap.

The numbers on this page are produced by the scripts in benches/ and read from the JSON files written next to this document. The benchmark chunks on this page are intentionally not executed during preview or render, so small content edits reuse the existing JSON files instead of rerunning the benchmarks. Refresh the benchmark data explicitly with the commands at the bottom of this page, then delete docs/_freeze/guide/performance/ and re-render if you want the page to pick up newly generated results.

data = FileAttachment("performance_data.json").json();

documentMeta = new Map(data.documents.map((d) => [d.id, d]));

panacheByDoc = d3.rollup(
    data.results.filter((r) => r.formatter === "panache"),
    (v) => v[0].mean_ms,
    (r) => r.document,
);

records = data.results.map((r) => ({
    formatter: r.formatter,
    document: r.document,
    documentName: documentMeta.get(r.document)?.name ?? r.document,
    mean_ms: r.mean_ms,
    runs: r.runs,
    ratio: r.mean_ms / panacheByDoc.get(r.document),
}));

formatters = Array.from(
    d3.rollup(
        records,
        (v) => d3.mean(v, (d) => d.ratio),
        (d) => d.formatter,
    ),
)
    .sort(([, a], [, b]) => a - b)
    .map(([k]) => k);

documentIds = data.documents.map((d) => d.id);

formatterRanges = d3.rollup(
    records.filter((r) => r.formatter !== "panache"),
    (v) => {
        const lo = Math.round(d3.min(v, (d) => d.ratio));
        const hi = Math.round(d3.max(v, (d) => d.ratio));
        return `${lo}\u2013${hi}x`;
    },
    (d) => d.formatter,
);

prettierRange = formatterRanges.get("prettier") ?? "n/a";
pandocRange = formatterRanges.get("pandoc") ?? "n/a";
rumdlRange = formatterRanges.get("rumdl") ?? "n/a";
mdformatRange = formatterRanges.get("mdformat") ?? "n/a";

html`<div class="callout-note">
  <div class="callout-header" style="font-weight: 600">Test Environment</div>
  <ul style="margin: 0">
    <li>Generated: ${new Date(data.meta.generated_at).toUTCString()}</li>
    <li>Host: ${data.meta.host.cpu} (${data.meta.host.os}/${data.meta.host.arch})</li>
    <li>Minimum runs: ${data.meta.min_runs}</li>
    <li>panache ${data.meta.tools.panache.version}, prettier ${data.meta.tools.prettier?.version ?? "n/a"}, pandoc ${data.meta.tools.pandoc?.version ?? "n/a"}, rumdl ${data.meta.tools.rumdl?.version ?? "n/a"}, mdformat ${data.meta.tools.mdformat?.version ?? "n/a"}, yamark ${data.meta.tools.yamark?.version ?? "n/a"}, mado ${data.meta.tools.mado?.version ?? "n/a"}, markdownlint ${data.meta.tools.markdownlint?.version ?? "n/a"}, markdownlint-cli2 ${data.meta.tools["markdownlint-cli2"]?.version ?? "n/a"}</li>
  </ul>
</div>`;

Formatting

Single-Document

In Figure 1, we compare formatting time per document across panache, Prettier, Pandoc, rumdl, mdformat, and Yamark. Each dot is one document; the y-axis shows time relative to panache (×, lower is faster). Panache sits at 1× by construction (dashed baseline); points above it are slower. Hover a point to see the absolute wall-clock time in milliseconds. Formatters are ordered left-to-right from fastest to slowest on average.

Plot.plot({
    marginLeft: 60,
    marginRight: 20,
    marginBottom: 40,
    marginTop: 20,
    height: 360,
    x: { label: null, domain: formatters },
    y: {
        label: "Mean time relative to panache (×, lower is faster)",
        grid: true,
    },
    color: {
        legend: true,
        domain: documentIds,
        tickFormat: (id) => documentMeta.get(id)?.name ?? id,
        scheme: "tableau10",
    },
    marks: [
        Plot.ruleY([1], {
            stroke: "#2a9d8f",
            strokeWidth: 2,
            strokeDasharray: "4 3",
        }),
        Plot.dot(records, {
            x: "formatter",
            y: "ratio",
            fill: "document",
            r: 6,
            fillOpacity: 0.85,
            stroke: "white",
            strokeWidth: 1,
            channels: {
                document: "documentName",
                "time (ms)": (d) => d.mean_ms.toFixed(1),
                "× panache": (d) => d.ratio.toFixed(2),
            },
            tip: true,
        }),
    ],
});

Figure 1: Formatting time per document, comparing panache to Prettier, Pandoc, rumdl, mdformat, and Yamark. Each dot is one document; the y-axis shows time relative to panache (×, lower is faster).

prepareRepoSuiteRows = (data, toolLabels) => {
    const repoMeta = new Map(data.repos.map((r) => [r.id, r]));
    const baselineByRepo = new Map(
        data.results
            .filter((r) => r.tool === "panache" && !r.failed && r.mean_ms != null)
            .map((r) => [r.repo, r.mean_ms]),
    );

    return data.results
        .filter((r) => !r.failed && r.mean_ms != null && baselineByRepo.has(r.repo))
        .map((r) => {
            const repo = repoMeta.get(r.repo);
            const repoLabel = repo?.name ?? r.repo;
            const toolLabel = toolLabels[r.tool] ?? r.tool;
            return {
                ...r,
                repoLabel,
                toolLabel,
                fileCount: repo?.file_count ?? null,
                totalBytes: repo?.total_bytes ?? null,
                barLabel: `${repoLabel} — ${toolLabel}`,
                ratio: r.mean_ms / baselineByRepo.get(r.repo),
            };
        });
};

Repository-Wide

repoMarkdownFormat = FileAttachment(
    "performance_repo_markdown_format_data.json",
).json();

repoMarkdownFormatResults = prepareRepoSuiteRows(repoMarkdownFormat, {
    panache: "panache",
    prettier: "prettier",
    rumdl: "rumdl",
    mdformat: "mdformat",
    yamark: "yamark",
});
repoMarkdownFormatTools = Array.from(
    d3.rollup(
        repoMarkdownFormatResults,
        (v) => d3.mean(v, (d) => d.ratio),
        (d) => d.toolLabel,
    ),
)
    .sort(([, a], [, b]) => a - b)
    .map(([k]) => k);
repoMarkdownFormatRepos = repoMarkdownFormat.repos.map((r) => r.id);
repoMarkdownFormatRepoMeta = new Map(
    repoMarkdownFormat.repos.map((r) => [r.id, r]),
);

In Figure 2, we compare formatting time across panache, Prettier, rumdl, mdformat, and Yamark on tracked Markdown files from several repositories.

Plot.plot({
    marginLeft: 60,
    marginRight: 20,
    marginBottom: 40,
    marginTop: 20,
    height: 320,
    x: { label: null, domain: repoMarkdownFormatTools },
    y: {
        label:
            "Mean time relative to panache within each repo (×, lower is faster)",
        grid: true,
    },
    color: {
        legend: true,
        domain: repoMarkdownFormatRepos,
        tickFormat: (id) => repoMarkdownFormatRepoMeta.get(id)?.name ?? id,
        scheme: "tableau10",
    },
    marks: [
        Plot.ruleY([1], {
            stroke: "#2a9d8f",
            strokeWidth: 2,
            strokeDasharray: "4 3",
        }),
        Plot.dot(repoMarkdownFormatResults, {
            x: "toolLabel",
            y: "ratio",
            fill: "repo",
            r: 6,
            fillOpacity: 0.85,
            stroke: "white",
            strokeWidth: 1,
            channels: {
                repo: "repoLabel",
                tool: "toolLabel",
                files: (d) => d.fileCount,
                "time (ms)": (d) => d.mean_ms.toFixed(1),
                "× panache": (d) => d.ratio.toFixed(2),
            },
            tip: true,
        }),
    ],
});

Figure 2: Repository-wide formatting benchmarks on standard Markdown repos, comparing panache to Prettier, rumdl, mdformat, and Yamark. Each dot is one repo/tool pair; the y-axis shows time relative to panache within that repo (×, lower is faster).

repoQuartoFormat = FileAttachment(
    "performance_repo_quarto_format_data.json",
).json();

repoQuartoFormatResults = prepareRepoSuiteRows(repoQuartoFormat, {
    panache: "panache",
    rumdl: "rumdl",
    yamark: "yamark",
});
repoQuartoFormatTools = Array.from(
    d3.rollup(
        repoQuartoFormatResults,
        (v) => d3.mean(v, (d) => d.ratio),
        (d) => d.toolLabel,
    ),
)
    .sort(([, a], [, b]) => a - b)
    .map(([k]) => k);
repoQuartoFormatRepos = repoQuartoFormat.repos.map((r) => r.id);
repoQuartoFormatRepoMeta = new Map(
    repoQuartoFormat.repos.map((r) => [r.id, r]),
);

In Figure 3, we compare formatting time across panache, rumdl, and Yamark on tracked .qmd files from several Quarto repositories.

Plot.plot({
    marginLeft: 60,
    marginRight: 20,
    marginBottom: 40,
    marginTop: 20,
    height: 320,
    x: { label: null, domain: repoQuartoFormatTools },
    y: {
        label:
            "Mean time relative to panache within each repo (×, lower is faster)",
        grid: true,
    },
    color: {
        legend: true,
        domain: repoQuartoFormatRepos,
        tickFormat: (id) => repoQuartoFormatRepoMeta.get(id)?.name ?? id,
        scheme: "tableau10",
    },
    marks: [
        Plot.ruleY([1], {
            stroke: "#2a9d8f",
            strokeWidth: 2,
            strokeDasharray: "4 3",
        }),
        Plot.dot(repoQuartoFormatResults, {
            x: "toolLabel",
            y: "ratio",
            fill: "repo",
            r: 6,
            fillOpacity: 0.85,
            stroke: "white",
            strokeWidth: 1,
            channels: {
                repo: "repoLabel",
                tool: "toolLabel",
                files: (d) => d.fileCount,
                "time (ms)": (d) => d.mean_ms.toFixed(1),
                "× panache": (d) => d.ratio.toFixed(2),
            },
            tip: true,
        }),
    ],
});

Figure 3: Repository-wide formatting benchmarks on Quarto repos. Each dot is one repo/tool pair; the y-axis shows time relative to panache within that repo (×, lower is faster).

Each dot is one repo/tool pair. Panache sits at 1× by construction (dashed baseline), and points above it are slower on that repository. Hover a point to see the absolute wall-clock time in milliseconds and the corpus size.

Linting

Single-Document

lintSingleData = FileAttachment("performance_lint_single_data.json").json();

lintDocumentMeta = new Map(lintSingleData.documents.map((d) => [d.id, d]));
lintPanacheByDoc = d3.rollup(
    lintSingleData.results.filter((r) => r.tool === "panache"),
    (v) => v[0].mean_ms,
    (r) => r.document,
);
lintSingleRecords = lintSingleData.results.map((r) => ({
    tool: r.tool,
    document: r.document,
    documentName: lintDocumentMeta.get(r.document)?.name ?? r.document,
    mean_ms: r.mean_ms,
    runs: r.runs,
    ratio: r.mean_ms / lintPanacheByDoc.get(r.document),
}));
lintTools = Array.from(
    d3.rollup(
        lintSingleRecords,
        (v) => d3.mean(v, (d) => d.ratio),
        (d) => d.tool,
    ),
)
    .sort(([, a], [, b]) => a - b)
    .map(([k]) => k);
lintDocumentIds = lintSingleData.documents.map((d) => d.id);

In Figure 4, we compare linting time per document across panache lint, rumdl check, mado check, markdownlint, and markdownlint-cli2. Each dot is one document; the y-axis shows time relative to panache lint (×, lower is faster). Panache sits at 1× by construction.

Plot.plot({
    marginLeft: 60,
    marginRight: 20,
    marginBottom: 40,
    marginTop: 20,
    height: 300,
    x: { label: null, domain: lintTools },
    y: {
        label: "Mean time relative to panache lint (×, lower is faster)",
        grid: true,
    },
    color: {
        legend: true,
        domain: lintDocumentIds,
        tickFormat: (id) => lintDocumentMeta.get(id)?.name ?? id,
        scheme: "tableau10",
    },
    marks: [
        Plot.ruleY([1], {
            stroke: "#2a9d8f",
            strokeWidth: 2,
            strokeDasharray: "4 3",
        }),
        Plot.dot(lintSingleRecords, {
            x: "tool",
            y: "ratio",
            fill: "document",
            r: 6,
            fillOpacity: 0.85,
            stroke: "white",
            strokeWidth: 1,
            channels: {
                document: "documentName",
                "time (ms)": (d) => d.mean_ms.toFixed(1),
                "× panache lint": (d) => d.ratio.toFixed(2),
            },
            tip: true,
        }),
    ],
});

Figure 4: Linting time per document, comparing panache lint to rumdl check, mado check, markdownlint, and markdownlint-cli2. Each dot is one document; the y-axis shows time relative to panache lint (×, lower is faster).

Repository-Wide

This suite benchmarks linting on the same standard Markdown repositories as the formatting comparison, using panache lint, rumdl check, mado check, markdownlint, and markdownlint-cli2. In Figure 5, each dot is one repo/tool pair; the y-axis shows time relative to panache lint within that repo (×, lower is faster).

repoMarkdownLint = FileAttachment(
    "performance_repo_markdown_lint_data.json",
).json();

repoMarkdownLintResults = prepareRepoSuiteRows(repoMarkdownLint, {
    panache: "panache lint",
    rumdl: "rumdl check",
    mado: "mado check",
    markdownlint: "markdownlint",
    "markdownlint-cli2": "markdownlint-cli2",
});
repoMarkdownLintTools = Array.from(
    d3.rollup(
        repoMarkdownLintResults,
        (v) => d3.mean(v, (d) => d.ratio),
        (d) => d.toolLabel,
    ),
)
    .sort(([, a], [, b]) => a - b)
    .map(([k]) => k);
repoMarkdownLintRepos = repoMarkdownLint.repos.map((r) => r.id);
repoMarkdownLintRepoMeta = new Map(
    repoMarkdownLint.repos.map((r) => [r.id, r]),
);

Plot.plot({
    marginLeft: 60,
    marginRight: 20,
    marginBottom: 40,
    marginTop: 20,
    height: 300,
    x: { label: null, domain: repoMarkdownLintTools },
    y: {
        label:
            "Mean time relative to panache lint within each repo (×, lower is faster)",
        grid: true,
    },
    color: {
        legend: true,
        domain: repoMarkdownLintRepos,
        tickFormat: (id) => repoMarkdownLintRepoMeta.get(id)?.name ?? id,
        scheme: "tableau10",
    },
    marks: [
        Plot.ruleY([1], {
            stroke: "#2a9d8f",
            strokeWidth: 2,
            strokeDasharray: "4 3",
        }),
        Plot.dot(repoMarkdownLintResults, {
            x: "toolLabel",
            y: "ratio",
            fill: "repo",
            r: 6,
            fillOpacity: 0.85,
            stroke: "white",
            strokeWidth: 1,
            channels: {
                repo: "repoLabel",
                tool: "toolLabel",
                files: (d) => d.fileCount,
                "time (ms)": (d) => d.mean_ms.toFixed(1),
                "× panache lint": (d) => d.ratio.toFixed(2),
            },
            tip: true,
        }),
    ],
});

Figure 5: Repository-wide linting benchmarks on standard Markdown repos, comparing panache lint to rumdl check, mado check, markdownlint, and markdownlint-cli2. Each dot is one repo/tool pair; the y-axis shows time relative to panache lint within that repo (×, lower is faster).

In Figure 6, we compare linting time across panache and rumdl on tracked .qmd files from several Quarto repositories.

repoQuartoLint = FileAttachment(
    "performance_repo_quarto_lint_data.json",
).json();

repoQuartoLintResults = prepareRepoSuiteRows(repoQuartoLint, {
    panache: "panache lint",
    rumdl: "rumdl check",
});
repoQuartoLintTools = Array.from(
    d3.rollup(
        repoQuartoLintResults,
        (v) => d3.mean(v, (d) => d.ratio),
        (d) => d.toolLabel,
    ),
)
    .sort(([, a], [, b]) => a - b)
    .map(([k]) => k);
repoQuartoLintRepos = repoQuartoLint.repos.map((r) => r.id);
repoQuartoLintRepoMeta = new Map(repoQuartoLint.repos.map((r) => [r.id, r]));

Plot.plot({
    marginLeft: 60,
    marginRight: 20,
    marginBottom: 40,
    marginTop: 20,
    height: 320,
    x: { label: null, domain: repoQuartoLintTools },
    y: {
        label:
            "Mean time relative to panache lint within each repo (×, lower is faster)",
        grid: true,
    },
    color: {
        legend: true,
        domain: repoQuartoLintRepos,
        tickFormat: (id) => repoQuartoLintRepoMeta.get(id)?.name ?? id,
        scheme: "tableau10",
    },
    marks: [
        Plot.ruleY([1], {
            stroke: "#2a9d8f",
            strokeWidth: 2,
            strokeDasharray: "4 3",
        }),
        Plot.dot(repoQuartoLintResults, {
            x: "toolLabel",
            y: "ratio",
            fill: "repo",
            r: 6,
            fillOpacity: 0.85,
            stroke: "white",
            strokeWidth: 1,
            channels: {
                repo: "repoLabel",
                tool: "toolLabel",
                files: (d) => d.fileCount,
                "time (ms)": (d) => d.mean_ms.toFixed(1),
                "× panache lint": (d) => d.ratio.toFixed(2),
            },
            tip: true,
        }),
    ],
});

Figure 6: Repository-wide linting benchmarks on Quarto repos. Each dot is one repo/tool pair; the y-axis shows time relative to panache lint within that repo (×, lower is faster).

Language-Server Speed and Memory

This Linux-only benchmark compares the latency, runtime, and memory use of Panache and Marksman. It opens the five largest tracked Markdown files in a pinned revision of the Rust Book, requests diagnostics and navigation features, and then performs 1,000 fixed-width reference-label renames. Each rename is followed by a definition request, which forces the server to observe the updated label index.

The runner starts three fresh processes per server in alternating order. Each session opens the same files, obtains diagnostics using the server’s advertised pull or push mode, primes navigation requests, and waits for background work to settle. It then times warm requests and the edit workload. The committed JSON contains each run’s measurements; rendering this page never runs the benchmark.

lspMemory = FileAttachment("performance_lsp_memory_data.json").json();

lspMemoryMilestones = [
    { key: "baseline", label: "Baseline" },
    { key: "settled", label: "Files open" },
    { key: "edited", label: "After edits" },
    { key: "peak", label: "Peak" },
];

lspMemoryRows = lspMemory.servers.flatMap((server) =>
    lspMemoryMilestones.map((milestone) => ({
        server: server.label,
        milestone: milestone.label,
        rss: server.aggregate.milestones[milestone.key].rss_mb,
        pss: server.aggregate.milestones[milestone.key].pss_mb,
    })),
);

lspServers = lspMemory.servers.map((server) => server.label);

logTicks = (domain, multipliers) =>
    d3
        .range(
            Math.round(Math.log10(domain[0])),
            Math.round(Math.log10(domain[1])) + 1,
        )
        .flatMap((exponent) => multipliers.map((value) => value * 10 ** exponent))
        .filter((value) => value >= domain[0] && value <= domain[1]);

logAxisX = (domain) => [
    Plot.gridX({ ticks: logTicks(domain, [1]), strokeOpacity: 0.12 }),
    Plot.axisX({
        ticks: logTicks(domain, [1]),
        tickFormat: d3.format(",~g"),
        tickSize: 9,
        tickPadding: 4,
        labelAnchor: "center",
        labelArrow: "none",
        labelOffset: 38,
    }),
    Plot.axisX({
        ticks: logTicks(domain, [5]),
        tickFormat: () => "",
        tickSize: 6,
        label: null,
    }),
    Plot.axisX({
        ticks: logTicks(domain, [2, 3, 4, 6, 7, 8, 9]),
        tickFormat: () => "",
        tickSize: 3,
        label: null,
    }),
];

lspRowLabel = (label) =>
    width < 500
        ? label
                .replace("Document symbols", "Symbols")
                .replace("Go to definition", "Definition")
                .replace("Find references", "References")
                .replace("Edit to definition", "Edit")
                .replace("Workspace ready", "Workspace")
                .replace("Open files ready", "Open files")
                .replace("Edit workload", "Edits")
        : label;

html`<div class="callout-note">
  <div class="callout-header" style="font-weight: 600">Test Environment</div>
  <ul style="margin: 0">
    <li>Generated: ${new Date(lspMemory.meta.generated_at).toUTCString()}</li>
    <li>Host: ${lspMemory.meta.host.cpu} (${lspMemory.meta.host.os}/${lspMemory.meta.host.arch}, ${lspMemory.meta.host.memory_gb} GiB RAM)</li>
    <li>Corpus: ${lspMemory.corpus.name} at ${lspMemory.corpus.revision.slice(0, 12)}; ${lspMemory.session.file_count} open files (${(lspMemory.session.opened_bytes / 1024).toFixed(1)} KiB)</li>
    <li>Workload: ${lspMemory.meta.edit_count.toLocaleString()} edits; ${lspMemory.meta.runs} fresh processes per server</li>
    <li>Warm requests: ${lspMemory.meta.latency_warmups} warmup rounds and ${lspMemory.meta.latency_runs} measured rounds per target in each process</li>
    <li>${lspMemory.servers.map((server) => `${server.label}: ${server.version.replace(/^panache /, "")}`).join(", ")}</li>
  </ul>
</div>`;

Startup and Edit Runtime

Figure 7 shows four elapsed times in milliseconds. Initialize runs from process launch to the initialize response. Workspace ready runs from launch to the start of the final quiet window after initialization. Open files ready runs from the burst of file-open notifications through the initial diagnostics and navigation requests to the next quiet window. Edit workload times all reference-label edits and their definition responses, excluding the final diagnostics and memory-settling wait.

Readiness is estimated from process-tree CPU samples every 150 ms. A quiet window stays below 5% of one CPU core for five seconds; its start is reported only after those five seconds have elapsed. This excludes the deliberate wait from the plotted readiness times, but the estimate still has sampling error.

lspRuntimeMetrics = [
    { key: "init_seconds", label: "Initialize" },
    { key: "workspace_ready_seconds", label: "Workspace ready" },
    { key: "documents_ready_seconds", label: "Open files ready" },
    { key: "edit_work_seconds", label: "Edit workload" },
];

lspRuntimeRows = lspRuntimeMetrics.flatMap((metric) =>
    lspMemory.servers.map((server) => ({
        server: server.label,
        row: metric.label,
        median: server.aggregate.timings[metric.key] * 1000,
        min: d3.min(server.runs, (run) => run[metric.key]) * 1000,
        max: d3.max(server.runs, (run) => run[metric.key]) * 1000,
    })),
);

lspRuntimeDomain = d3
    .scaleLog()
    .domain(d3.extent(lspRuntimeRows.flatMap((d) => [d.min, d.max])))
    .nice()
    .domain();

Plot.plot({
    width,
    marginLeft: width < 500 ? 90 : 130,
    marginRight: 25,
    marginBottom: 50,
    height: 260,
    x: {
        type: "log",
        domain: lspRuntimeDomain,
        label: "Elapsed time (ms, log scale)",
    },
    y: {
        label: null,
        domain: lspRuntimeMetrics.map((d) => d.label),
        tickFormat: lspRowLabel,
    },
    color: { legend: true, domain: lspServers, scheme: "tableau10" },
    marks: [
        ...logAxisX(lspRuntimeDomain),
        Plot.ruleY(lspRuntimeRows, {
            y: "row",
            x1: "min",
            x2: "max",
            stroke: "server",
            strokeWidth: 2,
        }),
        Plot.dot(lspRuntimeRows, {
            y: "row",
            x: "median",
            fill: "server",
            r: 5,
            title: (d) =>
                `${d.row} — ${d.server}\nMedian: ${d.median.toFixed(2)} ms\nRange: ${d.min.toFixed(2)}–${d.max.toFixed(2)} ms`,
            tip: true,
        }),
    ],
});

Figure 7: Language-server startup and edit runtime on a logarithmic scale. Each row compares Panache and Marksman. Dots show medians; lines span the minimum and maximum across fresh processes. Lower is faster.

Request Latency

Figure 8 shows the median and 95th percentile (p95) of serial request round trips over stdio. Each target gets two unmeasured warmup rounds and 20 measured rounds per fresh process. Percentiles pool the measured samples from all three processes. Document-symbol requests cover the three largest open files. Hover, definition, references, and rename use the reference link appended to the largest file. References include the declaration; rename returns edits without applying them.

Edit to definition measures each change notification through its following definition response, including the work needed to observe the edited document. It pools all 1,000 edits per process and has no discarded warmups. Tooltips show sample counts, empty responses, returned item counts, files involved, and result sizes. These help explain differences in the work each server performs. A failed or timed-out request fails the benchmark. Points marked “empty” returned no items in any measured request; they measure the overhead of responding without a result.

lspLatencyRows = lspMemory.servers[0].aggregate.request_latencies.flatMap(
    (request) =>
        lspMemory.servers.map((server) => {
            const result = server.aggregate.request_latencies.find(
                (r) => r.key === request.key,
            );
            return {
                ...result,
                server: server.label,
                row: request.label,
            };
        }),
);

lspLatencyTip = (d) =>
    `${d.row} — ${d.server}\nMedian: ${d.median_ms.toFixed(3)} ms\np95: ${d.p95_ms.toFixed(3)} ms\nSamples: ${d.samples}; empty: ${d.empty_results}\nReturned items: ${d.result_count_min}–${d.result_count_max}\nFiles involved: ${d.result_files_min == null ? "n/a" : `${d.result_files_min}–${d.result_files_max}`}\nMedian result size: ${d.payload_bytes_median} bytes`;

lspLatencyExtent = d3.extent(
    lspLatencyRows.flatMap((d) => [d.median_ms, d.p95_ms]),
);

lspLatencyDomain = d3.scaleLog().domain(lspLatencyExtent).nice().domain();

Plot.plot({
    width,
    marginLeft: width < 500 ? 90 : 130,
    marginRight: 25,
    marginBottom: 50,
    height: 340,
    x: {
        type: "log",
        domain: lspLatencyDomain,
        label: "Request latency (ms, log scale)",
    },
    y: {
        label: null,
        domain: lspMemory.servers[0].aggregate.request_latencies.map(
            (d) => d.label,
        ),
        tickFormat: lspRowLabel,
    },
    color: { legend: true, domain: lspServers, scheme: "tableau10" },
    marks: [
        ...logAxisX(lspLatencyDomain),
        Plot.ruleY(lspLatencyRows, {
            y: "row",
            x1: "median_ms",
            x2: "p95_ms",
            stroke: "server",
            strokeWidth: 2,
        }),
        Plot.dot(lspLatencyRows, {
            y: "row",
            x: "median_ms",
            fill: "server",
            r: 5,
            title: lspLatencyTip,
            tip: true,
        }),
        Plot.text(
            lspLatencyRows.filter((d) => d.empty_results === d.samples),
            {
                y: "row",
                x: "median_ms",
                text: () => "empty",
                fill: "server",
                textAnchor: "start",
                dx: 7,
                dy: -10,
            },
        ),
    ],
});

Figure 8: Language-server request latency on a logarithmic scale. Each row compares Panache and Marksman. Dots show medians; lines extend to p95 and are not confidence intervals. Lower is faster.

Memory

The runner samples the complete process tree’s resident set size (RSS) and proportional set size (PSS) from /proc every 150 ms. RSS counts shared pages in each process; PSS divides them among the processes that map them. Each milestone follows a quiet window, except peak, which covers the whole session. Figure 9 shows median RSS across the three processes, with PSS in the tooltips.

Plot.plot({
    marginLeft: 55,
    marginRight: 20,
    marginBottom: 40,
    marginTop: 25,
    height: 320,
    x: { label: null },
    y: { label: "Median process-tree RSS (MB)", grid: true },
    fx: { label: null, domain: lspMemoryMilestones.map((d) => d.label) },
    color: { legend: true, domain: lspServers, scheme: "tableau10" },
    marks: [
        Plot.barY(lspMemoryRows, {
            fx: "milestone",
            x: "server",
            y: "rss",
            fill: "server",
            channels: { "PSS (MB)": "pss" },
            tip: true,
        }),
        Plot.text(lspMemoryRows, {
            fx: "milestone",
            x: "server",
            y: "rss",
            text: (d) => d.rss.toFixed(1),
            dy: -7,
        }),
        Plot.ruleY([0]),
    ],
});

Figure 9: Median whole-process-tree RSS during the language-server workload. Each milestone is measured after quiescence, except peak, which is the largest 150 ms sample from the complete session.

The comparison measures default architecture, not identical internal work. Marksman indexes the Markdown workspace eagerly, while Panache analyzes open documents and discovers project relationships lazily. Both servers receive the same open, edit, and navigation sequence, while diagnostics use each server’s advertised pull or push mode. Their memory totals reflect those different strategies.

Quarto Language Server

This separate Linux benchmark compares Panache with q2 lsp, the experimental Rust Quarto language server. It uses three authoring guides from a pinned q2 revision: computations, Markdown basics, and title blocks. The files live in a standalone project with no workspace index to discover. Both servers must accept every document without error diagnostics and return a nonempty outline.

Each run opens the files serially, measures document-symbol requests, and performs 100 full-document replacements that change an appended heading in the computations guide. Full replacements accommodate q2’s synchronization mode. A final outline request must contain the last edited heading. This track exercises diagnostics and symbols; q2 does not implement the navigation features used in the Markdown comparison above.

Panache uses pull diagnostics, with a request immediately after each open or change. q2 pushes diagnostics after processing the notification. The timings include those different delivery paths. They compare editor-visible latency under these client policies, with each server’s own checks, and do not measure Panache’s debounced push mode. Empty diagnostic sets are successful analyses; empty symbol results, protocol errors, and timeouts fail the benchmark.

quartoLsp = FileAttachment("performance_lsp_quarto_data.json").json();

quartoLspLatencies = quartoLsp.servers.flatMap((server) =>
    server.aggregate.request_latencies.map((metric) => ({
        server: server.label,
        metric: metric.label,
        median: metric.median_ms,
        p95: metric.p95_ms,
        samples: metric.samples,
        results: metric.result_count_median,
    })),
);

html`<div class="callout-note">
  <div class="callout-header" style="font-weight: 600">Test Environment</div>
  <ul style="margin: 0">
    <li>Generated: ${new Date(quartoLsp.meta.generated_at).toUTCString()}</li>
    <li>Host: ${quartoLsp.meta.host.cpu} (${quartoLsp.meta.host.os}/${quartoLsp.meta.host.arch}, ${quartoLsp.meta.host.memory_gb} GiB RAM)</li>
    <li>Corpus: ${quartoLsp.corpus.revision.slice(0, 12)}; ${quartoLsp.session.file_count} files (${(quartoLsp.session.opened_bytes / 1024).toFixed(1)} KiB)</li>
    <li>Workload: ${quartoLsp.meta.edit_count} edits; ${quartoLsp.meta.runs} fresh processes per server</li>
    <li>${quartoLsp.servers.map((server) => `${server.label}: ${server.version}; ${server.runs[0].diagnostic_mode} diagnostics`).join(". ")}</li>
  </ul>
</div>`;

Figure 10 pools samples across the three processes. File opens have no warmups. Symbol requests have two warmup rounds and 20 measured rounds per file per process. Each edit waits for diagnostics before the next begins.

Plot.plot({
    width,
    height: 230,
    marginLeft: width < 500 ? 80 : 140,
    marginRight: 30,
    x: { type: "log", label: "Time (ms)", grid: true },
    y: {
        label: null,
        domain: ["Open to diagnostics", "Document symbols", "Edit to diagnostics"],
        tickFormat: (label) =>
            width < 500
                ? {
                        "Open to diagnostics": "Open",
                        "Document symbols": "Symbols",
                        "Edit to diagnostics": "Edit",
                    }[label]
                : label,
    },
    color: { legend: true, scheme: "tableau10" },
    marks: [
        Plot.ruleY(quartoLspLatencies, {
            y: "metric",
            x1: "median",
            x2: "p95",
            stroke: "server",
        }),
        Plot.dot(quartoLspLatencies, {
            y: "metric",
            x: "median",
            fill: "server",
            r: 5,
            channels: {
                "p95 (ms)": "p95",
                samples: "samples",
                "results (median)": "results",
            },
            tip: true,
        }),
    ],
});

Figure 10: Quarto LSP latency. Dots show medians; lines extend to p95. Tooltips include sample counts and the median number of returned diagnostics or symbols.

Initialization measures process launch through the initialize response. Memory uses the same process-tree sampler and five-second quiet windows as the Markdown track. q2 performs open-document parsing and outline analysis; Panache also retains its syntax trees and analysis database. The table reports medians across processes. Neither server builds a whole-project index in this standalone workload.

quartoLspResourceMetrics = [
    [
        "Initialize (ms)",
        (s) => (s.aggregate.timings.init_seconds * 1000).toFixed(2),
    ],
    ["Baseline RSS (MB)", (s) => s.aggregate.milestones.baseline.rss_mb],
    ["Files open RSS (MB)", (s) => s.aggregate.milestones.settled.rss_mb],
    ["After edits RSS (MB)", (s) => s.aggregate.milestones.edited.rss_mb],
    ["After edits PSS (MB)", (s) => s.aggregate.milestones.edited.pss_mb],
    ["Peak RSS (MB)", (s) => s.aggregate.milestones.peak.rss_mb],
];

html`<table class="table">
  <thead><tr><th>Metric</th>${quartoLsp.servers.map((s) => html`<th>${s.label}</th>`)}</tr></thead>
  <tbody>${quartoLspResourceMetrics.map(
        ([label, value]) =>
            html`<tr><td>${label}</td>${quartoLsp.servers.map((s) => html`<td>${value(s)}</td>`)}</tr>`,
    )}</tbody>
</table>`;

Reproducing

All benchmarks are reproducible. The formatting and linting comparisons require hyperfine; the language-server memory and speed comparisons require Linux with /proc. The Markdown track needs Marksman; the Quarto track downloads a pinned q2 release and verifies its SHA-256 hash, or uses the executable specified by Q2_BIN. The scripts in benches/ are designed to be run from the repository root:

# Download test documents (idempotent)
cd benches/documents && ./download.sh && cd ../..

# Outside devenv: install Yamark and record its version
# The development environment already provides both.
uv tool install yamark==0.3.0
export PANACHE_BENCH_YAMARK_VERSION=0.3.0

# Run comparison benchmark and write JSON
bash benches/compare_all.sh --json --out docs/guide/performance_data.json

# Run repository-wide Markdown formatting benchmark
bash benches/compare_repo_suite.sh --mode format --track markdown --out docs/guide/performance_repo_markdown_format_data.json

# Run repository-wide Quarto formatting benchmark
bash benches/compare_repo_suite.sh --mode format --track quarto --out docs/guide/performance_repo_quarto_format_data.json

# Run repository-wide Markdown lint benchmark
bash benches/compare_repo_suite.sh --mode lint --track markdown --out docs/guide/performance_repo_markdown_lint_data.json

# Run repository-wide Quarto lint benchmark
bash benches/compare_repo_suite.sh --mode lint --track quarto --out docs/guide/performance_repo_quarto_lint_data.json

# Run per-document lint benchmark
bash benches/compare_lint_single.sh --out docs/guide/performance_lint_single_data.json

# Run the Linux language-server speed and memory benchmark
# bench:lsp-memory remains an alias.
task bench:lsp

# Run the Quarto diagnostics, symbols, and memory comparison
task bench:lsp-quarto

# Or run the human-readable text variant
bash benches/compare_all.sh

# Re-render this page (uses freeze cache by default)
quarto render docs/guide/performance.qmd

# Pick up refreshed JSON by invalidating the rendered page's freeze cache
rm -rf docs/_freeze/guide/performance
quarto render docs/guide/performance.qmd