When Rust Meets Transcript Quantification: A Reproducibility Benchmark of a Local-First Bioinformatics SDK

Linxira Bio SDK benchmark report · 2026-09-14 (§5.4 added 2026-09-15: a 26-sample deep-library backfill batch) Author: Linxira-OS project maintainer · License: AGPL-3.0-or-later (code) / CC-BY-4.0 (this article) Repository: https://github.com/Linxira-OS/linxira-bio-sdk

Product page: Linxira Bio SDK Self-assessment: this is an author-reported reproduction report, not independently verified by a third party. Self-assessed against the REFORMS checklist (a 32-item reporting standard for ML-based science, Science Advances 2024): 23 items met, 9 not applicable by study design (no ML modeling task here, so items on model selection, loss functions, data leakage, and statistical tests are naturally waived), 0 unmet. Third-party replication is planned as follow-up work.

Abstract

We ran two kinds of measurements on Linxira Bio SDK, a local-first bioinformatics execution toolkit: (1) three-backend consistency — whether independent Rust / Python / R implementations of the same algorithm produce identical results on identical input; and (2) exact reproduction of a reference pipeline — whether the SDK, orchestrating upstream salmon 2.7.0 on real public RNA-seq data, reproduces an existing quantification pipeline transcript by transcript.

Results: the three backends agree field by field within a 1e-6 tolerance (bit-identical on several items); on 10 normally covered paired-end samples, TPM Pearson r = 1.000000 (σ = 0), and NumReads per transcript is identical to the reference pipeline (relative difference = 0), with every metric passing a 3σ check (zero outliers). On top of that, we used the same validated pipeline to backfill 26 samples the existing pipeline had not yet computed (~520 million reads, every output carrying the complete 33,955-transcript ID set); the batch crossed a real scheduled reboot and auto-resumed 20 seconds after boot, finishing with zero manual intervention. This report discloses all data sources (public SRA accessions), the hardware and software environment, the methodology, and the reproduction commands.

1. Background

The core design of Linxira Bio SDK is “a deterministic Rust engine + controlled invocation of native tools + three-way cross-validation”. For statistical analyses the engine is implemented in Rust, and the benchmark pack ships independent Python and R implementations of the same algorithm. For ecosystem-level tools (salmon, fasterq-dump, kraken2, and so on), the engine performs controlled, shell-free orchestration and reduces their output to verifiable structured results.

This round answers two questions:

1. Can the three implementations corroborate each other? (trustworthiness)
2. On real data, with exactly the same parameters as an existing pipeline, can the SDK reproduce its output? (correctness)

2. Data Sources (all public)

Raw reads and the reference index stay on the lab workstation and are not committed; this report and the repository carry only metrics, methodology, and provenance.

3. Environment Disclosure (complete)

3.1 Batch-validation workstation (Linux)

3.2 Three-backend baseline environment (Windows + WSL2)

3.3 Software references

• Patro R, Duggal G, Love MI, Irizarry RA, Kingsford C. Salmon provides fast and bias-aware quantification of transcript expression. Nature Methods 14, 417–419 (2017).
• Leinonen R, Sugawara H, Shumway M; INSDC. The Sequence Read Archive. Nucleic Acids Research 39, D19–D21 (2011). (SRA Toolkit / fasterq-dump)
• Cock PJA et al. Biopython. Bioinformatics 25, 1422–1423 (2009).
• Page AJ et al. Biostrings. Bioconductor (used by the R reference implementation).
• Key Rust crates: flate2 (zlib-ng backend planned), csv, serde, rayon (roadmap in the repository’s docs/RUST_NATIVE_ROADMAP.md).

4. Methods

4.1 Three-backend consistency (bench-20260913-001)

Independent implementations of the same algorithm in three languages (Rust engine / Python pack / R pack), identical input, one warm-up per backend, then 5 timed runs: median wall time and peak RSS (/usr/bin/time -v). Consistency criterion: structured field-level diff with a numeric relative tolerance of 1e-6.

4.2 Exact reproduction of the reference pipeline (bench-20260914 series)

The complete flow for each sample (entirely through the public CLI, no internal shortcuts):

# 1) SRA → FASTQ (decompression segment timed)
fasterq-dump -e 8 --force -O <tmp>/<run>-fq <run>.sra
# 2) SDK orchestrates salmon quantification (quantification segment timed;
#    parameters identical to the reference pipeline)
export LINXIRA_BIO_SALMON=/path/to/salmon
linxira-bio expression quantify <run>_1.fastq <run>_2.fastq \
  --index <salmon_idx> --threads 8 --seq-bias --gc-bias \
  --output <results>/<run>/quant.sf --json

3σ acceptance rule: for each passing batch, compute mean/σ/3σ interval and outlier count per metric; a batch PASSES only with zero 3σ outliers and a worst-case TPM r ≥ 0.995.

5. Results

5.1 Three-backend consistency (repository fixtures, bench-20260913-001)

All four capabilities are Consistent across the three backends (against the Rust golden: PDB bit-identical at 0.0; PCA/Venn maximum relative error 2.26e-11). On small inputs the wall clock is dominated by interpreter start-up — which is exactly what this baseline characterizes; real-data throughput is covered in the next section.

5.2 Bit-level reproduction of the reference pipeline (10-sample batch, bench-20260914-003/004)

Transcript-by-transcript comparison against the existing pipeline’s quant.sf (parameters fully aligned: -l A -p 8 --validateMappings --seqBias --gcBias, same salmon 2.7.0, same index):

Batch-level 3σ acceptance: PASS (10/10 passed; zero outliers; worst r = 1.000000).

Per-run detail (decompression/quantification seconds; CPU utilization = (user+sys)/wall):

The first same-parameter control (SRR1460477, 16.7M mapped reads) likewise gave r = 1.000000 with identical NumReads per transcript.

5.3 Deployment validation and data guards

• A deep-library paired-end sample (SRA 3.68 GB → 2×9.07 GB FASTQ) ran end to end; the output quant.sf matches the reference format item by item (row count / IDs / header).
• Truncation guard: a truncated FASTQ whose line count is not a multiple of 4 is rejected with a structured error (zero silent output). This is not a theoretical concern — an interrupted transfer produced exactly such a file, and it had caused the existing pipeline to fail repeatedly on the same run.
• Automatic exposure of non-mRNA libraries: two independent implementations measured 0–0.0019% mapping rates on 14 samples, confirming small-RNA libraries (which belong on a miRNA track); they are annotated excluded: non-mRNA library. Two independent implementations producing the same “near-zero mapping” signal on the same data is a direct demonstration of the value of cross-validation.

5.4 Deep-library backfill batch (26 samples, ~520M reads, one real reboot)

From 2026-09-14 to 09-15, using exactly the same pipeline and parameters as §5.2 (same salmon 2.7.0, same index, -l A -p 8 --validateMappings --seqBias --gcBias, 8 threads pinned to cores 8–15), we backfilled 26 paired-end samples from the same 172-run study that the existing pipeline had not yet computed. These samples have no existing output to compare against — they are precisely the part waiting to be computed — so the acceptance criteria were threefold: ① complete output row count and transcript ID set (33,955/33,955); ② the pipeline itself had already achieved bit-level reproduction with r = 1.000000 on comparable samples in §5.2; ③ structured logging throughout (per-run quantify.json / logs / CPU accounting).

Batch results: 26/26 passed, every quant.sf exactly 33,955 rows; total decompression 8,891 s and quantification 6,551 s (≈4.3 hours of pure compute, excluding NAS pulls); median quantification CPU utilization 681% (8 threads nearly saturated). Per-run detail (reads = NumReads total, util = (user+sys)/wall):

† near-empty library edge case, see observation 2 below.

Observation 1: CPU-seconds are far more stable than wall time — which is exactly why CPU accounting is recorded per run. The same group of deep-library samples (SRR1904944x, 16.9–18.5 M reads) completed under three load conditions: concurrent with the main analysis workload (util 68–96%), quantification wall time was 806–1,063 s; after the reboot, with near-exclusive use of the pinned cores (util 659–694%), comparable samples took only 72–154 s; an intermediate state (SRR243 group, util ~213–241%) fell in between. But converted to CPU time (user+sys), samples of this size converge to roughly 475–772 CPU·s, while wall time spreads over 15× (72–1,063 s). The conclusion matches the honest boundaries in §5.5: in this mechanical-disk + shared-workstation deployment environment, wall time describes I/O and concurrency, while CPU-seconds describe the computation itself. Any wall-time comparison across load states should start from the util column.

Observation 2: the empty-library edge case was exposed automatically again. SRR24322347 has only 16,927 reads and 4,257 expressed transcripts (12.5% of the index) — a nearly idle library. Like the non-mRNA case in §5.3, it was not silently folded into any mean; it remains a separate line in the ledger for downstream researchers to judge, and its 64 s quantification time is proportional to its read count — the processing itself was normal.

Automatic resumption across a real reboot. The batch deliberately crossed the host’s daily 06:50 scheduled reboot: the machine rebooted at 06:50:30, and at 06:50:50 (20 seconds after boot) a user-level systemd oneshot unit automatically triggered the resume — completed samples were skipped by output existence (4), the remaining 22 continued in order, and the batch finished at 09:24:18 with zero manual intervention. Before starting, the resume entry cleans two kinds of interruption debris: half-decompressed FASTQ files in tmp, and partial writes where “quant.sf exists but the CSV has no final row” (preventing incomplete artifacts from being mistaken for completed ones). Each sample directory includes a README (new_computation annotation: source accession, quantifier and parameters, verification records); the batch ledger (per-run CPU model / pinned cores / threads / utilization) is in bench/tier26-benchmark.csv.

5.5 Honest boundaries (what we do not claim)

1. We do not claim to “beat salmon/fasterq-dump themselves” — the SDK orchestrates the very same engines; under identical parameters we aim for bit-level reproduction (achieved), not overtaking.
2. The wall-clock times in this batch are bounded by mechanical-disk throughput (the benchmark I/O volume is a 500GB 7200rpm HDD); they characterize the deployment environment and are not an engine speed claim.
3. Zero-mapping samples are retained as edge cases; TPM comparisons on them are noise against noise, any implementation would “fail”, and they are not used for scoring.
4. Numbers across machines, cache states (warm/cold), or code versions (each report embeds its git sha) are never mixed.
5. The backfilled samples in §5.4 have no existing output to compare against (which is why they needed backfilling); their correctness rests on the pipeline’s bit-level reproduction on comparable samples and on complete ID-set verification, not per-run correlation coefficients. The batch’s wall times are likewise affected by concurrent load and mechanical-disk I/O and are for deployment planning only.

6. Reproduction

The SDK and all benchmark tooling are open source:

# Three-backend baseline (any supported platform)
linxira-bio benchmark run sequence.stats.v1 \
  fasta=tests/fixtures/sequences/tiny.fa \
  --backends rust,python,r --repeat 5 --dataset-class sequence

# Real sample: SRA → quant.sf (public CLI; both timings are in the --json envelope)
export LINXIRA_BIO_SALMON=/path/to/salmon
fasterq-dump -e 8 --force -O tmp/run-fq SRR1460477.sra
linxira-bio expression quantify tmp/run-fq/SRR1460477_1.fastq \
  tmp/run-fq/SRR1460477_2.fastq --index <salmon_idx> --threads 8 \
  --seq-bias --gc-bias --output results/SRR1460477/quant.sf --json

# Batch comparison and 3σ summary (scripts/tier23-bench.sh + tier23-summarize.py)

# Batch backfill: checkpoint resume (completed samples auto-skipped by artifacts),
# per-run CPU accounting into a CSV ledger
scripts/tier23-bench.sh --cli <linxira-bio> --index <salmon_idx> \
  --sra-dir <inbox> --reference-dir <ref_quant> --output-dir <results> \
  --tmp-dir <tmp> --csv <ledger.csv> --pin "taskset -c 8-15 nice -n 10" \
  --cores 8 --threads 8 --pull-source <nas>:<tier23> --scp-identity <key> \
  --fasterq <fasterq-dump> --runs <RUN...>

The raw reports (per-run JSON envelopes, per-line CPU accounting, 3σ statistics objects) are in the repository under benchmark-results/2026-09-13|14/.

7. Closing

The first question a “local-first” bioinformatics toolkit must answer is not “how fast” but “is the result correct, and can others trust it”. This round answers: three languages corroborate the same algorithm; the existing pipeline is reproduced bit for bit under identical parameters; data anomalies are rejected loudly instead of quietly producing numbers; and a 4.3-hour backfill batch auto-resumed 20 seconds after a scheduled reboot and ran to completion with zero manual intervention. The speed figures (7.5–8.25× versus interpreters) came along for free — verifiability is the product.

Linxira-OS · AGPL-3.0-or-later (code) · this article CC-BY-4.0 · https://github.com/Linxira-OS/linxira-bio-sdk