Smoke Test Process

This document describes the smoke test process for the miniextendr project. It covers the full “demanding” smoke lane – the most thorough validation pass, intended for release gates and nightly CI runs.

The process below is the canonical reference for the demanding smoke lane.

🔗Overview

The smoke test validates the most failure-prone integration paths across the miniextendr ecosystem:

1. The example miniextendr package (rpkg/) can configure, build, install, and run core runtime paths in both dev and CRAN-like modes.
2. minirextendr can scaffold working projects (standalone + monorepo) that actually build and execute Rust-backed R functions.
3. Cross-package trait ABI interoperability remains intact (producer.pkg / consumer.pkg).
4. Template/vendor/configure drift is caught early before it causes release damage.

🔗Smoke Lanes

Lane	Scope	Time
quick	PR developer sanity	~15-25 min
standard	Pre-merge gate	~35-55 min
demanding	Release / nightly (this doc)	~90-150 min

🔗Risk Coverage

Risk	Covered By
Configure/autoconf breakage	Phases A2, B3, B4
Vendoring / CRAN mode divergence	Phases A4, C2
Rust/R wrapper desync	Phases A2, B3, B4
Thread/panic/GC regressions	Phase A3 targeted filters
Feature adapter drift	Phase A3 feature filters
Trait ABI interop drift	Phase A5, C3
Template drift (rpkg vs templates)	Phase A1, B2/B4
minirextendr API regressions	Phase B1, B5

---

🔗Prerequisites

Before running any smoke test phases, the following manual steps must be completed. These are NOT automated by minirextendr or the justfile.

🔗1. Install minirextendr

Required before any B-phase (scaffolding) tests can run:

just minirextendr-install
# or equivalently:
Rscript -e 'devtools::install("minirextendr")'

🔗2. Configure the example package (rpkg/)

Required before A2+ phases. If templates have drifted (e.g., after editing rpkg/configure.ac), approve them first:

just templates-approve   # only if templates-check fails
just configure           # generates Makevars, cargo config, etc.

🔗3. Keep templates patch in sync

After any change to files tracked by just templates-sources (notably rpkg/configure.ac, rpkg/src/Makevars.in, rpkg/src/rust/build.rs), the templates patch must be regenerated:

just templates-approve

Otherwise just templates-check will fail in Phase A1.

🔗4. Update all references when renaming functions

When exported R functions are renamed (e.g., miniextendr_check to miniextendr_validate), every test file referencing the old name must be updated. This includes:

• rpkg/R/miniextendr-wrappers.R (regenerated by just devtools-document)
• rpkg/NAMESPACE (regenerated by just devtools-document)
• All files under rpkg/tests/testthat/ and minirextendr/tests/testthat/

🔗5. Toolchain requirements

• Rust >= 1.85
• autoconf available on PATH
• R toolchain capable of compiling package shared libraries
• R packages: devtools, roxygen2, rcmdcheck, testthat, usethis, withr, desc, R6, S7, vctrs

Install R dependencies with:

just install_deps
just minirextendr-install-deps

---

🔗Execution Guide

🔗Environment Setup

Run from the repository root:

set -euo pipefail

export MX_ROOT="$(pwd)"
export MX_SMOKE_ROOT="$(mktemp -d "${TMPDIR:-/tmp}/mx-smoke-XXXXXX")"
export MX_ARTIFACTS="$MX_SMOKE_ROOT/artifacts"
mkdir -p "$MX_ARTIFACTS"

echo "MX_ROOT=$MX_ROOT"
echo "MX_SMOKE_ROOT=$MX_SMOKE_ROOT"
echo "MX_ARTIFACTS=$MX_ARTIFACTS"

# Record tool versions
R --version | tee "$MX_ARTIFACTS/r-version.txt"
rustc --version | tee "$MX_ARTIFACTS/rustc-version.txt"
cargo --version | tee "$MX_ARTIFACTS/cargo-version.txt"
autoconf --version | head -n 1 | tee "$MX_ARTIFACTS/autoconf-version.txt"

🔗Phase A1: Repo Invariants and Sync

Verifies that templates and vendored crates have not drifted from their sources. Hard fail – if these break, nothing downstream is trustworthy.

just templates-check
just vendor-sync-check

Pass criteria:

1. Both commands exit 0.
2. No unexpected drift requiring templates-approve during the smoke run.

Note: The plan file references just lint-sync-check – this recipe does not exist. Skip it.

🔗Phase A2: Dev-Mode Configure / Build / Install Baseline

just configure
just r-cmd-install        # compiles Rust code -- needs dangerouslyDisableSandbox in Claude Code
just devtools-test FILTER=basic
just devtools-test FILTER=conversions

Pass criteria:

1. configure generates src/Makevars.
2. R CMD INSTALL rpkg succeeds.
3. Both test filters pass with no failures.

🔗Phase A3: High-Risk Runtime Filters

Run after A2 with the example package (rpkg/) installed. These target the most crash-prone subsystems.

just devtools-test FILTER=gc-stress
just devtools-test FILTER=panic
just devtools-test FILTER=thread
just devtools-test FILTER=worker
just devtools-test FILTER=trait-abi
just devtools-test FILTER=externalptr
just devtools-test FILTER=class-systems
just devtools-test FILTER=altrep
just devtools-test FILTER=serde_r
just devtools-test FILTER=feature-adapters
just devtools-test FILTER=rayon

Test file mapping (verify with ls rpkg/tests/testthat/):

Filter	Matches
gc-stress	test-gc-stress.R (also test-gc-protect.R)
panic	test-panic.R
thread	test-thread.R, test-thread-broken.R
worker	test-worker.R
trait-abi	test-trait-abi.R
externalptr	test-externalptr.R, test-externalptr-main.R
class-systems	test-class-systems.R, test-class-system-matrix.R
altrep	test-altrep.R, test-altrep-*.R (multiple files)
serde_r	test-serde_r.R
feature-adapters	test-feature-adapters.R
rayon	test-rayon.R

Pass criteria:

1. No test failures.
2. No crashes or segfaults.
3. No hangs beyond timeout budget (suggested: 20 min cap for this phase).

🔗Phase A4: CRAN-Like Tarball Path

Validates the path where vendor/ is absent and inst/vendor.tar.xz is authoritative – the way CRAN sees the package.

just vendor
just r-cmd-check

The r-cmd-check recipe depends on vendor and runs rcmdcheck::rcmdcheck() with --as-cran --no-manual. It uses the rcmdcheck R package.

Alternatively, for manual control:

NOT_CRAN=true just configure
rm -rf rpkg/vendor
R CMD build rpkg
TARBALL="$(ls -1 miniextendr_*.tar.gz | tail -n1)"
R CMD check --as-cran --no-manual "$TARBALL"

Pass criteria:

1. Tarball builds.
2. Check completes without errors.
3. Configure/unpack works with only inst/vendor.tar.xz.

🔗Phase A5: Cross-Package Trait ABI Smoke

just cross-clean
just cross-configure
just cross-install
just cross-test

Pass criteria:

1. producer.pkg and consumer.pkg both install.
2. Consumer tests prove trait-dispatch against producer objects.

🔗Phase B1: minirextendr Package Tests and Check

just minirextendr-install   # prerequisite!
just minirextendr-test
just minirextendr-check

Pass criteria:

1. testthat suite passes.
2. devtools::check passes with no errors.

🔗Phase B2: Scaffolding Smoke – Standalone Package

Creates a real standalone R package from the template and builds it:

Important: Dev mode requires install → document → reinstall. The first R CMD INSTALL compiles Rust and generates R wrappers via the document binary, but NAMESPACE starts empty. devtools::document() runs roxygen2 to populate NAMESPACE with exports, then a second install picks up the updated exports. miniextendr_build(install = TRUE) goes through R CMD build first, which excludes vendor/ (via .Rbuildignore), so installing from the tarball fails in dev mode. Use direct R CMD INSTALL on the source directory instead.

Rscript - <<'EOF'
library(minirextendr)
library(usethis)
library(withr)
library(devtools)
library(desc)

root <- Sys.getenv("MX_ROOT")
tmp <- Sys.getenv("MX_SMOKE_ROOT")
pkg <- file.path(tmp, "standalone.smoke")
r_cmd <- file.path(R.home("bin"), "R")
lib <- file.path(tmp, "r-lib-standalone")
dir.create(lib, recursive = TRUE, showWarnings = FALSE)

create_package(pkg, open = FALSE)
proj_set(pkg, force = TRUE)
use_miniextendr(template_type = "rpkg", local_path = root)

# 1. First install: compiles Rust + generates R wrappers via document binary
system2(r_cmd,
  c("CMD", "INSTALL", "--no-multiarch", "-l", lib, pkg),
  env = "NOT_CRAN=true")

# 2. Document: populates NAMESPACE with exports from generated wrappers
devtools::document(pkg)

# 3. Reinstall with updated NAMESPACE
system2(r_cmd,
  c("CMD", "INSTALL", "--no-multiarch", "-l", lib, pkg),
  env = "NOT_CRAN=true")

# 4. Verify
pkg_name <- desc(file.path(pkg, "DESCRIPTION"))$get_field("Package")
with_libpaths(lib, action = "prefix", {
  library(pkg_name, character.only = TRUE)
  stopifnot(add(2, 3) == 5)
  stopifnot(hello("smoke") == "Hello, smoke!")
})
EOF

Pass criteria:

1. Generated package configures and installs.
2. Generated Rust wrappers are usable from R (add(2, 3) == 5).
3. Dev mode workflow: install → document → reinstall succeeds.

🔗Phase B3: External Dependency + Re-Vendor

In the standalone smoke package from B2, test adding a cargo dependency and re-vendoring. See the plan file for the full Rscript commands using cargo_add("itertools@0.13").

🔗Phase B4: Scaffolding Smoke – Monorepo Template

Creates a monorepo-style project and validates both the R package and the main Rust crate compile. See the plan file for the full Rscript and cargo check commands.

🔗Phase B5: API Helper Smoke

Validates high-touch helper APIs (miniextendr_status(), miniextendr_check_rust(), use_rayon(), use_serde(), use_vctrs(), use_feature_detection(), cargo_check(), cargo_test(), cargo_fmt(), cargo_clippy()) in a scaffolded project. See the plan file for details.

🔗Phase C: Failure-Injection Smoke (Demanding Lane Only)

These checks verify diagnostics and recovery paths:

• C1: Stale generated-file detection – Touch .in source, verify miniextendr_status() reports staleness, then re-configure to clear it.
• C2: Vendor fallback behavior – Remove vendor/, configure in CRAN-like mode, verify inst/vendor.tar.xz restores usable sources.
• C3: Cross-package install order constraint – Attempt consumer build before producer install, verify meaningful failure, then build in correct order.

---

🔗Known Issues from First Run (2026-02-09)

🔗Cosmetic: config.status “command not found” noise

When ./configure runs, config.status dumps the environment and prints hundreds of lines like:

/path/to/config.status: line 123: SOME_VAR: command not found

This is harmless cosmetic noise from autoconf’s environment dump, not real errors. It does not affect the configure output.

🔗just templates-check fails after example package changes

Any change to files tracked by just templates-sources (e.g., rpkg/configure.ac, rpkg/src/Makevars.in) will cause just templates-check to fail until just templates-approve is run to regenerate the approved patch.

Fix: Run just templates-approve after making changes, before running the smoke test.

🔗Function renames require test updates

Renaming an exported function (e.g., miniextendr_check to miniextendr_validate) breaks every test that calls the old name. R tests fail silently with “could not find function” rather than a clear rename error.

Fix: After renaming, grep all test files for the old name:

grep -r "old_function_name" rpkg/tests/ minirextendr/tests/

🔗just lint-sync-check does not exist

The plan file references just lint-sync-check in Phase A1. This recipe was never implemented. Skip it – the lint itself runs via just lint and checks #[miniextendr] source-level attribute consistency, which is a separate concern from sync checking.

🔗Test filter names may not exactly match

The FILTER argument to just devtools-test is passed to devtools::test(filter = ...) which uses regex matching against test file names (without the test- prefix or .R suffix). Some filters may match multiple files. For example:

• FILTER=altrep matches test-altrep.R, test-altrep-builtins.R, test-altrep-helpers.R, test-altrep-serialization.R, etc.
• FILTER=thread matches both test-thread.R and test-thread-broken.R.

To see the full list of available test files:

ls rpkg/tests/testthat/test-*.R

---

🔗CI Integration Notes

🔗Phase Timing Estimates

Phase	Time	Notes
A1 (repo sync)	~1 min	No compilation
A2 (dev baseline)	~5 min	Rust compile + basic R tests
A3 (high-risk runtime)	~20 min	11 test filter groups
A4 (CRAN-like tarball)	~30 min	Full R CMD check --as-cran
A5 (cross-package)	~10 min	Two packages built + tested
B1 (minirextendr tests)	~5 min	Pure R, no Rust compilation
B2-B4 (scaffolding)	~15-20 min	Template instantiation + build
B5 (API helpers)	~5 min	Helper validation
C (failure injection)	~10 min	Diagnostic/recovery paths

🔗Phased CI Adoption

1. Step 1 (immediate): Add Linux demanding smoke job on manual trigger + nightly schedule.
2. Step 2: Add macOS demanding subset (A2, A4, B1, B2).
3. Step 3: Promote Linux demanding smoke to required status for release branches/tags.
4. Step 4: Stabilize Windows and remove continue-on-error once repeated green baseline is established.

🔗Environment Matrix (Full Demanding)

Platform	R Version	Status
Linux	release	Required
Linux	oldrel-1	Required
macOS arm64	release	Required
macOS x86_64	release	Required
Windows (GNU)	release	Non-blocking until stabilized

🔗Mode Matrix

Each platform runs in two modes:

1. Dev mode: NOT_CRAN=true – cargo resolves deps via [patch], no vendoring.
2. CRAN-like mode: NOT_CRAN unset – install and check from built tarball using inst/vendor.tar.xz.

---

🔗Quick Reference: Just Recipes Used

Recipe	Phase	What It Does
just templates-check	A1	Verify templates match rpkg + approved patch
just templates-approve	prereq	Accept current template delta
just vendor-sync-check	A1	Verify vendored crates match workspace
just configure	A2	Generate build config (dev mode)
just r-cmd-install	A2	R CMD INSTALL rpkg
just devtools-test FILTER=X	A2, A3	Run R tests matching filter
just vendor	A4	Package deps into inst/vendor.tar.xz
just r-cmd-check	A4	rcmdcheck with --as-cran
just cross-clean	A5	Clean cross-package build artifacts
just cross-configure	A5	Configure both cross-package packages
just cross-install	A5	Build + install producer.pkg and consumer.pkg
just cross-test	A5	Run cross-package tests
just minirextendr-install	B1 prereq	Install minirextendr
just minirextendr-test	B1	Run minirextendr testthat suite
just minirextendr-check	B1	devtools::check on minirextendr
just lint	optional	Check #[miniextendr] source-level attributes