aboutsummaryrefslogtreecommitdiff
path: root/docs/coverage.md
diff options
context:
space:
mode:
authorAlexander M Pickering <alex@cogarr.net>2026-07-21 20:19:46 -0500
committerAlexander M Pickering <alex@cogarr.net>2026-07-21 20:19:46 -0500
commit657fb0007f39f07cc0401e0c5d03e25df6234aa4 (patch)
treef73fe23232dc80802f39caed738c38464a60ec6d /docs/coverage.md
downloadtrbldoc-657fb0007f39f07cc0401e0c5d03e25df6234aa4.tar.gz
trbldoc-657fb0007f39f07cc0401e0c5d03e25df6234aa4.tar.bz2
trbldoc-657fb0007f39f07cc0401e0c5d03e25df6234aa4.zip
Inital commit.
Diffstat (limited to 'docs/coverage.md')
-rw-r--r--docs/coverage.md55
1 files changed, 55 insertions, 0 deletions
diff --git a/docs/coverage.md b/docs/coverage.md
new file mode 100644
index 0000000..c6371d0
--- /dev/null
+++ b/docs/coverage.md
@@ -0,0 +1,55 @@
+# Test-suite coverage
+
+`scripts/coverage-tests.sh` reports which lines of trbldoc's shell sources the
+test suite actually executes.
+
+## Usage
+
+```sh
+# Coverage of trbldoc.sh from the integration suite (default)
+sh scripts/coverage-tests.sh
+
+# Whole suite, also cover a resolver, fail under 70%
+sh scripts/coverage-tests.sh -s all -t "trbldoc.sh ref_resolvers/lua.sh" -m 70
+```
+
+Options: `-s all|unit|integration`, `-o OUTDIR`, `-t "file.sh ..."` (targets),
+`-m MIN_PERCENT` (exit non-zero below this; default `0` = no gate).
+
+## Output
+
+Written to `reports/coverage-<timestamp>/` (or `-o`):
+
+- `index.html` — per-file coverage with bars and an overall percentage.
+- `html/<file>.html` — annotated source; green = executed, red = missed,
+ grey = non-executable.
+- `summary.csv` — `file,executable,covered,percent`.
+- `covered.tsv` — raw union of executed `file,line` pairs.
+
+## How it works
+
+The bats tests run the tool as `bash "$TRBLDOC"`. Coverage exports
+`BASH_ENV=scripts/lib/trace-init.sh`, so each target process enables `set -x`
+with a `PS4` that stamps `BASH_SOURCE:LINENO` to a per-PID trace file. After
+the run, the union of executed lines is compared against the executable lines
+in each target source to compute coverage. No changes to `trbldoc.sh` or the
+tests are needed.
+
+Only scripts whose basename is listed in `-t` are traced, so bats internals
+and the `md2html` stub stay out of the report.
+
+## Determining "executable" lines
+
+The denominator is a heuristic: a line counts as executable when, after
+trimming, it is non-blank, not a comment, and not a pure structural token
+(`then`, `do`, `done`, `else`, `fi`, `esac`, `{`, `}`, `(`, `)`, `;;`).
+
+## Limitations
+
+- Coverage is collected under `bash` (the shell the suite already uses).
+ Lines only reachable under a different shell are not distinguished.
+- xtrace reports the line where a command *starts*. Interior lines of
+ heredocs and multi-line strings are not individually reported and may show
+ as missed; exclude such files or read the numbers with that caveat.
+- Code in subprocesses spawned via `ash`/`xargs` is not traced (not bash),
+ so it is not counted toward coverage of the main script.