From 657fb0007f39f07cc0401e0c5d03e25df6234aa4 Mon Sep 17 00:00:00 2001 From: Alexander M Pickering Date: Tue, 21 Jul 2026 20:19:46 -0500 Subject: Inital commit. --- docs/coverage.md | 55 +++++++++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 55 insertions(+) create mode 100644 docs/coverage.md (limited to 'docs/coverage.md') diff --git a/docs/coverage.md b/docs/coverage.md new file mode 100644 index 0000000..c6371d0 --- /dev/null +++ b/docs/coverage.md @@ -0,0 +1,55 @@ +# Test-suite coverage + +`scripts/coverage-tests.sh` reports which lines of trbldoc's shell sources the +test suite actually executes. + +## Usage + +```sh +# Coverage of trbldoc.sh from the integration suite (default) +sh scripts/coverage-tests.sh + +# Whole suite, also cover a resolver, fail under 70% +sh scripts/coverage-tests.sh -s all -t "trbldoc.sh ref_resolvers/lua.sh" -m 70 +``` + +Options: `-s all|unit|integration`, `-o OUTDIR`, `-t "file.sh ..."` (targets), +`-m MIN_PERCENT` (exit non-zero below this; default `0` = no gate). + +## Output + +Written to `reports/coverage-/` (or `-o`): + +- `index.html` — per-file coverage with bars and an overall percentage. +- `html/.html` — annotated source; green = executed, red = missed, + grey = non-executable. +- `summary.csv` — `file,executable,covered,percent`. +- `covered.tsv` — raw union of executed `file,line` pairs. + +## How it works + +The bats tests run the tool as `bash "$TRBLDOC"`. Coverage exports +`BASH_ENV=scripts/lib/trace-init.sh`, so each target process enables `set -x` +with a `PS4` that stamps `BASH_SOURCE:LINENO` to a per-PID trace file. After +the run, the union of executed lines is compared against the executable lines +in each target source to compute coverage. No changes to `trbldoc.sh` or the +tests are needed. + +Only scripts whose basename is listed in `-t` are traced, so bats internals +and the `md2html` stub stay out of the report. + +## Determining "executable" lines + +The denominator is a heuristic: a line counts as executable when, after +trimming, it is non-blank, not a comment, and not a pure structural token +(`then`, `do`, `done`, `else`, `fi`, `esac`, `{`, `}`, `(`, `)`, `;;`). + +## Limitations + +- Coverage is collected under `bash` (the shell the suite already uses). + Lines only reachable under a different shell are not distinguished. +- xtrace reports the line where a command *starts*. Interior lines of + heredocs and multi-line strings are not individually reported and may show + as missed; exclude such files or read the numbers with that caveat. +- Code in subprocesses spawned via `ash`/`xargs` is not traced (not bash), + so it is not counted toward coverage of the main script. -- cgit v1.2.3-70-g09d2