diff options
| author | Alexander M Pickering <alex@cogarr.net> | 2026-07-21 20:19:46 -0500 |
|---|---|---|
| committer | Alexander M Pickering <alex@cogarr.net> | 2026-07-21 20:19:46 -0500 |
| commit | 657fb0007f39f07cc0401e0c5d03e25df6234aa4 (patch) | |
| tree | f73fe23232dc80802f39caed738c38464a60ec6d /docs/coverage.md | |
| download | trbldoc-657fb0007f39f07cc0401e0c5d03e25df6234aa4.tar.gz trbldoc-657fb0007f39f07cc0401e0c5d03e25df6234aa4.tar.bz2 trbldoc-657fb0007f39f07cc0401e0c5d03e25df6234aa4.zip | |
Inital commit.
Diffstat (limited to 'docs/coverage.md')
| -rw-r--r-- | docs/coverage.md | 55 |
1 files changed, 55 insertions, 0 deletions
diff --git a/docs/coverage.md b/docs/coverage.md new file mode 100644 index 0000000..c6371d0 --- /dev/null +++ b/docs/coverage.md @@ -0,0 +1,55 @@ +# Test-suite coverage + +`scripts/coverage-tests.sh` reports which lines of trbldoc's shell sources the +test suite actually executes. + +## Usage + +```sh +# Coverage of trbldoc.sh from the integration suite (default) +sh scripts/coverage-tests.sh + +# Whole suite, also cover a resolver, fail under 70% +sh scripts/coverage-tests.sh -s all -t "trbldoc.sh ref_resolvers/lua.sh" -m 70 +``` + +Options: `-s all|unit|integration`, `-o OUTDIR`, `-t "file.sh ..."` (targets), +`-m MIN_PERCENT` (exit non-zero below this; default `0` = no gate). + +## Output + +Written to `reports/coverage-<timestamp>/` (or `-o`): + +- `index.html` — per-file coverage with bars and an overall percentage. +- `html/<file>.html` — annotated source; green = executed, red = missed, + grey = non-executable. +- `summary.csv` — `file,executable,covered,percent`. +- `covered.tsv` — raw union of executed `file,line` pairs. + +## How it works + +The bats tests run the tool as `bash "$TRBLDOC"`. Coverage exports +`BASH_ENV=scripts/lib/trace-init.sh`, so each target process enables `set -x` +with a `PS4` that stamps `BASH_SOURCE:LINENO` to a per-PID trace file. After +the run, the union of executed lines is compared against the executable lines +in each target source to compute coverage. No changes to `trbldoc.sh` or the +tests are needed. + +Only scripts whose basename is listed in `-t` are traced, so bats internals +and the `md2html` stub stay out of the report. + +## Determining "executable" lines + +The denominator is a heuristic: a line counts as executable when, after +trimming, it is non-blank, not a comment, and not a pure structural token +(`then`, `do`, `done`, `else`, `fi`, `esac`, `{`, `}`, `(`, `)`, `;;`). + +## Limitations + +- Coverage is collected under `bash` (the shell the suite already uses). + Lines only reachable under a different shell are not distinguished. +- xtrace reports the line where a command *starts*. Interior lines of + heredocs and multi-line strings are not individually reported and may show + as missed; exclude such files or read the numbers with that caveat. +- Code in subprocesses spawned via `ash`/`xargs` is not traced (not bash), + so it is not counted toward coverage of the main script. |
