Next: (veritas diff), Previous: (veritas container), Up: API
Code coverage for Veritas.
Guile collects execution counts by firing a VM hook on every instruction it retires. That hook only exists in the ‘debug’ VM engine: in the ‘regular’ engine the same code runs silently, which is why a run that merely calls ‘(system vm coverage)’ records no hits at all. Every collection here therefore happens inside ‘with-debug-vm’.
What comes out of the VM is an IP-to-count table and a census of the source locations of every loaded program. It is turned into a report - one record per source file, holding the execution count of each instrumented line - and that report can be written as LCOV, as a self-contained HTML report, or as JSON, and merged with other reports (the files of a ‘–jobs’ run, say).
Undocumented macro.
Undocumented macro.
Undocumented macro.
Undocumented macro.
Undocumented macro.
Run thunk and return its values preceded by its coverage report.
The report is a coverage-report record; the values of thunk
follow it unchanged. The run happens on the debug engine, and the
engine in use before the call is restored afterwards.
(call-with-values (lambda () (collect-coverage (lambda () (run-the-tests)))) (lambda (report result) ...))
Return the path of the Cobertura XML report of a run.
The VERITAS_COBERTURA_PATH environment variable wins when it is
set, which is how a CI job puts the document where its dashboard looks
for it.
Turn Guile’s coverage data into a report.
Every instrumented source file is kept, with the count of each of its lines - including the lines that never ran, which is the whole point of a coverage report. include and exclude filter the files by absolute path.
Return the directory the LCOV report is written to.
dir wins; otherwise VERITAS_COVERAGE_DIR decides, and
"reports/lcov" is the default. The directory holds the LCOV
trace, the JSON summary and the Cobertura and JaCoCo documents.
Return #t when value (the environment by default) refuses
coverage.
An explicit refusal beats a test file that asks for measurement with
#:with-coverage? #t: a task that says ‘maak test-fast’ or passes
--no-coverage means it, and it is the only way to run a file that
always measures without paying for it.
Return #t when value (the environment by default) asks for
coverage.
Return the prefixes left out of a report.
An explicit exclude wins, otherwise
VERITAS_COVERAGE_EXCLUDE is read. Nothing is excluded by
default.
Return how many instrumented lines of file ran at least once.
Return how many lines of file hold code that could be counted.
Return how often line of file ran, or #f when it
holds no code.
A line that was instrumented but never reached counts 0; a line with no
code at all - a comment, a blank line - has no entry and gives
#f.
Return the percentage of file’s instrumented lines that ran.
Return the directory the HTML report is written to, or #f to skip
it.
html-dir wins; otherwise VERITAS_COVERAGE_HTML_DIR decides.
An empty value, "false" or "0" turns the HTML report off.
Return the path of the index page of the HTML report, or #f when
off.
Return the prefixes a report is limited to.
An explicit include wins; otherwise
VERITAS_COVERAGE_INCLUDE is read, and when neither is set the
current directory is the boundary, which is what keeps Guile, Fibers and
the other dependencies out of the report.
Return the path of the JaCoCo XML report of a run.
The VERITAS_JACOCO_PATH environment variable wins when it is set.
Return the path of the JSON summary of a coverage run.
Return the path of the merged LCOV file of a coverage run.
Return #t when path belongs in a coverage report.
include is a list of directory prefixes; an empty list means every file. exclude is a list of prefixes that win over it, so the tests of a project can be left out of the coverage of the code they exercise.
Return how many instrumented lines of report ran at least once.
Return how many lines of report hold code that could be counted.
Return how often line of path ran in report, or
#f.
path has to be the absolute path the report holds.
Return the percentage of report’s instrumented lines that ran.
Return report as the one line a run prints.
"Code coverage: 82.4% of 340 lines (280 covered) in 12 files"
Return a report that holds no file at all.
Return report as an LCOV trace file held in a string.
Return the coverage of path, given its lines.
lines is an alist of (line-number . execution-count). It
is sorted by line number here, so every report has one shape however it
was built.
Return a report holding files, sorted by path.
files may hold a number of coverage-file records for the
same path; they are merged, keeping the highest count seen for each
line, so a report read from several LCOV files can be handed to this
directly.
Return one report holding every file of reports.
The line counts of a file present in several reports are merged by taking the highest count of each line: a line that ran in any of the runs was covered.
Return the reports of every file of paths merged into one.
A path that does not exist is skipped, so the caller can pass the files a run was expected to write without checking each one.
Return the source file name points at.
The VM records source files the way the compiler saw them: an absolute path for a system library, a path relative to the current directory for a project file, or a path relative to one of Guile’s load path entries. Each is resolved here, in that order, so a report always holds absolute paths and two spellings of one file cannot end up as two rows.
Merge report into this run’s coverage and write the reports.
The report of the run so far is kept per destination directory and
merged with report, so a process that runs several test files ends
with all of them in one report. The merged report is written as LCOV,
as HTML (unless html-dir or the environment turns it off), as
JSON, and as the two XML formats the Java world reads: Cobertura and
JaCoCo. sources names the roots the XML documents report the
files relative to. log, when given, is called with a one-line
summary and with the paths written.
Nothing is written when no file has been measured yet: an empty report is the absence of coverage, and a report of zero lines is not something to publish.
Returns the merged report.
Return the LCOV trace data read from port as a report.
Only SF and DA records carry information here; the other
lines are accepted and ignored, so a file written by genhtml or
by another tool can be read as well.
Return the LCOV trace file at path as a report.
A missing file is an empty report, which is what a run that produced no coverage looks like.
Rebuild the HTML, JSON, Cobertura and JaCoCo reports from the last LCOV run.
lcov-file defaults to the report of the last run,
reports/lcov/veritas.info, and sources names the roots the
two XML documents report the files relative to (the include paths of the
environment by default). Returns the report read back, or #f
when no LCOV file has been written yet, in which case nothing is
written.
Call thunk on a VM whose engine is engine, then put it back.
Guile fires the VM hooks that count instructions only in the
debug engine, so this is what makes coverage - and any other hook
- record anything at all. The engine that was current before is
restored even when thunk raises, which keeps the switch invisible
to the rest of the run.
The switch is bracketed with call-with-fiber-safe-wind, not with
a plain dynamic-wind: a fiber that yields would otherwise put the
old engine back in the middle of the measured run, and everything after
the first suspension would be counted as not covered.
Write report as a Cobertura XML document to path.
Cobertura is the format Jenkins, GitLab, Azure DevOps and SonarQube accept most readily. Every line carries its execution count, each file is a class, and a directory is a package. sources defaults to the source roots of the run, and the filenames are written relative to them.
path defaults to cobertura.xml next to the LCOV file; the
VERITAS_COBERTURA_PATH environment variable overrides it. Returns
the path written.
Write a JSON summary of report to path.
The document holds the totals, and for each file its counts and the
numbers of the lines that never ran, which is what a CI job needs to
annotate a pull request without parsing HTML. path defaults to
coverage-summary.json next to the LCOV file.
Write report as one self-contained HTML page into directory.
The page holds the summary, the table of files and the annotated source of every file, so the whole project’s coverage is one document: nothing is fetched, nothing is split across pages, and it opens from a file:// URL. Source lines that never ran are marked, and the page can be filtered by path.
directory defaults to reports/html, or to whatever
VERITAS_COVERAGE_HTML_DIR says. Returns the path of the page.
Write report as a JaCoCo XML document to path.
JaCoCo’s format is what the coverage gutters of IDEs, SonarQube and Jenkins’ JaCoCo plugin read. Each file becomes a class and a source file, every line carries its missed and covered instruction counts, and the counters at class, package and report level let a tool compute the totals without reading a line.
path defaults to jacoco.xml next to the LCOV file; the
VERITAS_JACOCO_PATH environment variable overrides it. Returns
the path written.
Write report to port in the LCOV trace file format.
Write report to path as LCOV, creating the directory as needed.
Returns path, so a caller can report where the file went.
Next: (veritas diff), Previous: (veritas container), Up: API