1.15.1 Procedures

Procedure: discover-tests KEY: #:test-type #:scan-dir #:test-top-dir #:test-module #:dir ¶

Walk the cleaned file tree dir and collect candidate test files.

Descends through scan-dir, test-top-dir and test-module, then hands the contents of a folder named test-type to flatten-test-files.

Procedure: display-result-counts counts ¶

Print the global totals of a run: its files, suites, tests and assertions.

Procedure: merge-worker-counts indexes ¶

Return the totals of the workers at indexes added up.

A worker that left no counts - it failed before its file ran - contributes nothing, because there is nothing to count.

Procedure: merge-worker-coverage indexes ¶

Return the coverage of the workers at indexes merged into one report.

A worker that was not asked for coverage left no fragment, and a run without coverage merges to an empty report, which is then left alone.

Procedure: parse-jobs value ¶

Return the number of worker processes value asks for, or #f.

#f or an absent value means the run stays in one process. A value that is not a whole number of processes is an error rather than a silent fallback, because a CI job that asked for parallelism and got none should be told.

(parse-jobs "4")
⇒ 4

Procedure: parse-partition value ¶

Return the (index . total) a partition string describes, or #f.

A partition is written "2/3": this machine runs the second of three slices. A malformed or impossible partition raises instead of quietly running everything, because that would hide a misconfigured CI job.

Procedure: partition-files files index total ¶

Return the files of partition index of total from files.

The split is round-robin over files, so the same project always splits the same way and the partitions together are exactly files.

(partition-files '("a" "b" "c" "d" "e") 2 3)
⇒ ("b" "e")

Procedure: run-test-file KEY: #:test-file #:test-type #:entrypoint-function ¶

Load one test file and call its entrypoint.

test-file is a path relative to the test root and test-type is the module prefix, so "math/algebra.scm" with unit loads (veritas unit math algebra). entrypoint-function names the zero-argument procedure to call in that module.

Procedure: run-test-file-guarded KEY: #:test-file #:test-type #:entrypoint-function ¶

Run one test file, returning #t when it passed and #f when not.

veritas-run reports a failing file by exiting the process, and a file that cannot even be loaded raises; either would end the whole single-process run and take the files after it, and the totals of the run, with it. Both are caught here and turned into a returned answer, with what went wrong printed against the file.

Procedure: run-test-files files KEY: #:test-type #:entrypoint-function #:junit-data-paths ¶

Run files one after another in this process, returning the failed ones.

A file that fails no longer ends the run: the files after it still run, so the run reports every failure and its global totals before it exits, which is what --jobs already did for the files it spread over workers.

When junit-data-paths is given, the ith file leaves its JUnit report at that path as data instead of writing it as the run’s document, so the caller can merge one document out of all of them.

Procedure: run-tests ¶

Find every test file under the scan root and run each one.

Behaviour comes from the environment:

• VERITAS_SCAN_DIR: Root folder to scan. Defaults to "./".
• VERITAS_TEST_TOP_DIR: The subfolder boundary identifier containing test suites. Defaults to "test".
• VERITAS_ENTRYPOINT: The signature method executed inside test targets. Defaults to "spec".
• VERITAS_TEST_MODULE: Top level test parent folder filter. Defaults to "veritas".
• VERITAS_TEST_TYPE: Category discriminator keyword used during routing. Defaults to "unit".
• VERITAS_SHUFFLE_FILES: Set to string "true" to shuffle execution sequences using a Fisher-Yates algorithm. Defaults to "true".
• VERITAS_SEED: the seed every shuffle of this run uses, reported when a run is shuffled so a failing order can be replayed.
• VERITAS_PARTITION: INDEX/TOTAL to run only one slice of the discovered files, which is how a run is split across CI machines.
• VERITAS_JOBS: how many worker processes to run the discovered files in. Absent or 1 means one process, as before; 2 and up run one file per process and merge their output, their record of failures and their JUnit reports.
• VERITAS_COVERAGE: "true" measures code coverage even for a test file that does not ask for it, and the worker fragments are merged into one report.

Procedure: select-changed-files files KEY: #:test-type #:changed? #:since #:src-dir ¶

Return the files of files a change can reach.

Without changed? or since every file comes back, and the run is the whole suite it always was. With them, the changed files are read from git, the modules they reach are followed through imports, and only the test files that import one of those modules - or that the change touched - come back.

A change that touches nothing returns every file rather than none: a run is never silently empty, because an empty run looks exactly like a passing one.