Documentation for Veritas: PACTA SUNT SERVANDA
Table of Contents
Unit, Integration and Black Box testing framework powered by Lisp (Guile Scheme)
Veritas helps you verify the correctness of software and explore its functioning and robustness. It achieves this by providing a clear structure for writing tests and producing easy-to-read feedback in various formats.
The framework is built around the concepts of "suites," which group related "tests," and "assertions," which perform the actual checks. I'd encourage you to peruse the test/ folder of this project to see real examples of how to use veritas.
The power of veritas lies in its simplicity, expressive embedded domain-specific language (EDSL), and some clever features that promote robust testing practices and correctness, like order randomization and concurrent testing.
If you like my work, please support me by buying me a cup of coffee ☕ so I can continue with a lot of motivation.
An expressive, free (as in freedom), and extensible test framework is crucial for modern software engineering and is a great instrument to explore systems and algorithms
You can find the full technical Guile Scheme API documentation for Veritas here:
https://jointhefreeworld.org/api-docs/veritas/API.html
Normal arguments are denoted by their name, keyword arguments are denoted by their name prefixed with #: as in Scheme. When rest arguments are mentioned, it's another terminology for variadic.
Note: Veritas generally expects you to pass keyword arguments before variadic arguments.
Assertions: ASSERTIONES
assert.scm: Assertion functions
Note: If one desires to delay execution of the #:got argument of the assertions, one can simply quote it and wrap it inside a block with compute as car. You can use compute, computare, or comp as special keywords for lazy evaluation. This works for all assertions, except assert-error and related. See an example here:
;; got will evaluate eagerly
(assert-equal #:expect 1
#:got (+ 10 (- 0 9)))
;; got will evaluate lazily, inside test context
(assert-equal #:expect 1
#:got `(comp (begin
(sleep 4)
(+ 10 (- 0 9)))))
(assert-equal #:expect 144
#:got `(comp (begin
;; This block is evaluated by Veritas.
;; Useful for setup or complex expressions.
(let ((x 12))
(* x x))))
#:name "Complex computation inside an assertion")
Environment Variables: VARIABILES AMBITUS
Some functionalities, including the runner, are configured via environment variables for ease of use and integration in CI environments. In the Guile Scheme implementation this is done leveraging parameters and. All the variables have sane defaults, but are your entry-point for lots of customisation of the test process.
VERITAS_ENTRYPOINTdefaults tospecand determines which function should be called as entry-point to your tests.VERITAS_SCAN_DIRdefaults to./and determines which directory should be used to look for test directories and files.VERITAS_TEST_TYPEdefaults tounitand determines which type of test to use, and thus which directory should be used to load files from.VERITAS_TEST_TOP_DIRdefaults totestand determines which directory to use to look for test hierarchies, typically this means atestdirectory beside yoursrcdirectory, at top repository level.VERITAS_TEST_MODULEdefaults toveritasand determines the test module to load, and also should be equal to the Guile Scheme modules name you use. For example if you have a unit test file a./test/my-program/unit/my-test.scmthen you should change this variable to the valuemy-programand your module declaration should be(define-module (my-program unit my-test)). If you don't change the defaults, place your tests at for example./test/veritas/unit/my-test.scmand name them(define-module (veritas unit my-test)).VERITAS_SHUFFLE_FILESdefaults totrueand determines whether to shuffle test file order or not.VERITAS_RUN_BENCHMARKSdefaults totrueand determines whether benchmarks should be run.VERITAS_ONLY_TESTis an optional variable you can pass to limit the execution to test cases whose name partially matches this variable.VERITAS_ONLY_SUITEis an optional variable you can pass to limit the execution to suites whose name partially matches this variable.VERITAS_REPORTERis an optional variable, interpreted as symbol in Lisp, by defaultemoji-reporter, other choices aremonotone-reporter