Previous: (veritas table), Up: API
Run thunks concurrently in fibers and collect what they return.
Blocks until every thunk has reported back through a shared channel. A thunk that raises sends a formatted error string instead of a value, so one failure cannot hang the run. Results come back in completion order, which is not the order of thunks.
(evaluate-thunks-concurrently
(list (lambda () (sleep 1) 'delayed)
(lambda () 'instant)))
⇒ (instant delayed)
Undocumented macro.
Undocumented macro.
Default value:
#f
Default value:
"reports/last-failed.scm"
Default value:
#f
Define a suite named name; the procedure behind the suite
macro.
The arguments after name are the children. Each form is evaluated
when the suite runs, and every test, suite or benchmark built while it
is evaluated is a child: one already built, a list of them, or a thunk
of them are all understood. Children run concurrently unless
concurrent? is #f, and in shuffled order unless
shuffle? is #f. before-all and after-all run
once around them, timeout caps the whole suite in seconds
including those hooks, skip? drops it from the run and tags
decide which filter may select it.
Define a test named name; the procedure behind the test
macro.
The arguments after name are the body. Each form is evaluated when the test runs, and every assertion built while it is evaluated is a case: an assertion, a list of them, a thunk of them, or a helper procedure that builds them are all understood. before and after run around the body, timeout caps the whole test in seconds, hooks included, skip? drops it from the run and tags decide which filter may select it.
A body is read in order, because a body is a sequence: one form prepares what the next one checks. shuffle? asks for the order of the body forms to be randomized anyway, which is a way to find checks that depend on each other by accident.
(test "Addition Validation"
#:before (lambda (vc) (display "Initializing..."))
(assert-equal #:expect 4 #:got (+ 2 2)))
Assert that got is equal? to expect.
The usual choice for lists, strings, vectors and other compound values.
Assert that got is eqv? to expect.
Good for numbers, characters and symbols, where value equivalence is what matters.
Assert that got is within epsilon of expect.
The default epsilon is 1.0e-8.
Assert that the container expect contains got.
Works for strings (substring), lists and vectors (membership) and hash tables (key membership).
(assert-contains #:expect '(1 2 3) #:got 2) (assert-contains #:expect "hello world" #:got "world")
Assert that got is an empty list, vector or string.
Assert that got is eq? to expect.
The strictest of the equality assertions; use it for symbols, booleans and anything compared by identity.
Assert that got is equal? to expect.
The usual choice for lists, strings, vectors and other compound values.
Assert that got is eqv? to expect.
Good for numbers, characters and symbols, where value equivalence is what matters.
Return an assertion that thunk raises.
It fails if thunk returns normally. name labels it in the
report and timeout caps it in seconds, as in assert.
(assert-error (lambda () (sum-list-elements '(1 2 "a" 4)))
#:name "Should fail when list contains non-numbers")
Assert that x is #f.
Assert that got is within the inclusive range min–max.
Assert that got has expect elements (list, vector or string length).
Assert that the string got matches pattern.
pattern is a string or a compiled regexp.
(assert-matches #:pattern "^veri.*s$" #:got "veritas")
Return an assertion that thunk does not raise.
It fails if thunk raises. name labels it in the report and
timeout caps it in seconds, as in assert.
Assert that got differs from expect by at least epsilon.
Inverse of assert-approximate.
Assert that the container expect does not contain got.
Inverse of assert-contains.
Assert that got is a non-empty list, vector or string.
Assert that got is not eq? to expect.
Inverse of assert-eq.
Assert that got is not equal? to expect.
Inverse of assert-equal.
Assert that got is not eqv? to expect.
Inverse of assert-eqv.
Assert that string got does not match regular expression pattern.
Inverse of assert-matches.
Assert that got is equal? to one of the elements of list
expect.
Assert that calling thunk raises an error.
By default any error satisfies the assertion. Pass on to require
a specific error key (for example 'wrong-type-arg), or
predicate, a procedure of (key args) returning #t
for the expected error.
(assert-raises (lambda () (/ 1 0))) (assert-raises (lambda () (+ 1 'a)) #:on 'wrong-type-arg)
Assert that x is #t.
Define a benchmark named name that times thunk over runs runs.
Statistics come from Guile’s statprof, so benchmarks are reported
separately from tests. timeout caps all the runs together in
seconds and skip? drops the benchmark from the run.
Run thunk with an optional deadline of seconds seconds.
When seconds is a positive real number the work runs in its own
fiber and, if it has not finished by then, the fiber is aborted and the
result of calling on-timeout is returned instead. When
seconds is #f (the default, meaning no deadline)
thunk is simply called and its result is returned.
thunk must return a single value. The caller never waits for more than seconds, and the runaway fiber is terminated rather than left running.
(call-with-timeout 5
(lambda () (expensive-computation))
#:on-timeout (lambda () 'gave-up))
Return the outcome of every assertion in xs as a flat list.
Booleans only: #t for a pass, #f for a failure.
Assert that the string got matches pattern.
pattern is a string or a compiled regexp.
(assert-matches #:pattern "^veri.*s$" #:got "veritas")
Assert that the container expect contains got.
Works for strings (substring), lists and vectors (membership) and hash tables (key membership).
(assert-contains #:expect '(1 2 3) #:got 2) (assert-contains #:expect "hello world" #:got "world")
Return the seed this run shuffles with, choosing one when none was set.
seed (a number or a numeric string) wins when given, otherwise the
value of VERITAS_SEED is used, and when neither is set a fresh
seed is drawn from the platform. The chosen seed is remembered, so the
order the runner gives the test files and every order chosen here come
from one seed, and the run can be replayed by setting
VERITAS_SEED to the seed that was reported.
(current-seed #:seed 7) ⇒ 7
Wrap each of forms in a thunk, so a body is evaluated when its task runs.
Deferring is what makes a body form contribute both its own value and the cases it builds: the form is evaluated inside the body’s collector, at run time, instead of while the test file is being loaded.
Run result when name matches restriction.
The check is case-insensitive, and a restriction counts as a match when
it appears anywhere in name. A #f restriction accepts
everything; one that does not appear gives #f.
Return an assertion that thunk raises.
It fails if thunk returns normally. name labels it in the
report and timeout caps it in seconds, as in assert.
(assert-error (lambda () (sum-list-elements '(1 2 "a" 4)))
#:name "Should fail when list contains non-numbers")
Return the names that failed in all-results, for a last-failed rerun.
Every test whose assertions failed is named. A suite that failed as a
whole, through its own deadline for instance, contributes the names of
the tests it holds, because a suite name is not something a rerun can
select. A failure with no test name attached at all contributes
#all, which makes the next run run everything rather than
silently run nothing. Names come back in the order recorded, without
duplicates.
Return #t when name should run in a last-failed rerun.
only is the record of failing names to honour; it defaults to the
one read from reports/last-failed.scm when
VERITAS_ONLY_FAILED asked for a rerun. #f means there is
no record, so everything runs; an empty list means the last run passed,
so nothing does.
Assert that x is #f.
Return the elements of xs in random order (Fisher-Yates).
Shuffling happens on a vector copy, so xs is left alone.
seed makes the order reproducible, which is how a failing order is
replayed; without it a fresh seed is pulled from
random-state-from-platform on every call, so two calls on the
same list differ. Lists of one element or fewer come back as they are.
(equal? (fisher-yates-shuffle '(1 2 3) #:seed 7)
(fisher-yates-shuffle '(1 2 3) #:seed 7))
⇒ #t
Return a new seed for a shuffle, drawn from the platform’s entropy source.
Report the seed of a shuffled run and the run can be replayed: the same seed and the same list always produce the same order.
Return a generator of printable ASCII characters, 32 to 126.
Return a generator of integers between 1 and max-bound (inclusive).
Call the generator with a random state:
(define gen (generate-integer #:max-bound 100)) (gen (seed->random-state 42))
Return a generator of lists whose elements come from elem-generator.
Lists are min-len to max-len long.
Return a generator of printable-ASCII strings.
Strings are min-len to max-len characters long.
Return a generator of lowercase symbols, min-len to max-len long.
Symbols stay unescaped because the characters are always a-z.
Assert that got is eq? to expect.
The strictest of the equality assertions; use it for symbols, booleans and anything compared by identity.
Assert that got is an empty list, vector or string.
Assert that got is within the inclusive range min–max.
Return #t when a and b differ by less than
epsilon.
Useful for floating point, where exact equality rarely holds.
Assert that got has expect elements (list, vector or string length).
Return assertion, a procedure of one validation client, as a case to run.
The returned procedure is tagged so the runner can tell an assertion
from a task, and it records having run, which is how an assertion the
body itself consumes - with value or ok?, say - stays an
input to another case instead of becoming a case of its own.
It is also registered with the collector of the body being evaluated, if
there is one. That registration is what keeps a body form that builds
several assertions, such as the body of a let, from contributing
only the value of its last expression. Building an assertion outside
any body - at load time, or inside an assertion that runs later - simply
registers nothing.
Combine what several runs recorded as failing into one record.
records holds one record per run: a list of names, or #f
when a run left no record at all. When every run recorded an empty list
the result is an empty list, so a parallel run that passed does not look
like a run to rerun. When no run left a record the result is #f,
which stands for no information rather than for nothing having failed.
Names are deduplicated and keep the order they were found in.
Assert that got is not equal? to expect.
Inverse of assert-equal.
Assert that got is not eqv? to expect.
Inverse of assert-eqv.
Assert that string got does not match regular expression pattern.
Inverse of assert-matches.
Assert that the container expect does not contain got.
Inverse of assert-contains.
Return an assertion that thunk does not raise.
It fails if thunk raises. name labels it in the report and
timeout caps it in seconds, as in assert.
Assert that got is not eq? to expect.
Inverse of assert-eq.
Assert that got is a non-empty list, vector or string.
Assert that got differs from expect by at least epsilon.
Inverse of assert-approximate.
Return an assertion comparing got with expect using
compare, a two-argument predicate such as equal?.
The assert-* wrappers below are built on this one. got is
usually a plain value, but a list starting with compute,
comp or computare is evaluated when the assertion runs
instead: (assert #:got '(comp (+ 1 1)) ...) compares against
2. Deferring work that way is what makes it something a
timeout can actually catch.
name labels the assertion in the report. timeout caps it in
seconds; #f, the default, means no deadline.
(assert #:expect 3
#:got (+ 1 1 1)
#:compare equal?
#:name "Ensuring addition works")
Assert that got is within epsilon of expect.
The default epsilon is 1.0e-8.
Return #t when a and b differ by less than
epsilon.
Useful for floating point, where exact equality rarely holds.
Return the test names recorded as failing at path, or #f.
path defaults to the record of the whole run. #f means
there is no record of a previous run: the file is missing or unreadable,
and nothing is filtered.
Return xs in the order this run’s seed prescribes.
Both the runner, which shuffles test files, and veritas-run,
which shuffles tasks, tests and cases, shuffle through this, so a single
seed reproduces a whole run.
Split args, the syntax of an argument list, into its parts.
Two values come back: the #:keyword value pairs in the order they
were written, and the forms that are not part of a pair, also in order.
The test and suite macros use it to tell a body from the
flags around it, wherever those flags were written.
Assert that calling thunk raises an error.
By default any error satisfies the assertion. Pass on to require
a specific error key (for example 'wrong-type-arg), or
predicate, a procedure of (key args) returning #t
for the expected error.
(assert-raises (lambda () (/ 1 0))) (assert-raises (lambda () (+ 1 'a)) #:on 'wrong-type-arg)
Assert that got is equal? to one of the elements of list
expect.
Run tasks, the top-level tests, suites and benchmarks.
shuffle? randomizes the order of the tasks and concurrent?
runs them in parallel fibers. reporter names a reporter,
otherwise VERITAS_REPORTER decides. seed makes the shuffle
reproducible, and the seed actually used is reported, so a failing order
can be replayed by setting VERITAS_SEED to it.
with-coverage? writes an LCOV report and debug? prints the
raw results.
Assert that x is #t.
Record the names that failed, so the next run can rerun just those.
Each worker process writes its own record - the path is a parameter -
and the --jobs parent merges them into the run’s, which is why
the writing and the merging are separate.
Record names at path as the tests a rerun should select.
An empty list is written when everything passed, which is what makes the
next VERITAS_ONLY_FAILED run do nothing instead of rerunning the
world. A trouble with the file is swallowed: the run has already
happened.
"#all"
Previous: (veritas table), Up: API