Commit Graph
17 Commits
Author SHA1 Message Date
Mattias Walström b8ba1a1042 test: spec: Allow to generate test specification without logical topology 2026-06-27 10:57:22 +02:00
Joachim Wiberg b4a5b35c0e test/spec: refactor test spec & report generation
Import new 9pm version for improved test report generation.  With this
in place we can take the opportunity to also refactor and simplify the
test spec. generation.

Fixes #1129

Signed-off-by: Joachim Wiberg <troglobit@gmail.com>
2025-09-15 14:26:03 +02:00
Joachim Wiberg fe2d0f488d test/case: simplify AsciiDoc image references
This commit greatly simplifies AsciiDoc image references in generated
Readme.adoc files.  The two focused use-cases that remain after this
change are working references in:

 - Generated output/images/test-report.pdf
 - Viewing test's Readme.adoc from GitHub

Previously we aimed to have working images also when the test's Readme
was included in the parent directory's Readme.adoc.  This, however, is
not supported as of this commit.  It seems unlikely also to ever be a
supported feature of AsciiDoc on GitHub, for details, see the following
issue: <https://github.com/github/markup/issues/1095>

Signed-off-by: Joachim Wiberg <troglobit@gmail.com>
2025-09-15 06:40:38 +02:00
Joachim Wiberg 87f9b58957 test/spec: pep-8 fixes, minor cleanup only
Signed-off-by: Joachim Wiberg <troglobit@gmail.com>
2025-09-15 06:40:37 +02:00
Richard Alpe b388e080ad test: fix image paths in test specifications (again)
{topdoc} is needed for asciidoctor to find the image during test
report generation.

This is the same functional change as 02d8288863 but with the
generation script actually updated. This was forgotten in the last
commit.

Signed-off-by: Richard Alpe <richard@bit42.se>
2025-03-03 14:11:44 +01:00
Joachim WibergandTobias Waldekranz 773683bfd5 test: spec: catch and format dot file errors
Signed-off-by: Joachim Wiberg <troglobit@gmail.com>
2025-02-12 23:21:08 +01:00
Joachim Wiberg 9198383742 test: spec: for new tests a Readme.adoc does not exist
Signed-off-by: Joachim Wiberg <troglobit@gmail.com>
2025-01-31 13:55:35 +01:00
Richard Alpe 28f67ff608 test: add topdoc dir to all test specifications
This should be a non functional change. The idea is to use the topdoc
dir from 9pm when generating the test report.

Signed-off-by: Richard Alpe <richard@bit42.se>
2025-01-21 15:02:11 +01:00
Mattias Walström 907401f6f2 generate_spec: Allow to run script manually and show all output 2025-01-14 13:43:59 +01:00
Mattias Walström 9a831217e4 test-specification: Add support for python f-strings in step and description
This is only availible for tests that require paramters. If the test have a
parameter --data the description and test steps can contain {data} which will
be replaced with tha actual data of the parameter data in the test specification.
2025-01-14 10:54:05 +01:00
Joachim WibergandMattias Walström 1aab75d86a test: spec: remove .tmp files when done
Signed-off-by: Joachim Wiberg <troglobit@gmail.com>
2025-01-13 20:16:37 +01:00
Mattias Walström 29031be4e9 test-specification: Generate test specification from suite file
The suite can also contain meta data for the test specification,
One meta data that is implemented is title, which is used when the
same test code is used twice with parameters.

Also if a test case should not be present in the test specification
for some case. You can add specification: False as meta data.
2025-01-10 21:42:10 +01:00
Mattias Walström c7db71d095 test-spec: Add support for images in description 2024-11-25 14:03:03 +01:00
Mattias Walström 3b3656c8f5 test: spec: change image format from PNG to SVG
SVG is a plain text file, better to see why it changed
than binary PNG. Also fix #744 by adding infix/OEM hash
to test specification

Signed-off-by: Mattias Walström <lazzer@gmail.com>
2024-10-23 14:21:26 +02:00
Mattias Walström 0d98ee5716 test-spec: Fail build on failed test specification build
This can fail on two ways:
* Generate topology files
  Unfortunatly the python-graphwiz has no way of detect faults
  it just show 'Warning' on stderr, catch it and if != empty, exit 1

* Generate pdf
  Here the default error level was FATAL, but for example a file
  that is included does not exist it just show an error on stdout.
  the error level is now INFO, it should always be silent.
2024-10-02 23:31:14 +02:00
Mattias Walström d79d6b7f4d test: spec: Always generate specifications with relative path 2024-09-09 15:11:36 +02:00
Mattias Walström b3831ebe9f Add generation of test specification
The test specifcation is written in Asciidoc and is autogenerated from the content
in the test scripts.

Fonts imported from https://github.com/adobe-fonts/source-code-pro/

To generate the test script:

$ make test-spec
2024-09-06 10:18:16 +02:00