As a community project, DPDK CI is distributed and welcome any contribution.
There are various ways of contributing to the CI effort:
- suggest improvement
- write a new test
- improve infrastructure scripts
- run a lab
One specific lab instance, named community lab, is hosted at UNH-IOL. A view of the current test coverage of the lab is automatically generated based on recent test runs.
CI Testing Goals
Ideally, the CI infrastructure should have these properties:
- reliable: always run, without false positive
- vendor neutral: run same tests and configurations on all HW/SW platforms
- transparent: show environment, configuration, source code and result
- coverage summary
- trending graphs
- reports to author and firstname.lastname@example.org
- easy to add a test
- easy to reproduce a test locally
There are three kinds of inputs to be tested:
- per-patch testing integrated with patchwork
- per-commit testing after merges in mainline, stable or dpdk-next-* repositories
- per-release testing including release candidates of all branches
The tests should cover these categories:
- source code checks:
- static analysis
- runtime tests:
When a patch is submitted, it is received via the mailing list dpdk-dev. For simple tests, like checkpatch, it can be run on email reception.
For most tests, the patch must be applied on the right tree after its series dependencies. Dealing with series and dependencies is easier via patchwork database request. That’s why the common workflow is to poll patchwork. The patches are saved and organized in patchwork after being distributed by the mailing list.
Once the patch is ready, the automatic tests can run.
When a group of tests is finished, the author should be notified of any error, and a public report must be sent to email@example.com.
The reports are formatted with a specific syntax for automatic parsing. When such report email is received, the bot firstname.lastname@example.org will update patchwork, referencing the report archive and incrementing one of the counters (success, warning or failure).
A maintainer should wait the end of the tests (should be less than a day), and check the results before accepting a patch.
Requesting a Patch Retest
If CI reports a failure for a test label on Patchwork, it is possible to request a retest of a patch or series on those label(s). For a list of Patchwork testing labels, send an email reply on the patch or cover letter like so:
Recheck-request: iol-compile-amd64-testing, iol-broadcom-Performance, iol-unit-arm64-testing, github-robot
Note that you may omit any labels which are not failing.
This is currently supported by the UNH-IOL Community Lab, and the GitHub Actions robot, so just those Patchwork testing labels beginning wih “iol-” or “github-robot”.
The repository mirror on GitHub is preferred for tests integration:
- If polling for updates, better to load the GitHub mirror.
- Some services like GitHub Actions can be hooked.
When a new release is available, especially for release candidates, some validation reports are requested to the community members.
Such release validations are sent as replies to the release announcement.
Some PMDs can be tested only by their owners. That is why it is expected that device vendors send a release validation report, at least in the first release that the PMD is upstreamed.
The testing infrastructure scripts are shared in the dpdk-ci git repository.
The website dashboard and API of the community lab is a Django project shared in the dpdk-ci-site git repository.
The testing infrastructure work is coordinated in three complementary forums: