Testing

As a community project, DPDK CI is distributed and welcome any contribution.

There are various ways of contributing to the CI effort:

One specific lab instance, named community lab, is hosted at UNH-IOL. A view of the current test coverage of the lab is automatically generated based on recent test runs.

CI Testing Goals


General Properties

Ideally, the CI infrastructure should have these properties:

Inputs

There are three kinds of inputs to be tested:

Categories

The tests should cover these categories:

Workflow


Patch Testing

When a patch is submitted, it is received via the mailing list dpdk-dev. For simple tests, like checkpatch, it can be run on email reception.

For most tests, the patch must be applied on the right tree after its series dependencies. Dealing with series and dependencies is easier via patchwork database request. That’s why the common workflow is to poll patchwork. The patches are saved and organized in patchwork after being distributed by the mailing list.

Once the patch is ready, the automatic tests can run.

When a group of tests is finished, the author should be notified of any error, and a public report must be sent to test-report@dpdk.org.

The reports are formatted with a specific syntax for automatic parsing. When such report email is received, the bot ci@dpdk.org will update patchwork, referencing the report archive and incrementing one of the counters (success, warning or failure).

A maintainer should wait the end of the tests (should be less than a day), and check the results before accepting a patch.

Requesting a Patch Retest

If CI reports a failure for a test label on Patchwork, it is possible to request a retest of a patch or series on those label(s). For a list of Patchwork testing labels, send an email reply on the patch or cover letter like so:

Recheck-request: iol-compile-amd64-testing, iol-broadcom-Performance, iol-unit-arm64-testing, github-robot

Note that you may omit any labels which are not failing.

This is currently supported by the UNH-IOL Community Lab, and the GitHub Actions robot, so just those Patchwork testing labels beginning wih “iol-” or “github-robot”.

Mainline Monitoring

The repository mirror on GitHub is preferred for tests integration:

Release Validation

When a new release is available, especially for release candidates, some validation reports are requested to the community members.

Such release validations are sent as replies to the release announcement.

Some PMDs can be tested only by their owners. That is why it is expected that device vendors send a release validation report, at least in the first release that the PMD is upstreamed.

Tools


Infrastructure

The testing infrastructure scripts are shared in the dpdk-ci git repository.

The website dashboard and API of the community lab is a Django project shared in the dpdk-ci-site git repository.

The testing infrastructure work is coordinated in three complementary forums:

Known Test Frameworks and Tools