☰
Navigation :

Testing

As a community project, DPDK CI is distributed and welcome any contribution.

There are various ways of contributing to the CI effort:

One specific lab instance, named community lab, is hosted at UNH-IOL. A view of the current test coverage of the lab is automatically generated based on recent test runs.

CI Testing Goals

General Properties

Ideally, the CI infrastructure should have these properties:

Inputs

There are three kinds of inputs to be tested:

Categories

The tests should cover these categories:

Workflow

Patch Testing

When a patch is submitted, it is received via the mailing list dpdk-dev. For simple tests, like checkpatch, it can be run on email reception.

For most tests, the patch must be applied on the right tree after its series dependencies. Dealing with series and dependencies is easier via patchwork database request. That’s why the common workflow is to poll patchwork. The patches are saved and organized in patchwork after being distributed by the mailing list.

Once the patch is ready, the automatic tests can run.

When a group of tests is finished, the author should be notified of any error, and a public report must be sent to test-report@dpdk.org.

The reports are formatted with a specific syntax for automatic parsing. When such report email is received, the bot ci@dpdk.org will update patchwork, referencing the report archive and incrementing one of the counters (success, warning or failure).

A maintainer should wait the end of the tests (should be less than a day), and check the results before accepting a patch.

Note: At this time, the depends-on tag is supported in CI testing by only the Github Robot. Patchwork test reports under labels “iol”, “intel”, and “loongson” do not yet support this feature.

Requesting a Patch Retest

If CI reports a failure for a test label on Patchwork, it is possible to request a retest of a patch or series on those label(s). The default behavior is that your DPDK patch will be retested “as-is,” i.e. your series is not re-applied before testing. For a list of Patchwork testing labels, send an email reply on the patch or cover letter like so:

Recheck-request: iol-compile-amd64-testing, iol-broadcom-Performance, iol-unit-arm64-testing, github-robot

Note that you may omit any labels which are not failing.

You may also specify that your series must be re-applied to a given branch before running testing. This is useful in a variety of situations, such as when your patch originally got applied to a branch which was in a bad state, or when the CI testing systems apply your patch to the wrong branch.

In order to select the “re-apply” option, use the rebase={branch_name} option when sending your recheck email. An example usage is provided below:

Recheck-request: rebase=main, iol-intel-Functional, iol-intel-Performance

Rechecks are currently supported by the UNH-IOL Community Lab, and the GitHub Actions robot, so just those Patchwork testing labels beginning wih “iol-” or “github-robot”. IOL is the only lab supporting the re-apply (rebase) option.

Mainline Monitoring

The repository mirror on GitHub is preferred for tests integration:

Release Validation

When a new release is available, especially for release candidates, some validation reports are requested to the community members.

Such release validations are sent as replies to the release announcement.

Some PMDs can be tested only by their owners. That is why it is expected that device vendors send a release validation report, at least in the first release that the PMD is upstreamed.

Tools

Infrastructure

The testing infrastructure scripts are shared in the dpdk-ci git repository.

The website dashboard and API of the community lab is a Django project shared in the dpdk-ci-site git repository.

The testing infrastructure work is coordinated in three complementary forums:

Known Test Frameworks and Tools