Testing

As a community project, DPDK CI is distributed and welcome any contribution.

There are various ways of contributing to the CI effort:

One specific lab instance, named community lab, is hosted at UNH-IOL.

CI Testing Goals


General Properties

Ideally, the CI infrastructure should have these properties:

Inputs

There are three kinds of inputs to be tested:

Categories

The tests should cover these categories:

Workflow


Patch Testing

When a patch is submitted, it is received via the mailing list dpdk-dev. For simple tests, like checkpatch, it can be run on email reception.

For most tests, the patch must be applied on the right tree after its series dependencies. Dealing with series and dependencies is easier via patchwork database request. That’s why the common workflow is to poll patchwork. The patches are saved and organized in patchwork after being distributed by the mailing list.

Once the patch is ready, the automatic tests can run.

When a group of tests is finished, the author should be notified of any error, and a public report must be sent to test-report@dpdk.org.

The reports are formatted with a specific syntax for automatic parsing. When such report email is received, the bot ci@dpdk.org will update patchwork, referencing the report archive and incrementing one of the counters (success, warning or failure).

A maintainer should wait the end of the tests (should be less than a day), and check the results before accepting a patch.

Mainline Monitoring

The repository mirror on GitHub is preferred for tests integration:

Release Validation

When a new release is available, especially for release candidates, some validation reports are requested to the community members.

Such release validations are sent as replies to the release announcement.

Tools


Infrastructure

The testing infrastructure scripts are shared in the dpdk-ci git repository.

The website dashboard and API of the community lab is a Django project shared in the dpdk-ci-site git repository.

The testing infrastructure work is coordinated in three complementary forums:

Known Test Frameworks and Tools