We are working on getting more of the MariaDB source covered by our mysql-test-run (MTR) test suite. This is an ongoing (and slow) task as there is still a lot of old code with not very good coverage.

Goals for new code

For new code in MariaDB, we aim much higher:

The goals are:

  1. All new lines of code should ideally be tested by MTR.
  2. Code which cannot reasonably be tested by MTR needs to be tested by another tool and those code lines marked with /* purecov: tested */.
    • In this case the tool used for testing should be documented in the worklog entry for the code or in the commit message.
  3. Code that can't reasonably be tested (such as error conditions) should be marked with '/* purecov: inspected */' so that a reviewer of the code can easily spot this code.
  4. Code that is suspected to be deadcode should have an 'DBUG_ASSERT(0)' or be marked with '/* purecov: deadcode */' so that we have a chance to notice if the code is ever executed.

The reason we are using 'purecov' to mark lines is an attribution to the purecov tool we originally used for code coverage in the early years of MySQL.

Markers

The recommended markers are:

/* purecov: tested */

  • For code lines that are tested by something other than mysql-test-run:

/* purecov: inspected */

  • For code lines that are hard to test but for which one has read the line multiple times to ensure it is correct. A code reviewer should also inspect these lines with care as they have not been properly tested.

/* purecov: deadcode */

  • For code lines that one suspects will never be called. Having this marker allows us to generate a warning during mysql-test-run code coverage if this line is executed.

The comment must be placed on the line/lines that are affected.

For code blocks larger than 1 line one can use the block syntax:

/* purecov: begin tested */
....
/* purecov: end */

Running mysql-test-run with gcov

To be able to see the level of coverage within the current test suite, do the following:

  1. Make sure gcov is installed
  2. Compile MariaDB with BUILD/compile-pentium-gcov (if your machine does not have a pentium CPU, hack this script, or just live with the pentium-specific stuff)
  3. In the mysql-test directory, run this command: ./mysql-test-run -gcov
  4. To see the level of coverage for a given source file:
    • grep source_file_name /tmp/gcov.out
  5. To see which lines are not yet covered, look at source_file_name.gcov in the source tree. Then think hard about a test case which will cover those lines, and write one.

Using dgcov

The dgcov tool allows to check the coverage for the new code. (Original documentation: http://forge.mysql.com/wiki/DGCov) (code: "bzr branch lp:mariadb-tools", or http://bazaar.launchpad.net/~maria-captains/mariadb-tools/trunk/view/head:/dgcov.pl)

Tools

For code coverage one can also use the lcov tool.

Work in progress

We are working on enhancing our buildbot builds to do automatic code coverage for all new pushed code.

Comments

Comments loading...