Coverage Post-processing Guide

This guide explains how to generate human-readable and machine-parseable coverage reports using the CoveragePostprocessing extension of CTBase.jl.

⚠️ Prerequisites

Important: The Coverage package must be available when running coverage.jl. The simplest approach is to add it to your test environment:

julia --project=test -e 'using Pkg; Pkg.add("Coverage")'

Alternatively, install it globally so it is always accessible:

julia -e 'using Pkg; Pkg.add("Coverage")'

Setting up Coverage

To generate actionable coverage reports, we use a dedicated coverage.jl script. This script processes the raw .cov files generated by Julia and produces summaries that are easy to read.

Example test/coverage.jl

# Add the test directory to the load path so Julia can find dependencies from
# test/Project.toml.
pushfirst!(LOAD_PATH, @__DIR__)

using Pkg
using CTBase # Provides CTBase.postprocess_coverage
using Coverage

# This function:
# 1. Resets the coverage/ directory.
# 2. Removes stale .cov files, keeping the most complete PID run.
# 3. Generates an LCOV file (coverage/lcov.info).
# 4. Generates a markdown summary (coverage/cov_report.md).
# 5. Moves .cov files into coverage/cov/.
CTBase.postprocess_coverage(;
    root_dir=dirname(@__DIR__),  # Point to the package root
    # dest_dir="coverage",       # Output directory (default: "coverage")
    # worst_n_files=20,          # Number of lowest-covered files to list
    # max_uncovered_lines=200,   # Max uncovered lines to show
)

Running with Coverage

To run tests and generate the report:

julia --project -e '
    using Pkg; 
    Pkg.test("MyPackage"; coverage=true); 
    include("test/coverage.jl")
'

The resulting coverage/cov_report.md will contain a list of files with their coverage percentages and, crucially, a list of uncovered lines. This allows identifying exactly which parts of the code need more tests.

Coverage Artifacts

The post-processing script produces:

• coverage/lcov.info: LCOV format for CI integration (e.g., Codecov).

• coverage/cov_report.md: Human-readable summary with uncovered lines.

• coverage/cov/: Archived .cov files.

Complete Workflow

Here's a complete workflow from running tests to analyzing coverage:

# 1. Run tests with coverage enabled
julia --project -e 'using Pkg; Pkg.test("MyPackage"; coverage=true)'

# 2. Generate coverage report (resets coverage/ automatically)
julia --project -e 'include("test/coverage.jl")'

# 3. View the report
cat coverage/cov_report.md

Troubleshooting

Issue: Coverage package not found

Error: ArgumentError: Package Coverage not found in current path

Solution: Install Coverage in your base Julia environment:

julia --project=@v1.12 -e 'using Pkg; Pkg.add("Coverage")'

Issue: No .cov files generated

Problem: Tests run but no coverage data is collected

Solution: Ensure you're running tests with coverage=true:

julia --project -e 'using Pkg; Pkg.test("MyPackage"; coverage=true)'

Issue: Coverage report shows zero percent for all files

Problem: Coverage data exists but shows no coverage

Solution: Check that your package source is in the src/ directory and that the root_dir parameter in coverage.jl points to the correct location.

Issue: ExtensionError when running coverage

Error: ExtensionError: missing dependencies

Solution: The Coverage package must be available. Install it globally:

julia -e 'using Pkg; Pkg.add("Coverage")'

Understanding Coverage Reports

The cov_report.md file has three sections:

Overall

A global summary computed by Coverage.get_summary:

## Overall

```shell
(826, 854)

Global coverage: 96.72% (826, 854)
```

The tuple (covered_lines, total_lines) is followed by the computed percentage.

Lowest-covered files

A markdown table of the worst_n_files least-covered files (default: 20), sorted by ascending coverage:

## Lowest-covered files (top 20)

| file | covered | total | % |
|---|---:|---:|---:|
| `src/MyModule.jl` | 60 | 70 | 85.71 |
| `ext/MyExt/foo.jl` | 10 | 20 | 50.0  |

Only src/ and ext/ files are included (test files are excluded).

Uncovered lines

A shell code block listing filename:line pairs, up to max_uncovered_lines entries (default: 200):

## Uncovered lines (first 200)

```shell
src/MyModule.jl:15
src/MyModule.jl:23
src/MyModule.jl:24
src/MyModule.jl:42
```

Interpreting Results

• High coverage (>90%): Good test coverage, most code paths tested

• Medium coverage (70-90%): Acceptable, but room for improvement

• Low coverage (<70%): Needs more tests, many code paths untested

Focus on:

1. Critical paths: Ensure core functionality is well-tested

2. Error handling: Test exception paths

3. Edge cases: Test boundary conditions

Best Practices

1. Run coverage regularly: Include in your development workflow

2. Focus on quality, not just quantity: 100% coverage doesn't mean bug-free code

3. Test meaningful paths: Cover important logic, not just trivial getters

4. Use coverage to find gaps: Identify untested code paths

5. Integrate with CI: Automate coverage reporting in your CI pipeline

6. Set coverage thresholds: Maintain or improve coverage over time

7. Review uncovered lines: Understand why code isn't covered

CI/CD Integration

GitHub Actions with Codecov

name: Coverage
on: [push, pull_request]
jobs:
  coverage:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v2
      - uses: julia-actions/setup-julia@v1
      - name: Install Coverage
        run: julia -e 'using Pkg; Pkg.add("Coverage")'
      - name: Run tests with coverage
        run: julia --project -e 'using Pkg; Pkg.test(coverage=true)'
      - name: Process coverage
        run: julia --project -e 'include("test/coverage.jl")'
      - name: Upload to Codecov
        uses: codecov/codecov-action@v3
        with:
          files: coverage/lcov.info

See Also

• Test Runner Guide: Setting up modular tests

• Exception Handling: Testing exception paths