Coverage Post-processing Guide

This guide explains how to generate human-readable and machine-parseable coverage reports using the CoveragePostprocessing extension of CTBase.jl.

⚠️ Prerequisites

Important: The Coverage package must be installed in your base Julia environment for coverage post-processing to work properly:

# In your base Julia environment (not the project environment)
julia --project=@v1.12 -e 'using Pkg; Pkg.add("Coverage")'

This is required because coverage processing happens at the Julia level and needs the Coverage package to be available globally.

Setting up Coverage

To generate actionable coverage reports, we use a dedicated coverage.jl script. This script processes the raw .cov files generated by Julia and produces summaries that are easy to read.

Example `test/coverage.jl`

# Add the test directory to the load path so Julia can find dependencies from 
# test/Project.toml.
pushfirst!(LOAD_PATH, @__DIR__)

using Pkg
using CTBase # Provides postprocess_coverage
using Coverage

# This function:
# 1. Aggregates coverage data.
# 2. Generates an LCOV file (coverage/lcov.info).
# 3. Generates a markdown summary (coverage/cov_report.md).
# 4. Archives used .cov files to keep the directory clean.
CTBase.postprocess_coverage(; 
    root_dir=dirname(@__DIR__) # Point to the package root
)

Running with Coverage

To run tests and generate the report:

julia --project -e '
    using Pkg; 
    Pkg.test("MyPackage"; coverage=true); 
    include("test/coverage.jl")
'

The resulting coverage/cov_report.md will contain a list of files with their coverage percentages and, crucially, a list of uncovered lines. This allows identifying exactly which parts of the code need more tests.

Coverage Artifacts

The post-processing script produces:

coverage/lcov.info: LCOV format for CI integration (e.g., Codecov).
coverage/cov_report.md: Human-readable summary with uncovered lines.
coverage/cov/: Archived .cov files.

Complete Workflow

Here's a complete workflow from running tests to analyzing coverage:

# 1. Clean previous coverage data
rm -rf coverage/

# 2. Run tests with coverage enabled
julia --project -e 'using Pkg; Pkg.test("MyPackage"; coverage=true)'

# 3. Generate coverage report
julia --project -e 'include("test/coverage.jl")'

# 4. View the report
cat coverage/cov_report.md

Troubleshooting

Issue: Coverage package not found

Error: ArgumentError: Package Coverage not found in current path

Solution: Install Coverage in your base Julia environment:

julia --project=@v1.12 -e 'using Pkg; Pkg.add("Coverage")'

Issue: No .cov files generated

Problem: Tests run but no coverage data is collected

Solution: Ensure you're running tests with coverage=true:

julia --project -e 'using Pkg; Pkg.test("MyPackage"; coverage=true)'

Issue: Coverage report shows 0% for all files

Problem: Coverage data exists but shows no coverage

Solution: Check that your package source is in the src/ directory and that the root_dir parameter in coverage.jl points to the correct location.

Issue: ExtensionError when running coverage

Error: ExtensionError: missing dependencies

Solution: The Coverage package must be available. Install it globally:

julia -e 'using Pkg; Pkg.add("Coverage")'

Understanding Coverage Reports

The cov_report.md file contains:

File Coverage Summary

File: src/MyModule.jl
Coverage: 85.7% (60/70 lines)
Uncovered lines: 15, 23-25, 42, 58-60

This shows:

Total coverage: 85.7% of lines are executed
Line counts: 60 out of 70 lines covered
Uncovered lines: Specific line numbers that need tests

Interpreting Results

High coverage (>90%): Good test coverage, most code paths tested
Medium coverage (70-90%): Acceptable, but room for improvement
Low coverage (<70%): Needs more tests, many code paths untested

Focus on:

Critical paths: Ensure core functionality is well-tested
Error handling: Test exception paths
Edge cases: Test boundary conditions

Best Practices

Run coverage regularly: Include in your development workflow
Focus on quality, not just quantity: 100% coverage doesn't mean bug-free code
Test meaningful paths: Cover important logic, not just trivial getters
Use coverage to find gaps: Identify untested code paths
Integrate with CI: Automate coverage reporting in your CI pipeline
Set coverage thresholds: Maintain or improve coverage over time
Review uncovered lines: Understand why code isn't covered

CI/CD Integration

GitHub Actions with Codecov

name: Coverage
on: [push, pull_request]
jobs:
  coverage:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v2
      - uses: julia-actions/setup-julia@v1
      - name: Install Coverage
        run: julia -e 'using Pkg; Pkg.add("Coverage")'
      - name: Run tests with coverage
        run: julia --project -e 'using Pkg; Pkg.test(coverage=true)'
      - name: Process coverage
        run: julia --project -e 'include("test/coverage.jl")'
      - name: Upload to Codecov
        uses: codecov/codecov-action@v3
        with:
          files: coverage/lcov.info