API

benchmark(;
    outpath,
    problems,
    solver_models,
    grid_sizes,
    disc_methods,
    tol,
    ipopt_mu_strategy,
    print_trace,
    max_iter,
    max_wall_time,
    grid_size_max_cpu
) -> Nothing

Run benchmarks on optimal control problems and save results to a JSON file.

This function performs the following steps:

1. Detects CUDA availability and filters out :exa_gpu if CUDA is not functional
2. Runs benchmarks using benchmark_data() to generate a DataFrame of results
3. Collects environment metadata (Julia version, OS, machine, timestamp)
4. Builds a JSON-friendly payload combining results and metadata
5. Saves the payload to outpath/data.json as pretty-printed JSON

The JSON file can be easily loaded and converted back to a DataFrame using:

using JSON, DataFrames
data = JSON.parsefile("path/to/data.json")
df = DataFrame(data["results"])

Arguments

• outpath: Path to directory where results will be saved (or nothing to skip saving)
• problems: Vector of problem names (Symbols)
• solver_models: Vector of Pairs mapping solver => models (e.g., [:ipopt => [:JuMP, :adnlp], :madnlp => [:exa, :exa_gpu]])
• grid_sizes: Vector of grid sizes (Int)
• disc_methods: Vector of discretization methods (Symbols)
• tol: Solver tolerance (Float64)
• ipopt_mu_strategy: Mu strategy for Ipopt (String)
• print_trace: Boolean - whether to print solver output (for debugging)
• max_iter: Maximum number of iterations (Int)
• max_wall_time: Maximum wall time in seconds (Float64)
• grid_size_max_cpu: Maximum grid size for CPU models (Int)

Returns

• nothing

benchmark_data(;
    problems,
    solver_models,
    grid_sizes,
    disc_methods,
    tol,
    ipopt_mu_strategy,
    print_trace
    max_iter,
    max_wall_time,
    grid_size_max_cpu
) -> DataFrame

Run benchmarks on optimal control problems and return results as a DataFrame.

For each combination of problem, solver, model, and grid size, this function:

1. Sets up and solves the optimization problem
2. Captures timing and memory statistics using @btimed or CUDA.@timed
3. Extracts solver statistics (objective value, iterations)
4. Stores all data in a DataFrame row

Arguments

• problems: Vector of problem names (Symbols)
• solver_models: Vector of Pairs mapping solver => models (e.g., [:ipopt => [:JuMP, :adnlp], :madnlp => [:exa, :exa_gpu]])
• grid_sizes: Vector of grid sizes (Int)
• disc_methods: Vector of discretization methods (Symbols)
• tol: Solver tolerance (Float64)
• ipopt_mu_strategy: Mu strategy for Ipopt (String)
• print_trace: Boolean - whether to print solver output (for debugging)
• max_iter: Maximum number of iterations (Int)
• max_wall_time: Maximum wall time in seconds (Float64)

Returns

A DataFrame with columns:

• problem: Symbol - problem name
• solver: Symbol - solver used (:ipopt or :madnlp)
• model: Symbol - model type (:JuMP, :adnlp, :exa, or :exa_gpu)
• disc_method: Symbol - discretization method
• grid_size: Int - number of grid points
• tol: Float64 - solver tolerance
• mu_strategy: Union{String, Missing} - mu strategy for Ipopt (missing for MadNLP)
• print_level: Any - print level for solver (Int for Ipopt, MadNLP.LogLevels for MadNLP)
• max_iter: Int - maximum number of iterations
• max_wall_time: Float64 - maximum wall time in seconds
• benchmark: NamedTuple - full benchmark object from @btimed or CUDA.@timed
• objective: Union{Float64, Missing} - objective function value (missing if failed)
• iterations: Union{Int, Missing} - number of solver iterations (missing if failed)
• status: Any - termination status (type depends on solver/model)
• success: Bool - whether the solve succeeded

build_payload(results::DataFrame, meta::Dict) -> Dict

Combine benchmark results DataFrame and metadata into a JSON-friendly dictionary. The DataFrame is converted to a vector of dictionaries (one per row) for easy JSON serialization and reconstruction.

filter_models_for_backend(models::Vector{Symbol}, disc_method::Symbol) -> Vector{Symbol}

Filter solver models depending on backend availability and discretization support.

• GPU models (ending with _gpu) are kept only if CUDA is available.
• JuMP models are kept only when disc_method == :trapeze.

generate_metadata() -> Dict{String, String}

Return metadata about the current environment:

• timestamp (UTC, ISO8601)
• julia_version
• os
• machine hostname
• pkg_status - output of Pkg.status() with ANSI colors
• versioninfo - output of versioninfo() with ANSI colors
• pkg_manifest - output of Pkg.status(mode=PKGMODE_MANIFEST) with ANSI colors

is_cuda_on() -> Bool

Return true if CUDA is functional on this machine.

prettymemory(bytes::Integer) -> String

Format a memory footprint bytes into a human-readable string using binary prefixes (bytes, KiB, MiB, GiB) with two decimal places.

prettytime(t::Real) -> String

Format a duration t expressed in seconds into a human-readable string with three decimal places and adaptive units (ns, μs, ms, s).

print_benchmark_line(model::Symbol, stats::NamedTuple)

Print a formatted line summarizing benchmark statistics for model with colors. Handles both CPU benchmarks (from @btimed) and GPU benchmarks (from CUDA.@timed).

Displays: time, allocations/memory, objective, iterations, and success status

Run the benchmarks for a specific version.

Arguments

• version::Symbol: version to run (:complete or :minimal)
• outpath::Union{AbstractString, Nothing}: directory path to save results (nothing for no saving)
• print_trace::Bool: whether to print the trace of the solver

Returns

• nothing

sanitize_for_json(obj)

Recursively replace NaN and Inf values with null for JSON compatibility.

save_json(payload::Dict, outpath::AbstractString)

Save a JSON payload to a file. Creates the parent directory if needed. Uses pretty printing for readability. Sanitizes NaN and Inf values to null for JSON compatibility.

set_print_level(solver::Symbol, print_trace::Bool) -> Int

Set print level based on solver and print_trace flag.

solve_and_extract_data(problem, solver, model, grid_size, disc_method, 
                      tol, mu_strategy, print_level, max_iter, max_wall_time) -> NamedTuple

Solve an optimal control problem and extract performance and solver statistics.

This internal helper function handles the solve process and data extraction for different model types (JuMP, adnlp, exa, exa_gpu).

Arguments

• problem::Symbol: problem name (e.g., :beam, :chain)
• solver::Symbol: solver to use (:ipopt or :madnlp)
• model::Symbol: model type (:JuMP, :adnlp, :exa, or :exa_gpu)
• grid_size::Int: number of grid points
• disc_method::Symbol: discretization method
• tol::Float64: solver tolerance
• mu_strategy::Union{String, Missing}: mu strategy for Ipopt (missing for MadNLP)
• print_level::Union{Int, MadNLP.LogLevels, Missing}: print level for solver (Int for Ipopt, MadNLP.LogLevels for MadNLP)
• max_iter::Int: maximum number of iterations
• max_wall_time::Float64: maximum wall time in seconds

Returns

A NamedTuple with fields:

• benchmark: full benchmark object from @btimed (CPU) or CUDA.@timed (GPU)
• objective::Union{Float64, Missing}: objective function value (missing if failed)
• iterations::Union{Int, Missing}: number of solver iterations (missing if failed)
• status::Any: termination status (type depends on solver/model)
• success::Bool: whether the solve succeeded

strip_benchmark_value(bench)

Remove the value field from benchmark outputs (NamedTuple or Dict) to ensure JSON-serializable data while preserving all other statistics.