The optimal control problem object: structure and usage

In this manual, we'll first recall the main functionalities you can use when working with an optimal control problem (OCP). This includes essential operations like:

• Solving an OCP: How to find the optimal solution for your defined problem.
• Computing flows from an OCP: Understanding the dynamics and trajectories derived from the optimal solution.
• Printing an OCP: How to display a summary of your problem's definition.

After covering these core functionalities, we'll delve into the structure of an OCP. Since an OCP is structured as a OptimalControl.Model struct, we'll first explain how to access its underlying attributes, such as the problem's dynamics, costs, and constraints. Following this, we'll shift our focus to the simple properties inherent to an OCP, learning how to determine aspects like whether the problem:

• Is autonomous: Does its dynamics depend explicitly on time?
• Has a fixed or free initial/final time: Is the duration of the control problem predetermined or not?

Content

• The optimal control problem object: structure and usage
• • Main functionalities
  • Model struct
  • Times
  • State
  • Control
  • Variable
  • Dynamics
  • Objective
  • Constraints
  • Problem definition
  • Time dependence

Main functionalities

Let's define a basic optimal control problem.

using OptimalControl

t0 = 0
tf = 1
x0 = [-1, 0]

ocp = @def begin
    t ∈ [ t0, tf ], time
    x = (q, v) ∈ R², state
    u ∈ R, control
    x(t0) == x0
    x(tf) == [0, 0]
    ẋ(t)  == [v(t), u(t)]
    0.5∫( u(t)^2 ) → min
end
nothing # hide

<< @example-block not executed in draft mode >>

To print it, simply:

ocp

<< @example-block not executed in draft mode >>

We can now solve the problem (for more details, visit the solve manual):

using NLPModelsIpopt
solve(ocp)
nothing # hide

<< @example-block not executed in draft mode >>

You can also compute flows (for more details, see the flow manual) from the optimal control problem, providing a control law in feedback form. The pseudo-Hamiltonian of this problem is

\[ H(x, p, u) = p_q\, v + p_v\, u + p^0 \frac{u^2}{2},\]

where $p^0 = -1$ since we are in the normal case. From the Pontryagin maximum principle, the maximising control is given in feedback form by

\[u(x, p) = p_v\]

since $\partial^2_{uu} H = p^0 = - 1 < 0$.

u = (x, p) -> p[2]          # control law in feedback form

using OrdinaryDiffEq        # needed to import numerical integrators
f = Flow(ocp, u)            # compute the Hamiltonian flow function

p0 = [12, 6]                # initial covector solution
xf, pf = f(t0, x0, p0, tf)  # flow from (x0, p0) at time t0 to tf
xf                          # should be (0, 0)

<< @example-block not executed in draft mode >>

Model struct

The optimal control problem ocp is a OptimalControl.Model struct.

struct Model{TD<:CTModels.OCP.TimeDependence, TimesModelType<:CTModels.OCP.AbstractTimesModel, StateModelType<:CTModels.OCP.AbstractStateModel, ControlModelType<:CTModels.OCP.AbstractControlModel, VariableModelType<:CTModels.OCP.AbstractVariableModel, DynamicsModelType<:Function, ObjectiveModelType<:CTModels.OCP.AbstractObjectiveModel, ConstraintsModelType<:CTModels.OCP.AbstractConstraintsModel, DefinitionType<:CTModels.OCP.AbstractDefinition, BuildExaModelType<:Union{Nothing, Function}} <: CTModels.OCP.AbstractModel

julia> using CTModels

julia> # Models are typically created via the @def macro or PreModel
julia> ocp = CTModels.Model  # Type reference

Each field can be accessed directly (ocp.times, etc) or by a getter:

• times
• state
• control
• variable
• dynamics
• objective
• constraints
• definition

For instance, we can retrieve the times and definition values.

times(ocp)  # returns the TimesModel struct containing time information

<< @example-block not executed in draft mode >>

definition(ocp)

<< @example-block not executed in draft mode >>

To illustrate the various methods in the sections below, we define a more complex optimal control problem with free final time, variables, and various types of constraints:

ocp = @def begin
    v = (w, tf) ∈ R²,   variable
    s ∈ [0, tf],        time
    q = (x, y) ∈ R²,    state
    u ∈ R,              control
    0 ≤ tf ≤ 2,         (1)
    u(s) ≥ 0,           (cons_u)
    x(s) + u(s) ≤ 10,   (cons_mixed)
    w == 0
    x(0) == -1
    y(0) - tf == 0,     (cons_bound)
    q(tf) == [0, 0]
    q̇(s) == [y(s)+w, u(s)]
    0.5∫( u(s)^2 ) → min
end
nothing # hide

<< @example-block not executed in draft mode >>

Times

The time component defines the temporal domain of the optimal control problem.

Times model

Get the times model:

times(ocp)  # returns the TimesModel struct containing time information

<< @example-block not executed in draft mode >>

You can also access initial and final times separately:

initial_time(ocp)  # returns the initial time value

<< @example-block not executed in draft mode >>

For the final time, if it is free (part of the variable), you need to provide the variable value:

v = [1, 2]  # example variable values: w=1, tf=2
final_time(ocp, v)  # returns tf value from variable

<< @example-block not executed in draft mode >>

If you try to get the final time without providing the variable when it's free, an error occurs:

final_time(ocp)  # error: tf is free, need variable

<< @repl-block not executed in draft mode >>

Time variable names

Get the names of the time variable and time bounds:

time_name(ocp)  # returns "s" (the time variable name in this OCP)

<< @example-block not executed in draft mode >>

initial_time_name(ocp)  # returns "0" (initial time is fixed at 0)

<< @example-block not executed in draft mode >>

final_time_name(ocp)  # returns "tf" (final time is a variable)

<< @example-block not executed in draft mode >>

Time fixedness predicates

Check whether initial or final times are fixed or free:

has_fixed_initial_time(ocp)  # true if t0 is fixed

<< @example-block not executed in draft mode >>

has_free_initial_time(ocp)  # true if t0 is free (part of variable)

<< @example-block not executed in draft mode >>

Similarly for final time:

has_fixed_final_time(ocp)  # false (tf is free in this OCP)

<< @example-block not executed in draft mode >>

has_free_final_time(ocp)  # true (tf is part of variable v)

<< @example-block not executed in draft mode >>

Autonomy

Check if the dynamics and Lagrange cost are autonomous (time-independent):

is_autonomous(ocp)  # false if dynamics or cost depend on time

<< @example-block not executed in draft mode >>

For more details on autonomy, see the Time dependence section below.

Summary table

State

The state component represents the state variables of the optimal control problem.

State component information

Get the name, dimension, and component names of the state:

state_name(ocp)  # returns "q" (the state variable name)

<< @example-block not executed in draft mode >>

state_dimension(ocp)  # returns 2 (dimension of state)

<< @example-block not executed in draft mode >>

state_components(ocp)  # returns ["x", "y"] (component names)

<< @example-block not executed in draft mode >>

State box constraints

Get the box constraints on the state (lower and upper bounds):

state_constraints_box(ocp)  # returns box constraints if any

<< @example-block not executed in draft mode >>

Get the dimension of state box constraints:

dim_state_constraints_box(ocp)  # returns number of box constraints on state

<< @example-block not executed in draft mode >>

Summary table

Control

The control component represents the control variables of the optimal control problem.

Control component information

Get the name, dimension, and component names of the control:

control_name(ocp)  # returns "u" (the control variable name)

<< @example-block not executed in draft mode >>

control_dimension(ocp)  # returns 1 (dimension of control)

<< @example-block not executed in draft mode >>

control_components(ocp)  # returns ["u"] (component names)

<< @example-block not executed in draft mode >>

Control box constraints

Get the box constraints on the control:

control_constraints_box(ocp)  # returns box constraints if any

<< @example-block not executed in draft mode >>

Get the dimension of control box constraints:

dim_control_constraints_box(ocp)  # returns number of box constraints on control

<< @example-block not executed in draft mode >>

Control presence

Check whether the problem has a control input:

has_control(ocp)  # true if problem has a control input

<< @example-block not executed in draft mode >>

Summary table

Variable

The variable component represents the optimization variables (parameters) of the optimal control problem.

Variable component information

Get the name, dimension, and component names of the variable:

variable_name(ocp)  # returns "v" (the variable name)

<< @example-block not executed in draft mode >>

variable_dimension(ocp)  # returns 2 (dimension of variable)

<< @example-block not executed in draft mode >>

variable_components(ocp)  # returns ["w", "tf"] (component names)

<< @example-block not executed in draft mode >>

Variable box constraints

Get the box constraints on the variable:

variable_constraints_box(ocp)  # returns box constraints if any

<< @example-block not executed in draft mode >>

Get the dimension of variable box constraints:

dim_variable_constraints_box(ocp)  # returns number of box constraints on variable

<< @example-block not executed in draft mode >>

Variable presence

Check whether the problem has optimization variables:

has_variable(ocp)  # true if problem has optimization variables

<< @example-block not executed in draft mode >>

Summary table

Dynamics

The dynamics component defines the differential equations governing the state evolution.

Dynamics function

The dynamics are stored as an in-place function of the form f!(dx, t, x, u, v):

f! = dynamics(ocp)
s = 0.5  # time
q = [0.0, 1.0]  # state
u = 2.0  # control
v = [1.0, 2.0]  # variable
dq = similar(q)
f!(dq, s, q, u, v)
dq  # returns the derivative q̇

<< @example-block not executed in draft mode >>

The first argument dx is mutated upon call and contains the state derivative. The other arguments are:

• t: time
• x: state
• u: control
• v: variable

Summary table

Objective

The objective component defines the cost function to minimize or maximize.

Criterion

The criterion indicates whether the problem is a minimization or maximization:

criterion(ocp)  # returns :min or :max

<< @example-block not executed in draft mode >>

Objective form

The objective function can be in Mayer form, Lagrange form, or Bolza form (combination of both):

• Mayer: $g(x(t_0), x(t_f), v) \to \min$
• Lagrange: $\int_{t_0}^{t_f} f^0(t, x(t), u(t), v)\, \mathrm{d}t \to \min$
• Bolza: $g(x(t_0), x(t_f), v) + \int_{t_0}^{t_f} f^0(t, x(t), u(t), v)\, \mathrm{d}t \to \min$

Check which form is present:

has_mayer_cost(ocp)  # true if Mayer cost exists

<< @example-block not executed in draft mode >>

has_lagrange_cost(ocp)  # true if Lagrange cost exists

<< @example-block not executed in draft mode >>

Mayer cost

Get the Mayer cost function with signature g(x0, xf, v):

g = mayer(ocp)  # error if no Mayer cost

<< @repl-block not executed in draft mode >>

Lagrange cost

Get the Lagrange cost function with signature f⁰(t, x, u, v):

f⁰ = lagrange(ocp)
s = 0.5
q = [0.0, 1.0]
u = 2.0
v = [1.0, 2.0]
f⁰(s, q, u, v)  # returns the integrand value

<< @example-block not executed in draft mode >>

Summary table

Constraints

The constraints component defines the constraints on the optimal control problem.

Individual constraints

Retrieve a specific constraint by its label using the constraint function. It returns a tuple (type, f, lb, ub):

(type, f, lb, ub) = constraint(ocp, :eq1)
println("type: ", type)
x0 = [0, 1]
xf = [2, 3]
v  = [1, 4]
println("val: ", f(x0, xf, v))
println("lb: ", lb)
println("ub: ", ub)

<< @example-block not executed in draft mode >>

The function signature depends on the constraint type:

• For :boundary and :variable constraints: f(x0, xf, v)
• For other constraints (:control, :state, :mixed): f(t, x, u, v)

Examples of different constraint types:

(type, f, lb, ub) = constraint(ocp, :cons_bound)
println("type: ", type)
println("val: ", f(x0, xf, v))

<< @example-block not executed in draft mode >>

(type, f, lb, ub) = constraint(ocp, :cons_u)
println("type: ", type)
s = 0.5
q = [1.0, 2.0]
u = 3.0
println("val: ", f(s, q, u, v))

<< @example-block not executed in draft mode >>

(type, f, lb, ub) = constraint(ocp, :cons_mixed)
println("type: ", type)
println("val: ", f(s, q, u, v))

<< @example-block not executed in draft mode >>

All constraints

Get all constraints as a collection:

constraints(ocp)  # returns all constraints

<< @example-block not executed in draft mode >>

Nonlinear constraints

Get nonlinear path and boundary constraints:

path_constraints_nl(ocp)  # returns nonlinear path constraints

<< @example-block not executed in draft mode >>

boundary_constraints_nl(ocp)  # returns nonlinear boundary constraints

<< @example-block not executed in draft mode >>

Get the dimensions of nonlinear constraints:

dim_path_constraints_nl(ocp)  # number of nonlinear path constraints

<< @example-block not executed in draft mode >>

dim_boundary_constraints_nl(ocp)  # number of nonlinear boundary constraints

<< @example-block not executed in draft mode >>

Summary table

Problem definition

Get the problem definition as a string:

definition(ocp)  # returns the OCP definition as AbstractDefinition

<< @example-block not executed in draft mode >>

To extract the expression from the definition, use:

expr = expression(ocp)  # returns the Expr from the definition
nothing # hide

<< @example-block not executed in draft mode >>

Definition presence

Check whether the problem carries an abstract definition:

has_abstract_definition(ocp)  # true if definition is present (not EmptyDefinition)

<< @example-block not executed in draft mode >>

Time dependence

Optimal control problems can be autonomous or non-autonomous. In an autonomous problem, neither the dynamics nor the Lagrange cost explicitly depends on the time variable.

The following problem is autonomous.

ocp = @def begin
    t ∈ [ 0, 1 ], time
    x ∈ R, state
    u ∈ R, control
    ẋ(t)  == u(t)                       # no explicit dependence on t
    x(1) + 0.5∫( u(t)^2 ) → min         # no explicit dependence on t
end
is_autonomous(ocp)

<< @example-block not executed in draft mode >>

The following problem is non-autonomous since the dynamics depends on t.

ocp = @def begin
    t ∈ [ 0, 1 ], time
    x ∈ R, state
    u ∈ R, control
    ẋ(t)  == u(t) + t                   # explicit dependence on t
    x(1) + 0.5∫( u(t)^2 ) → min
end
is_autonomous(ocp)

<< @example-block not executed in draft mode >>

Finally, this last problem is non-autonomous because the Lagrange part of the cost depends on t.

ocp = @def begin
    t ∈ [ 0, 1 ], time
    x ∈ R, state
    u ∈ R, control
    ẋ(t)  == u(t)
    x(1) + 0.5∫( t + u(t)^2 ) → min     # explicit dependence on t
end
is_autonomous(ocp)

<< @example-block not executed in draft mode >>

The variant predicate is_nonautonomous is also available and returns the opposite of is_autonomous:

is_nonautonomous(ocp)  # true if dynamics or cost depend on time

<< @example-block not executed in draft mode >>