Skip to content

Mountain car ​

  • 2 state
  • 1 control
  • 1 variable
  • Mayer cost
  • free final time
  • 2 state box
  • 1 control box
  • 1 variable box
  • 4 boundary
  • OptimalControl valid
  • OptimalControl, scalarized valid
  • JuMP valid
  • ModelingToolkit valid

The Mountain Car problem is a classic benchmark in control and reinforcement learning. It involves an underpowered car that must climb a steep hill to reach a target position. Because the car's engine is too weak to climb the hill directly, it must first drive away from the goal to gain momentum by oscillating in a valley. The goal is to reach the target position in minimum time.

Mathematical formulation ​

The problem can be stated as

Qualitative behaviour ​

  • The car must oscillate back and forth in the valley to build enough kinetic energy to climb the right hill.

  • The optimal strategy typically involves a sequence of full-throttle accelerations in alternating directions (bang-bang control).

  • The minimum time objective makes the problem sensitive to the initial guess for the final time.

Characteristics ​

  • Serves as a benchmark for minimum-time problems whose solution first moves away from the goal.

References ​

  • Dymos Documentation. The Mountain Car Problem. Provides a detailed description and implementation of the Mountain Car problem using the Dymos library.

  • OpenAI Gym. MountainCar-v0. A standard reinforcement learning environment for the Mountain Car problem.

  • Moore, A. W. (1990). Efficient Memory-based Learning for Robot Control. PhD thesis, University of Cambridge. Introduces the Mountain Car problem as a benchmark for reinforcement learning and control.

Parameters ​

ParameterDefault valueUnitMeaning
t00.0–initial time
tf_start110.0–final time of the initial guess
tf_min0.1–lower bound of the final time
pos_t0-0.5–initial position
vel_t00.0–initial velocity
pos_tf0.5–final position, the top of the hill
vel_tf_min0.0–lower bound of the final velocity
pos_min-1.2–lower bound of the position
pos_max0.5–upper bound of the position
vel_min-0.07–lower bound of the velocity
vel_max0.07–upper bound of the velocity
u_min-1.0–lower bound of the control u
u_max1.0–upper bound of the control u
a0.001–gain of the control (engine power)
b0.0025–coefficient of the gravity term b cos(c pos)
c3.0–frequency of the hill profile

Every modeling takes other values with the keyword parameters, merged with these: mountain_car(backend; parameters=(t0=…,)).

Reference objective ​

The best objective among the solves that converged, in each modeling, on the default grid of the modeling: the modelings are independent, each has its own reference. See how the references are computed.

ModelingSchemeGridReference objectiveSolvesDistinct solutionsConfigurations agree
JuMP:trapeze200102.380995282no
OptimalControl:trapeze200102.380995182no
OptimalControl, scalarized:trapeze200102.380995231yes, within 1.4e-14

In JuMP with :trapeze, the solves reached 2 distinct solutions: besides the best one, 102.3975702 (gap 1.6e-04, 4 solves). See the results of the modeling for which configurations reached which.

In OptimalControl with :trapeze, the solves reached 2 distinct solutions: besides the best one, 102.39757 (gap 1.6e-04, 7 solves). See the results of the modeling for which configurations reached which.