Mountain car
- 2 state
- 1 control
- 1 variable
- Mayer cost
- free final time
- 2 state box
- 1 control box
- 1 variable box
- 4 boundary
- OptimalControl valid
- OptimalControl, scalarized valid
- JuMP valid
- ModelingToolkit valid
The Mountain Car problem is a classic benchmark in control and reinforcement learning. It involves an underpowered car that must climb a steep hill to reach a target position. Because the car's engine is too weak to climb the hill directly, it must first drive away from the goal to gain momentum by oscillating in a valley. The goal is to reach the target position in minimum time.
Mathematical formulation
The problem can be stated as
Qualitative behaviour
The car must oscillate back and forth in the valley to build enough kinetic energy to climb the right hill.
The optimal strategy typically involves a sequence of full-throttle accelerations in alternating directions (bang-bang control).
The minimum time objective makes the problem sensitive to the initial guess for the final time.
Characteristics
- Serves as a benchmark for minimum-time problems whose solution first moves away from the goal.
References
Dymos Documentation. The Mountain Car Problem. Provides a detailed description and implementation of the Mountain Car problem using the Dymos library.
OpenAI Gym. MountainCar-v0. A standard reinforcement learning environment for the Mountain Car problem.
Moore, A. W. (1990). Efficient Memory-based Learning for Robot Control. PhD thesis, University of Cambridge. Introduces the Mountain Car problem as a benchmark for reinforcement learning and control.
Parameters
| Parameter | Default value | Unit | Meaning |
|---|---|---|---|
t0 | 0.0 | – | initial time |
tf_start | 110.0 | – | final time of the initial guess |
tf_min | 0.1 | – | lower bound of the final time |
pos_t0 | -0.5 | – | initial position |
vel_t0 | 0.0 | – | initial velocity |
pos_tf | 0.5 | – | final position, the top of the hill |
vel_tf_min | 0.0 | – | lower bound of the final velocity |
pos_min | -1.2 | – | lower bound of the position |
pos_max | 0.5 | – | upper bound of the position |
vel_min | -0.07 | – | lower bound of the velocity |
vel_max | 0.07 | – | upper bound of the velocity |
u_min | -1.0 | – | lower bound of the control u |
u_max | 1.0 | – | upper bound of the control u |
a | 0.001 | – | gain of the control (engine power) |
b | 0.0025 | – | coefficient of the gravity term b cos(c pos) |
c | 3.0 | – | frequency of the hill profile |
Every modeling takes other values with the keyword parameters, merged with these: mountain_car(backend; parameters=(t0=…,)).
Reference objective
The best objective among the solves that converged, in each modeling, on the default grid of the modeling: the modelings are independent, each has its own reference. See how the references are computed.
| Modeling | Scheme | Grid | Reference objective | Solves | Distinct solutions | Configurations agree |
|---|---|---|---|---|---|---|
| JuMP | :trapeze | 200 | 102.3809952 | 8 | 2 | no |
| OptimalControl | :trapeze | 200 | 102.380995 | 18 | 2 | no |
| OptimalControl, scalarized | :trapeze | 200 | 102.3809952 | 3 | 1 | yes, within 1.4e-14 |
In JuMP with :trapeze, the solves reached 2 distinct solutions: besides the best one, 102.3975702 (gap 1.6e-04, 4 solves). See the results of the modeling for which configurations reached which.
In OptimalControl with :trapeze, the solves reached 2 distinct solutions: besides the best one, 102.39757 (gap 1.6e-04, 7 solves). See the results of the modeling for which configurations reached which.