Photo by IvÃ¡n DÃaz onÂ Unsplash

#### A soft guide to solving differential algebraic equations and applying dynamic optimization

With our world becoming more and more digitized, data collection is expanding rapidly. This data has allowed us to create more accurate models that have helped us to solve problems and find optimized solutions in many fields of business and technology. Since these models are built on real world data, which is stochastic by nature, things such as randomness, noise, and anomalies within the datasets are often inevitable. Much time is often spent doing EDA, data preprocessing, and outlier detection. These methods have become commonplace to understand and refine the data prior to fitting a model. However, any model built using such data is always an approximate to the real solution. This is due to data again being stochastic (Aleatoric uncertainty) and subject to many unknown influences (Knightian uncertainty). Along with these, a lack of sufficient data or having biased imbalanced datasets can further degrade a modelâ€™s accuracy (Epistemic uncertainty). Even against these odds, models built in this manner are invaluable and the inherent stochasticity can sometimes be a tool to help better understand the intricacies of the real world. Howeverâ€¦ an alternative approach exists: enter deterministic modeling.

Deterministic modeling is the antithesis to modeling with randomness. It is a method to simplify certain aspects of the modeling process by removing the messiness introduced by the real world. These models are idealistic in nature, meaning that they represent an ideal solution based on specific assumptions. These assumptions can stem from a set of opinionated, probabilistic, or generalized rules, and assumes that a well-defined pattern exists between the inputs and outputs. While this may sound similar to modeling with stochastic data, the main difference is that the model now is determined by the these predefined assumptions rather than training data. A question might now be popping into your head asking, â€œ*well if Iâ€™m not training the model on the data, where do I get these model assumptions from*?â€�. The answer to this is quite simple yet sometimes overlooked, **domain knowledge.** Domain knowledge is a set of rules and concepts known on a specific topic. It is through domain knowledge that the assumptions are derived, serving as the foundation for deterministic modeling. In any programming language these deterministic models can be written using custom code, but as models become more complex, libraries have been written to specifically tackle this task. In python one such library called *GEKKO* will be explored.

*GEKKO* is a Python library to facilitate the execution of the modeling language Advanced process monitor (**APMonitor**). It offers a range of features and functions tailored to the needs of deterministic modeling, facilitating the creation and analysis of complex models. It can solve both mixed-integer and differential algebraic equations, and is coupled with large-scale solvers for linear, quadratic, nonlinear, and mixed integer programming. *GEKKO *has 9 problem types with which to define a deterministic model setup. However, for this article, only a Moving Horizon Estimation (**MHE**) that solves all equations simultaneously will be considered. The MHE mode is used to estimate the states of a dynamic system by minimizing the discrepancy between the measured outputs and the model predictions formulated by an optimization problem with an objective. This is done by applying a recursive estimation or a moving window estimation.

Whew! That was quite a lot a jargon, but it should all make sense soon with the following practical example.

Space flight is an incredible human achievement. The feat of sending something man-made into space is a marvel of both engineering and ingenuity. But to send something into space is pretty expensive. So, building rockets with confidence of their ability to perform as expected is crucial. Collecting data of a real live launch is great to help improve a future rocketâ€™s performance. But a rocket had to be launched before that to get the data, and therein lies the problem. No stochastic data no stochastic based model. Enter the deterministic model and *GEKKO*. The assumptions for a deterministic model of a rocket are quite simply, you guessed it, rocket science! That is to say, that without even having to launch a rocket one inch off the ground, we can instead use the laws of physics to determine how a specific rocket setup will perform in an idealistic scenario. For example, letâ€™s replicate SpaceXâ€™s Falcon 1 Stage 1 rocket setup usingÂ *GEKKO*.

To start, letâ€™s use publicly known information [1][2] for the Falcon 1 Stage 1 rocket setup and begin by defining some simulation constants and knownÂ values.

https://medium.com/media/9098d65f53c82110a7492b4f71a572e2/href

Now we can really start with the cool stuff in *GEKKO*. Since we are working with a temporal based simulation, letâ€™s first define our time components:

The rocket flight time will be 200Â secondsThe fuel burn time is a known 161.6Â secondsAnd an additional time component to use forÂ later

With the time array setup, we can initialize a *GEKKO* model and set the modelâ€™s time component. The next step is to define the variables that must be solved at every time step. These variables are determined by the assumptions we will make for our rocket, but they can also include variables that we might be interested in evaluating at the end of the simulation. To keep it simple for this simulation the variables are:

MassDistanceVelocityForce generated byÂ thrustAdditional dummy time variable to use forÂ later

For each of these variables we can give it a starting value, and also set lower and upper bounds to constrain them. *GEKKO* variables are the most generic way to define variables that we are interested in. However, there are actually quite a few other options such as Fixed, Controlled, and Manipulated variables that give the user much more control on how they are allowed to evolve. You should check themÂ out!

https://medium.com/media/c78b9528f897417a9dda60f052d5c430/href

Before we continue, it is important to note that for the sake of your reading time and not to be too math heavy, this rocket model will neglect quite a few known assumptions on rocket physics. These include, the change in air density and gravity as the rockets ascends, the effects of the Coriolis force, and a good many mechanical engineering components. However, with the assumptions that are made, the main forces affecting a rocketâ€™s flight and dynamics will be covered. And now, it is probably a good time to introduce them as well as how simple it is to apply them inÂ *GEKKO*.

As with any classical moving object, Newtonâ€™s laws of motion apply. This means that we already have a good idea how this rocket is going to move. But letâ€™s quickly recap on the differential equations we will need to do this. Donâ€™t worry they are really simple. Letâ€™s start with defining the equation for how the rocket will move and cover distance,

Wow! This one is basic physics, *the objectâ€™s velocity is simply a change in distance over a change in time. *Next, lets add the resistive force that the rocket will experience as it moves through the air, P.S itâ€™s such aÂ drag.

With that, we can now make another relationship between force and velocity. Remember,

Well, with this we can relate all the forces the rocket will experience (forward thrust, drag, and of course gravity) with velocity. Hereâ€™sÂ how,

I promise that was the hardest equation for this simulation. Only two more assumptions will be made for our rocket before it takes off. One of them has to do with the fuel in the rocket. If the rocket is burning up itâ€™s fuel as a propellant, a good idea then is to describe how the mass of the rocket must change. To do this, letâ€™s just say that the loss of mass has a linear relationship with the maximum thrust and the burn time likeÂ this,

The final assumption is really what happens when the fuel is all used up. This one is quite cool because it is a logical statement and *GEKKO* has something for thisÂ too.

With all this done, we can now put all the pieces together and build our deterministic model.

https://medium.com/media/3be04d3ed76991a67ec905150003b224/href

Note that there are some extra things in the code above that were not mentioned. First a dummy time equation was called, this is to keep track of the time for when the fuel ends. Next, an optimization objective was used. Why was this done? Well, the MHE solves for an objective, in this case to try have the rocket reach the greatest height from the ground at every time step. You can try other objective functions based on what you want to optimize for (e.g., give m.fix_final(h, 50e3) and then m.Obj(thrust*v) a whirl to try and be thrifty with your rocket juice). One last thing is that *GEKKO* will print out whether the simulation was able to find an optimal solution or not. For this simulation, you will see that we actually had negative degrees of freedom. This is a quirk of using m.if3 and this can usually cause the simulation to not solve. So, if you want to try something more elaborate, perhaps stick with m.if2Â instead.

Finally, it’s time to see the fruits of our simulation. To get a good and quick look at what the simulation solved, plotting the results is a great way to visualize it.

https://medium.com/media/5267e6b5031b391af5bdbfbb3e7a3f4a/href

Analyzing the rocketâ€™s performance provides some interesting insights. Firstly, it achieved in reaching a maximum height of 73.83 km and maximum speed of 802.15 m/s. Secondly, the rocket had an average thrust of 322.68 kN while burning through a whopping 19.59 GJ of energy during itsÂ flight.

These results offer us crucial information about the rocketâ€™s capabilities and performance even before building said rocket. Even better, the parameters can easily be adjusted and fine-tuned along with increasingly added complexity to truly showcase the incredible potential deterministic modeling brings to the table. With all this information available, such knowledge can be further used to aid engineers in design optimizations and help budget for such costly projects.

While rockets are cool and all, more realistic applications of deterministic modeling include optimizations of many different types of systems. Examples of such systems are: the processing line at a manufacturing plant, traffic control and route planning, simulating the complex spread of infections and diseases, modeling climate changes and their effects, and even using deterministic-based game theory models inÂ finance.

To wrap up, deterministic modeling is a great option to find solutions for idealistic scenarios that can be well-described by domain knowledge. They can be used to model complex non-linear systems and provided strong predictive and probabilistic outcomes prior to real world data availability. All in all, deterministic modeling is a powerful tool for those who wish to masterÂ it.

[1] Propulsion Falcon-I** **(http://www.b14643.de/Spacerockets_2/United_States_2/Falcon-I/Propulsion/engines.htm)

[2] Falcon 1 (https://en.wikipedia.org/wiki/Falcon_1)

Modeling the World in a Deterministic Way with GEKKO was originally published in Towards Data Science on Medium, where people are continuing the conversation by highlighting and responding to this story.