moptipyapps.dynamic_control.controllers package¶
Several possible controllers for the dynamic control scenarios.
linearoffers simple linear controllers.quadraticoffers simple quadratic controllers.cubicoffers simple cubic controllers.annoffers poor man’s artificial neural networks (ANNs), i.e., networks represented as plain functions whose parameter vectors are subject to optimization.predefinedprovides a set of controllers taken from the works of NOACK, CORNEJO MACEDA, LI, and SUN.partially_linearoffers several linear controllers anchored at certain points in the state space and always uses the controller closest to the current state.min_annoffers controllers similar tomin_ann, but instead of using the ANN output as controller value, these ANNs have an additional input z and then use the value z* as controller output for which the ANNs take on the smallest value.peaksprovides controllers similar to ANNs but using peak functions (exp(-a²)) as activation functions.
These controller blueprints can be used in two ways. The basic use case is to synthesize, well, controllers for the dynamic system. In the dynamic system controller synthesis problem, we have a system of differential equations D=ds/dt where D depends on the current system state s and the output of a controller c. So D is actually a function D(s, c). Now the goal is to get the dynamic system to move into a nice and stable state. We want to find a controller c that can do this. Now c is actually a function c(s, p) that takes as input the current system state s and its own parameterization p. Imagine c to be, for example, an artificial neural network (an ann), then p is its weight vector and p is subject to optimization. We would try to find the values p that minimize a certain objective function that could, let’s say, represent the cost or energy in the system. So use case number one of our controllers is to represent the controller blueprints for this process.
Use case number two is to use the controllers as blueprints for system models M. Basically, a system model M should be a drop-in replacement for the system equations D. In the real world, obviously, we do not have the system equations that govern a complex system like a helicopter or something. But we can test a helicopter controller in a wind tunnel. In such a scenario, evaluating the objective function is very costly. If, instead, we could learn a model M that can reasonably accurately describe the helicopter-experiment behavior D, then we could replace the actual experiment with a simulation. Since computing power is cheap, that would be awesome. Now it turns out that the structure of the equations, i.e., a parameterized function relating inputs to outputs, that we need for this, is pretty much the same as in use case number one above. So we can re-use the same code, the basic Poor man’s Artificial Neural Networks. Here, artificial neural networks (ANNs) are defined as plain mathematical functions which are parameterized by their weights. The weights are subject to black-box optimization and all together put into a single vector. In other words, we do not use proper back-propagation learning or any other sophisticated neural network specific training strategy. Instead, we treat the neural networks as black boxes that can be parameterized using the weight vector. Different ANN architectures have different weight vectors. As activation functions, we use arctan. The neural networks here are automatically generated via code generation provided by module Create poor man’s ANNs fitting to a given system. Based on the dimensionality of the state space, we generate a set of ANNs with different numbers of layers and neurons. The weights of the neurons can then directly be optimized by a numerical optimization algorithm. This is, of course, probably much less efficient that doing some proper learning like back-propagation. However, it allows us to easily plug the ANNs into the same optimization routines as other controllers. system ( the ANNs A simple code generator. A cubic controller. A cubic controller is a function where all value of the state vector enter the computation plainly, squared, and raised to the third power. The powers of their combinations do not exceed the third power, e.g., the controller is a linear combination of A, B, A², AB, B², A³, A²B, B²A, and B³ if the state has values A and B. The controller represents the multipliers for these coefficients. A linear controller. In a linear controller, all values of the state vector enter only as-is, i.e., it is a linear combination of A and B if the state is composed of the two values A and B. We then optimize the weights of these coefficients. Poor man’s Artificial Neural Networks with minimized input. ANNs that include the state as input variable together with an additional variable, say z. The controller output is then the value z* for which the ANN takes on the smallest value (under the current state). In other words, the ANN is supposed to model the system’s objective function. The idea is similar to the golden ratio Create poor man’s ANNs for modeling the objective function. Based on the dimensionality of the state space, we generate a set of ANNs with different numbers of layers and neurons. The weights of the neurons can then directly be optimized by a numerical optimization algorithm. This is, of course, probably much less efficient that doing some proper learning like back-propagation. However, it allows us to easily plug the ANNs into the same optimization routines as other controllers. system ( the ANNs Partially linear controllers. Partially linear controllers are encoded as sets of linear controllers and anchor points. The anchors are coordinates in the state space. For each state, the linear controller with the closest anchor point is used. In other words, these controllers are basically choices among multiple Peak functions. Instead of synthesizing Create poor man’s PNNs fitting to a given system. Based on the dimensionality of the state space, we generate a set of PNNs with different numbers of layers and neurons. The weights of the neurons can then directly be optimized by a numerical optimization algorithm. This is, of course, probably much less efficient that doing some proper learning like back-propagation. However, it allows us to easily plug the PNNs into the same optimization routines as other controllers. system ( the PNNs A set of pre-defined controllers. In this module, we provide a set of pre-defined controllers taken from the works of NOACK, CORNEJO MACEDA, LI, and SUN of the Harbin Institute of Technology in Shenzhen, China (哈尔滨工业大学(深圳)). We ignore the parameterizations offered in the original works and instead synthesize the parameter values by ourselves. Yuxiang LI (李宇翔). Jet Mixing Enhancement using Deep Reinforcement Learning (基于深度强化学习的射流混合增强控制). MSc Thesis. Harbin Institute of Technology in Shenzhen, China (哈尔滨工业大学(深圳)). January 2023. Wei SUN (孙伟). Wake Control of 1-2-3 Fluidic Pinball using Deep Reinforcement Learning (基于深度强化学习方法的 1-2-3 流体弹球尾流控制). MSc Thesis. Harbin Institute of Technology in Shenzhen, China (哈尔滨工业大学(深圳)). January 2023. Guy Yoslan CORNEJO MACEDA, François LUSSEYRAN, and Bernd R. NOACK. xMLC: A Toolkit for Machine Learning Control, First Edition. Machine Learning Tools in Fluid Mechanics, Vol 2. Shenzhen & Paris; Universitätsbibliothek der Technischen Universität Braunschweig. 2022 https://doi.org/10.24355/dbbs.084-202208220937-0 A quadratic controller. A quadratic controller is a function where all value of the state vector enter the computation plainly and squared. The powers of their combinations do not exceed two, e.g., the controller is a linear combination of A, B, A², AB, and B² if the state has values A and B. The controller represents the multipliers for these coefficients.controller blueprints, also for the purpose of synthesizing models using a model_objective function.Submodules¶
moptipyapps.dynamic_control.controllers.ann module¶
codegen. This allows us to have networks of arbitrary shape for arbitrary input and output dimensions. Since code generation followed by on-the-fly compilation via numba is time and memory consuming, we cache all neural network-based instances of Controller.System) – the equations objectmoptipyapps.dynamic_control.controllers.codegen module¶
moptipyapps.dynamic_control.controllers.cubic module¶
moptipyapps.dynamic_control.controllers.linear module¶
moptipyapps.dynamic_control.controllers.min_ann module¶
ann, but instead of using the output of the ANNs as controller values, we use the value z* for which the output of the ANN becomes minimal as controller value.System) – the equations objectmoptipyapps.dynamic_control.controllers.partially_linear module¶
linear controllers.moptipyapps.dynamic_control.controllers.peaks module¶
ann with arctan as activation function, we here use exp(-a²) as activation function. This function does not represent a transition from -1 to 1, but a single peak around 0.System) – the equations objectmoptipyapps.dynamic_control.controllers.predefined module¶
moptipyapps.dynamic_control.controllers.quadratic module¶