AI framework for autonomous systems

What are autonomous systems?

KI Framework
© Shutter2U - stock.adobe.com

Autonomous systems are characterized by the fact that they use sensors to detect their environment and can interact with it independently via actuators. Such a system consists of sensors for environment detection, components for aggregating, evaluating and interpreting data as well as situation assessment, action planning and actuators. Deep reinforcement learning (DRL) is a method for implementing decision-making in autonomous systems or autonomous agents. Autonomous AI agents are everywhere: they can be found in self-driving cars and drones, intelligent production facilities and logistics, intelligent home control and smart homes. Robots and prostheses are also learning their behavior. Even recommendation systems and virtual assistants for appointments use reinforcement learning techniques, and reinforcement learning with human feedback is also used to tune and adapt AI models such as ChatGPT.

© Pixabay / Fraunhofer IIS

Use-Case: Behavioral planning for driving assistance

© Fraunhofer IIS
Deep reinforcement learning (DLR) applied to autonomous driving.

As part of "AI Framework for Autonomous Systems", a reliable driving assistant is being developed to demonstrate the technology. The typical processing chain of an autonomous vehicle consists of perception, route planning, behavior planning, motion planning and actuators. We focus on behavior planning through reliable deep reinforcement learning. Algorithms are developed for navigation in complex and critical situations with safe behavior in the form of a safe policy. The training of the AI is designed in such a way that the policy is both efficient and follows strict safety conditions. Furthermore, by extracting decision trees with imitation learning, the behavior of the AI agent can be made not only interpretable but also verifiable. This demonstrates that our method offers efficient and reliable solutions even for safety-critical use cases.

Making autonomous driving safer with reliable reinforcement learning

The typical processing chain of an autonomous vehicle contains components for perception, behavioral planning, motion planning and actuators. We focus on implementing behavioral planning with reinforcement learning.

© Adobe Stock / Fraunhofer IIS

We demonstrate that effective driving strategies in critical situations can be learned through simulation. This demonstrates the strength of reinforcement learning: millions of driving hours in the simulator enable the AI agent to master even difficult driving situations with ease. We evaluate a wide range of highly varied scenarios. Driving behaviour is optimized for various factors, such as occupant safety by taking collision zones into account.

With SafeDQN, we have implemented a clear approach that learns from its own mistakes. Again, we see the architecture of the reinforcement learning agent for a constrained problem. It receives the state and the reward and must adhere to constraints. For each state, it must learn what the best action is. Two neural networks are trained, one for the benefit of the actions, it optimizes driving with regard to reaching the goal quickly. Another network is trained for the risk of the actions; it learns independently from mistakes which traffic situations are risky. Both are trained together for all possible actions and a combination factor is determined during training, which leads to the selection of the optimal action.

With the SafeVIPER approach, we have shown how to become even safer and more comprehensible for a specific situation. The task is to overtake several vehicles, for example on the highway. The car in front of you may be faster than the one in front of you and may overtake you. The task is to find the optimum time to change lanes. The SafeVIPER algorithm works in three steps. During training, a reinforcement learning agent is trained with a neural network. A restricted Markov process is considered in the safe training, which forces the agent to maintain a safe distance. During extraction, imitation learning is used to learn a decision tree with the trained neural network. Our safe extraction implements three extensions for safety. During verification, we now take advantage of the fact that the decision tree can be easily converted into propositional logic. To do this, we also formulate the dynamics of the environment and an accident using logical formulas. If we now run an equation solver on the summarized formula decision tree and environment leads to accident, and it does not find a satisfying assignment, we have proven that our agent is safe.

Both methods are shown in the following movie.

Privacy warning

With the click on the play button an external video from www.youtube.com is loaded and started. Your data is possible transferred and stored to third party. Do not start the video if you disagree. Find more about the youtube privacy statement under the following link: https://policies.google.com/privacy

Cooperation at the ADA Lovelace Center

Two teams from the ADA Lovelace Center for Analytics, Data and Applications have joined forces to show how complementary methods can form a larger whole: Fraunhofer IKS has developed a perception module that not only recognizes objects in the camera image, but also provides a security assessment for each object. The Fraunhofer IIS team has developed an agent with reinforcement learning that knows when it is safe to drive and when it is advisable to use other sources of information. You can see exactly how this works in this video.

Privacy warning

With the click on the play button an external video from www.youtube.com is loaded and started. Your data is possible transferred and stored to third party. Do not start the video if you disagree. Find more about the youtube privacy statement under the following link: https://policies.google.com/privacy