Operational Design Domain (ODD) is a structured, formal description of the circumstances under which an autonomous system is intended to operate – the scene layout, static and dynamic actors, environmental constraints, sensing coverage, and operational limits. In industrial automation, defining operational boundaries is especially important as humans, mobile robots, and other autonomous systems operate in close proximity. Standardized industrial ODD descriptions do not exist so far, as they can vary significantly across diverse applications, requiring extensive human effort.
At Fraunhofer IKS, we explore how foundation models can automate the generation of structured industrial ODDs. Using free-form prompting such as “describe the operating conditions of this scene” can produce fluent text, but this is rarely sufficient for safety-oriented engineering workflows. The answer may miss application-specific details, mix visual observations with assumptions, or produce narrative descriptions that are difficult to compare across scenes. For a safety-oriented workflow – the output must remain structured, auditable, and grounded in safety standards, application-specific safety documents or user-provided information.
We develop ODD Agent – a multimodal agentic workflow that combines the knowledge of safety experts with multi-view scene observations, user context, and application-specific safety documents into a structured ODD description. The goal is to assist safety experts, by automating the manual effort of retrieving relevant information from safety documents and organizing scene information according to a defined ODD taxonomy, while keeping the output transparent, auditable and easy to augment.
The ODD Agent Workflow
The workflow shown in Figure 2 combines a multi-agent framework with the processing of multimodal inputs, such as user-provided images, text describing the application, or domain-specific safety documents. In particular, we define different agentic roles:
- Multi-view visual perception: Perception agents inspect the available camera views and extract scene-level information such as layout, operating area, floor surface, lighting, occlusions, humans, vehicles, robot interaction, and carry-on items.
- Retrieval-augmented generation: RAG agents retrieve relevant chunks from uploaded domain-specific documents and standards corresponding to a defined ODD taxonomy, such as environmental limits, response-time requirements, calibration requirements, sensing-zone definitions, and use limits.
- User context and corrections: An update parser agent clarifies information that is not directly visible, such as operational assumptions, environmental values, task-specific notes, or corrections made after inspecting the output.
- Orchestration and synthesis: The orchestrator merges the sources into taxonomy fields defined in the ODD template, applies source-priority logic, runs visual and consistency checks, and exposes missing or uncertain fields for review.
Human-in-the-loop by design
After the perception, RAG, and synthesis steps, the user can review the current status to inspect generated fields, missing fields, source attribution, uploaded documents, and selected scene images. This gives the user control over each artifact.
If a taxonomy field is empty, the user can provide a natural-language update such as “there is no sunlight because the operating area is indoors”. The update parser maps these corrections back to structured ODD fields and marks the changed values as user-provided input. The Q/A component can then answer follow-up questions using the current ODD template, user context, retrieved evidence, and scene images when available. In this way, the ODD description evolves through iterative generation, inspection, questioning, correction, and regeneration.
Why this matters
The main advantage of the ODD Agent is not only that it queries and fills taxonomy fields automatically. It generates ODDs in an auditable structured format with proper source attribution. This structured approach allows safety experts to inspect outputs field by field, track evidence sources (visual context or retrieved documents), and easily identify missing data.
Furthermore, this structure mitigates hallucination risks by making outputs traceable and correctable, rather than opaque and fluent. In complex industrial settings, this enables seamless fusion of disparate data: visually rich elements (like human-robot proximity and occlusions) can be cross-referenced with document-retrieved information (such as environmental limits and calibration requirements). Ultimately, for safety-critical domains, the core value of AI lies in generating transparent intermediate artifacts that human experts can verify and trust.
Authors: Shirsha Bose, Poulami Sinhamahapatra, Florian Geissler from Fraunhofer IKS
All visuals on this page are copyrighted by © Fraunhofer IKS
