L.R. Thistlethwaite, M. Zang, K. Workman, P. Rosenberg, C. Wrinkle, M. Ford, D. Harner, J.A. Steets
Illumination Works LLC,
United States
Keywords: risk management framework, automation, cybersecurity, decision support system, retrieval augmented generation, AI trust, explainable AI
Summary:
In today’s rapidly evolving technology landscape, organizations face increasing exposure to cyber threats, driving the need for more automated approaches to quickly identify and document cybersecurity controls. Traditional static cyber risk management approaches, such as the Department of Defense (DoD)’s Risk Management Framework (RMF)/Authority to Operate (ATO) processes, are among the most time-and labor-intensive aspects of deploying new mission systems. The manual selection and assessment of security controls and development of requisite documentation can take over six months to complete, delaying the fielding of critical technologies for the warfighter. Recognizing these limitations, the DoD introduced the Cybersecurity Risk Management Construct to infuse automation, continuous monitoring, and cyber survivability to strengthen mission readiness. As artificial intelligence (AI) is central to these automation efforts, a new challenge emerges: ensuring users can trust generative AI outputs. AI models can produce inaccurate or unverifiable information—known as hallucination—making them unreliable without clear evidence or traceability of their reasoning. Illumination Works (ILW) is addressing both challenges—the dual need for automation and trustworthy AI for cyber risk management—through its Odin-Risk Management Framework (Odin-RMF) Cybersecurity Decision Support System. Through a Phase I Small Business Innovative Research (SBIR) effort, ILW demonstrated the feasibility of Odin-RMF, a modular, AI-enabled solution that automates key RMF steps. Odin-RMF’s mature data ingestion and knowledge management technologies seamlessly curate, extract and integrate structured and vectorized representations of RMF inputs for downstream context retrieval. Odin-RMF applies domain-augmented large language models (LLMs), co-developed alongside in-house cybersecurity subject matter experts, to extend the power of LLMs with deep operational understanding. Odin-RMF’s domain-augmented LLM is leveraged in a Retrieval Augmented Generation (RAG) architecture to ground responses in authoritative RMF and ATO relevant data. These components are connected through an intuitive user interface that facilitates human-AI teaming by allowing cybersecurity professionals to trace each AI-generated response back to its supporting evidence, enabling collaborative refinement and maintenance of ATO packages. In Phase I, ILW evaluated Odin-RMF on a real-world computing system and achieved 96% accuracy in assessing control compliance. Beyond accuracy and time savings, Odin-RMF's distinguishes itself through its focus on trust and transparency. Odin-RMF's RAG approach enhances trustworthiness by grounding model outputs in authoritative internal data sources, thereby reducing the likelihood of LLM hallucination. To further promote transparency, ILW has implemented uncertainty quantification methods that estimate response confidence, empowering users to evaluate the reliability of AI-generated content. Ultimately, Odin-RMF will speed baseline control identification, guide software risk assessments, and streamline development of security implementation guides. Odin-RMF will improve efficiency while decreasing costs by cutting cyber-professional manual effort by at least 50%, with a minimum estimated 3-month time savings. With cybersecurity automation representing a multi-billion-dollar and globally expanding industry, Odin-RMF's transparent AI framework offers strong commercialization potential across government, banking, insurance, healthcare, and utility sectors seeking trustworthy AI-enabled automation.