A DEVS-Based Framework for Integrating Reinforcement Learning Agents into Discrete Event Simulation Environments

Abstract

Discrete event simulation increasingly requires intelligent decision-making agents, yet no standardized formal approach exists for incorporating them within the Discrete Event System Specification (DEVS) formalism. We propose a framework providing reusable, formally specified DEVS building blocks for constructing simulation environments, formally modeled as Generalized Semi-Markov Decision Processes (GSMDP), exposing standardized interfaces for Reinforcement Learning (RL) agents. Three composable models are defined (StateObserver, EventGenerator and Agent) that encapsulate the RL interaction protocol, allowing domain-specific modeling to remain the central focus. The approach is validated on a real-world railway rescheduling study, where three fundamentally different RL algorithms are evaluated on the same DEVS environment without modifying it.