Efficient State Representation for Multi-Agent Reinforcement Learning in Traffic Signal Control

Abstract

With the continuous advancement of urbanization, traffic congestion has emerged as a critical bottleneck limiting the efficiency of urban transportation systems. Conventional traffic signal control strategies, which rely on fixed-time schemes or rule-based adaptive methods, struggle to cope with the highly dynamic and stochastic nature of real-world traffic conditions. In recent years, reinforcement learning (RL) has gained increasing attention in the field of traffic signal control (TSC) due to its ability to autonomously optimize decision-making in dynamic environments. As the foundation of agent decision-making, the representation of environmental states plays a decisive role in control performance. However, most existing studies construct traffic states using only a limited set of representative features, such as queue lengths and signal phase information, which are insufficient to comprehensively capture the complex spatiotemporal dynamics of traffic flows, thereby constraining the learning capability of agents in complex environments. To address these limitations, this paper proposes an Efficient State Representation for Multi-Agent Reinforcement Learning (ESR-MARL) framework. The proposed method incorporates richer traffic information for fine-grained modeling and employs a channel-wise attention mechanism to independently learn and effectively fuse heterogeneous traffic features, enabling the extraction of a more comprehensive and informative traffic state representation. Extensive experiments conducted on both synthetic and real-world traffic datasets demonstrate that ESR-MARL achieves at least a 27.37% improvement in average travel time compared with state-of-the-art baseline methods, thereby validating the effectiveness and superiority of the proposed approach.