Authors - Kostiantyn Hrishchenko, Oleksii Pysarchuk Abstract - Flexible Job Shop Scheduling Problems (FJSP) involve large discrete decision spaces and strict feasibility constraints, making them challenging for deep reinforcement learning methods. In this work, we study how state represen tation and feature extraction architecture influence the performance of action masked Proximal Policy Optimization (PPO) in flexible scheduling. The scheduling task is formulated as a sequential assignment of operations to machines with a fixed discrete action space, where infeasible actions are removed using a feasibility mask. The environment state is represented using three heter ogeneous feature blocks describing resource availability, operation readiness, and time-related attributes of assignment alternatives. We compare a baseline single-branch encoder with a multi-branch feature extraction architecture that processes these blocks separately before aggregation. Experiments were conducted on the Brandimarte MK benchmark suite (MK01 MK10). Under identical training conditions, the multi-branch representation achieved lower makespan on 9 out of 10 instances, with relative improvements ranging from 2.4% to 27.8% compared to the single-branch baseline. The largest reductions were observed on MK06 (−27.8%) and MK10 (−25.2%), while per formance remained comparable on MK08. Training results indicate improved stability and more consistent convergence for structured representations. These results demonstrate that structured state design and feature extraction ar chitecture are critical factors in action-masked reinforcement learning for flexible job shop scheduling.
Open Zoom