Volume 68

Robust Sequence Policy Distillation with Dataset Aggregation for Deployable Multi-Objective HVAC Control Strategies Theodore Kabangu Nkashama, Anh Tuan Nguyen, Thi Ngoc Yen Huynh, Ahn Yonghan

Abstract

Deep reinforcement learning (DRL) has demonstrated its potential in multi-objective HVAC control, enabling simultaneous optimization of energy consumption, thermal comfort, and indoor air quality. However, DRL policies often produce high-variance, unstable control signals, limiting their deployment in real-world BMS. Policy distillation offers a solution toward stable and efficient controllers, while conventional imitation learning suffers from distribution shift, leading to performance degradation in closed-loop operation. Moreover, HVAC control systems exhibit slow, history-dependent dynamics, rendering the Markov assumption insufficient and motivating the use of sequence modeling. This paper proposes a robust sequence-based policy distillation framework with dataset aggregation for deployable HVAC control. A stability-aware multi-objective loss is introduced to enforce smooth and reliable control actions, while dataset aggregation mitigates distribution mismatch. The approach is validated across diverse weather conditions and occupancy scenarios in EnergyPlus simulation, demonstrating improved stability and consistent multi-objective performance compared to standard distillation baselines.

Keywords Deep reinforcement learning, imitation learning, policy distillation, multi-objective HVAC control, sequence modelling