Volume 63

A Load Forecasting Method Based on Large Language Model Considering Numerical-Text Separation Bing Sun1*, Junwei Yang1, Peng Li1, Hao Yu1, Haoran Ji1, Guanyu Song1, Jinli Zhao1

Abstract

To address the challenges of nonlinearity, non-stationarity, and multimodal fusion in power load forecasting under the new power system environment, this paper proposes a load forecasting method based on large language models that considers the separation of numerical and textual data. The method constructs a dual-channel architecture for numerical and textual separation processing, using a frozen large language model encoder to extract textual features and a multi-layer perceptron (MLP) to extract numerical features. After feature concatenation, the input is fed into the frozen large language model decoder, and the results are finally output through a numerical prediction head. By freezing the core weights of the large language model and introducing only a small number of trainable layers, the method retains the model’s strong semantic understanding capability while reducing computational overhead. A case study based on real load data from a large city in northern China demonstrates that the proposed NTSForecast method outperforms mainstream time series forecasting models such as Informer, Autoformer, and PatchTST in various evaluation metrics, including mean square error (MSE) and mean absolute error (MAE), validating the effectiveness and superiority of this method.

Keywords load forecasting,large language models,numerical-text separation,dual-channel architecture,multimodal fusion