Volume 68

Comparative Study of Fine-Tuned Large Language Model for Short-Term Load Forecasting Under Multiple Scenarios Junwei Yang, Bing Sun, Hao Yu, Haoran Ji, Guanyu Song, Jinli Zhao, Peng Li

Abstract

Short-term load forecasting (STLF) is fundamental to
power system operation and optimal dispatch.
Traditional machine learning methods suffer significant
accuracy degradation when training and test data exhibit
distribution shift or when historical data is extremely
scarce. This paper systematically investigates the
performance of a supervised fine-tuned large language
model (GPT-4o-mini) for point-by-point STLF under three
representative scenarios with increasing difficulty:
stationary distribution, distribution shift, and few-shot
cross-distribution prediction. Experimental results on
real-world load data reveal a clear performance
crossover: while traditional methods (MLP, LSTM,
BiLSTM, RF) maintain competitive accuracy under
stationary conditions, the fine-tuned LLM achieves a
mean percentage error (MPE) of only 6.69% in the most
challenging few-shot cross-distribution scenario,
compared to over 42% for all traditional baselines. This
demonstrates that pre-trained knowledge enables LLMs
to achieve effective cross-distribution transfer with
minimal task-specific data.

Keywords Large language model,short-term load forecasting, supervised fine-tuning, few-shot learning, distribution shift, transfer learning