A new cross-scale load prediction model on building level based on the k-means clustering method is proposed in this paper. An office building with 26 conditioned thermal zones is the main research object. The data set is composed of 5785h cooling/heating load data by Energyplus simulation and real-world monitoring, besides, a kind of accumulative effect considered data is also included. The proposed model is based on quantifying the intra-cluster relationships. The quantification tool consists of a well-trained LSTM model and a representative load time series input which create by cluster centroid zone in one prediction cell. By combining the prediction cells under different scales, the cross-scale prediction model from the zone to building scale is built. To investigate the association between each explanatory variable and cluster belongings, ANN logistic regression model is applied. Some explanatory physical variables (e.g. the ratio of â€œnon-equilibriumâ€ temperature difference) calculated by â€œnon-equilibriumâ€ thermal insulation method are first proposed and used in logistic regression. Applying the simulation and accumulative effect considered data to the proposed model, the result shows that there is a trade-off between the ratio of the sample size of the cluster and mean cross-scale prediction accuracy, and the optimal prediction period can be obtained. In logistic regression, the result shows the maximum demand, start and end time of HVAC system, the west to the south ratio of temperature difference, and the exterior window area together determine the belonging of the cluster. At last, the proposed model is validated by real-world data and showed itâ€™s effectiveness, and the cumulative effect makes the cross-scale prediction accuracy better.