Contents
1　Introduction to Power Market Data　1
1.1　Overview of Electricity Markets　1
1.2　Organization and Data Disclosure of Electricity Market　4
1.2.1　Transaction Data　5
1.2.2　Price Data　7
1.2.3　Supply and Demand Data　7
1.2.4　System Operation Data　8
1.2.5　Forecast Data　8
1.2.6　Confidential Data　9
1.3　Conclusions　9
References　9
PartⅠ　Load Modeling and Forecasting
2　Load Forecasting with Smart Meter Data　13
2.1　Introduction　13
2.2　Framework　14
2.3 Ensemble Learning for Probabilistic Forecasting　16
2.3.1　Quantile Regression Averaging　17
2.3.2　Factor Quantile Regression Averaging　18
2.3.3　LASSO Quantile Regression Averaging　18
2.3.4　Quantile Gradient Boosting Regression Tree　19
2.3.5　Rolling Window-Based Forecasting　20
2.4　Case Study　20
2.4.1　Experimental Setups　2
2.4.2　Evaluation Criteria　21
2.4.3　Experimental Results　22
2.5　Conclusions　24
References　24
3　Load Data Cleaning and Forecasting　27
3.1　Introduction　27
3.2　Characteristics of Load Profiles　29
3.2.1　Low-Rank Property of Load Profiles　29
3.2.2　Bad Data in Load Profiles　30
3.3　Methodology　31
3.3.1　Framework　31
3.3.2　Singular Value Thresholding (SVT)　32
3.3.3　Quantile RF Regression　34
3.3.4　Load Forecasting　35
3.4　Evaluation Criteria　35
3.4.1　Data Cleaning-Based Criteria　35
3.4.2　Load Forecasting-Based Criteria　35
3.5　Case Study　36
3.5.1　Result of Data Cleaning　36
3.5.2　Day Ahead Point Forecast　37
3.5.3　Day Ahead Probabilistic Forecast　38
3.6　Conclusions　40
References　40
4　Monthly Electricity Consumption Forecasting　43
4.1　Introduction　43
4.2　Framework　46
4.2.1　Data Collection and Treatment　46
4.2.2　SVECM Forecasting　47
4.2.3　Self-adaptive Screening　48
4.2.4　Novelty and Characteristics of SAS-SVECM　48
4.3　Data Collection and Treatment　48
4.3.1　Data Collection and Tests　49
4.3.2　Seasonal Adjustments Based on X-12-ARIMA　49
4.4　SVECM Forecasting　49
4.4.1　VECM Forecasting　49
4.4.2　Time Series Extrapolation Forecasting　52
4.5　Self-adaptive Screening　53
4.5.1　Influential EEF Identification　53
4.5.2　Influential EEF Grouping　53
4.5.3　Forecasting Performance Evaluation Considering Different EEF Groups　55
4.6　Case Study　56
4.6.1　Basic Data and Tests　56
4.6.2　Electricity Consumption Forecasting Performance Without SAS　58
4.6.3　EC Forecasting Performance with SAS　61
4.6.4　SAS-SVECM Forecasting Comparisons with Other Forecasting Methods　65
4.7　Conclusions　67
References　67
5　Probabilistic Load Forecasting　71
5.1　Introduction　71
5.2　Data and Model　73
5.2.1　Load Dataset Exploration　73
5.2.2　Linear Regression Model Considering Recency-Effects　73
5.3　Pre-Lasso Based　Feature Selection　76
5.4　Sparse Penalized　Quantile　Regression (Quantile-Lasso)　77
5.4.1　Problem Formulation　77
5.4.2　ADMM Algorithm　78
5.5　Implementation　80
5.6　Case Study　81
5.6.1　Experiment Setups　81
5.6.2　Results　82
5.7　Concluding Remarks　86
References　86
Part Ⅱ　Electricity Price Modeling and Forecasting
6　Subspace Characteristics of LMP Data　91
6.1　Introduction　91
6.2　Model and Distribution of LMP　93
6.3　Methodology　
6.3.1　Problem Formulation　96
6.3.2　Basic Framework　97
6.3.3　Principal Component Analysis　98
6.3.4　Recursive Basis Search (Bottom-Up)　98
6.3.5　Hyperplane Detection (Top-down)　100
6.3.6　Short Summary　103
6.4　Case Study　103
6.4.1　Case 1: IEEE 30-Bus System　104
6.4.2　Case 2: IEEE 118-Bus System　106
6.4.3　Case 3: Illinois 200-Bus System　106
6.4.4　Case 4: Southwest Power Pool (SPP)　107
6.4.5　Time Consumption　108
6.5　Discussion and Conclusion　110
6.5.1　Discussion on Potential Applications　110
6.5.2　Conclusion　110
References　111
7　Day-Ahead Electricity Price Forecasting　113
7.1　Introduction　113
7.2　Problem Formulation　116
7.2.1　Decomposition of LMP　116
7.2.2　Short-Term Forecast for Each Component　117
7.2.3　Summation and Stacking of Individual Forecasts　118
7.3　Methodology　119
7.3.1　Framework　119
7.3.2　Feature Engineering　121
7.3.3　Regression Model Selection and Parameter Tuning　122
7.3.4　Model Stacking with Robust Regression　123
7.3.5　Metrics　124
7.4　Case Study　124
7.4.1　Model Selection Results　125
7.4.2　Componential Results　126
7.4.3　Stacking Results (Overall Improvements)　128
7.4.4　Error Distribution Analysis　129
7.5　Conclusion　132
References　132
8　Economic Impact of Price Forecasting Error　135
8.1　Introduction　135
8.2　General Bidding Models　137
8.2.1　Deterministic Bidding Model　138
8.2.2　Stochastic Bidding Model　139
8.3　Methodology and Framework　141
8.3.1　Forecasting Error Modeling　141
8.3.2　Multiparametric Linear Programming (MPLP)Theory　141
8.3.3　Error Impact Formulation　142
8.3.4　Overall Framework　144
8.4　Case Study　145
8.4.1　Measurement of STPF Error Level　145
8.4.2　Case　1: LSE with Deman