The increasing share of variable renewable energy sources in power systems poses new challenges to policy makers and network planners alike because of the sources’ variability and insufficient energy storage capability. This requires the development of new optimization models that consider the inter-temporal connection between different periods aggregating them on a more general level (e.g., days or weeks). However, aggregation models are often empirical and based on common clustering algorithms. In this paper, we carry out a numerical exploration of the relationship between the structure of the system and the hyperparameters required for these aggregation procedures. Our findings indicate that there is valuable information from the power system that can be used to improve the aggregation. This is important because, in most cases, the accuracy of the aggregation cannot be measured exactly.
Notes
S. Wogrin is an OVE member.
Publisher’s Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
1 Introduction
In recent years countries around the world have mobilized to mitigate and reduce emission of greenhouse gasses and decarbonize the electrical grid which is leading to major challenges such as the control and stability due to the VRES variability, partial unpredictability, and location-dependence. These challenges add a layer of complexity in the aim of representing the energy system into optimization models. In this paper we focus on the importance of representing time-varying input data for energy system models (ESM), where in recent years time series aggregation (TSA) has been used by reducing the overall complexity of ESMs so that they maintain their tractability.
ESMs have played a fundamental role in understanding and planning energy systems [1]. Now, with the increase in VRE generation, the complexity of such models has grown by orders of magnitude as traditional ESM have been expanded to include large system with transmission lines, storage facilities, planning and construction of new power plants as well as the operation of existing plants. The core element of ESM of this kind is the optimization of a cost-related objective function (OF) [2]. Depending on the formulation, the model is usually solved either by linear programming (LP), a mixed-integer MIP1, a rMIP2 or even other convexity-based techniques such as SOCP3 and MIQCP4 as used in LEGO – Low-carbon Expansion Generation Optimization [7].
Advertisement
The complexity of ESM has also risen because of the need of greater granularity in the time domain, which causes the models to be intractable for real-size problems. Among the techniques to reduce this complexity is time series aggregation (TSA) which aims to reduce the dimensionality of the input data while preserving the characteristics of the original time series; thorough and up-to-date reviews of TSA for power systems optimization can be found in [8] and [9].
The most widely used approach for TSA in ESM is clustering, in which a time series is divided into equally sized slices and then those are considered as individual data points over which an algorithm is then applied. Clustering is a form of unsupervised machine learning whose goal is to group or cluster similar data points together; this can be done by either increasing the similarity between the elements of a cluster (group of data points) and/or increase the dissimilarity between them. One of the key elements in a clustering algorithm is the measure of (dis)similarity where the most common the euclidean distance or \(L^{2}\) norm, however other metrics exist; an introduction to clustering can be found in [10].
One important difficulty that arises when using clustering algorithms and unsupervised machine learning models is the validation of results. Because these models work with unlabeled data, there’s no ground-truth to compare them to, and explicit information about the goal of the clustering (e.g: customer segmentation) is usually absent or impossible to have. Assessment of clustering results is a current research topic and some challenges still remain [11]; a review of those challenges is outside the scope of this work, however, it is worth mentioning how they relate to the particular application of TSA in ESM.
Most of the algorithms being used for TSA in ESM work with similarity measures that are akin to multidimensional averages, this has the drawback of filtering extreme periods which are important in ESM (e.g.: past extreme weather events) [9]. Another drawback is the lack of temporal interlinking in results; meaning that clusters do not represent the real-world temporal connection between days, weeks or whatever the size of the time slice chosen for the clustering algorithm; this leads to the misrepresentation of structural patterns in the ESM (e.g.: weekly demand cycle, operational constraints of thermal generators) [9].
Advertisement
This work focuses on alternative ways to assess the TSA performance with respect to the ESM using the k‑means algorithm [10]. We measure the clustering performance of the aggregation under different set-ups and technical characteristics of the model using a simplified model that represents some of the fundamental characteristics of real world systems like ramp rate constraints and variability of renewable sources.
In general, we observe that the number of clusters seems to have a relationship with the distribution of the hourly marginal cost of the system while the TS length is greatly affected by the number of interconnected hourly periods; in our model this interconnection come from the ramp rate constraint of the thermal generators.
The remainder of this paper is organized in the following way: In Sect. 2 we describe the experimental setup used to explore the performance of different clustering procedures compared to the true OF value; then in Sect. 3 we present the main results of the numerical experiments, then, in Sect. 4 we apply what we learned from those experiments into a stylized power system; finally the conclusions and future work are presented in Sect. 5.
2 Methods
The experimental framework is based on [12]. To analyze the performance of different clustering arrangement we developed a highly simplified power system optimization model which includes a VRE source and a traditional OCGT. Then we ran the model for each hour of a reference year for some given demand and VRE capacity factors. The results of this run were analyzed to determine the main drivers of the total system cost.
Then we ran the model using an aggregated approach, where instead of running a full hourly model, we used a set of (weighted) representative periods to approximate the full model. For this case we modified the ESM such that different ramping characteristics were explored. A k‑means clustering was run for 1000 times for each centroid and time-slice length pair. This was done to infer how the clustering characteristics affect the clustering performance for the ESM.
2.1 The Simplified Energy System Model
A simple benchmark ESM has been developed using GAMS. In order for the computational burden to be as low as possible, due to the high number of different clustering combinations, a simple single node model has been developed which represents some of the main characteristics of a ESM. This model can be run in two ways which are described below:
Mode 1: The Model is solved with the full time series length of the 8736 hours.
Mode 2: The Model is solved with the computed centroids only. That means that only a discrete number of representative periods are solved to obtain the objective function.
Fig. 1 represents the ESM: a single node model with a variable demand, thermal generation as well as a wind generation unit. The input data to the system are the demand and the wind capacity factor. Both Mode 1 as well as Mode 2 have been solved only using the mentioned input data.
Fig. 1
Circuit diagram of the analyzed ESM
×
The parameters of the ESM are defined in Table 1, there we can see that the thermal generator is enough to supply all of the demand but constrained by the maximum ramping; also the cost of non-supplied power (NSP) is set to such a value that the model resorts to this alternative only in the most extreme cases.
Table 1
ESM Parameters
Parameter
Value
Thermal Unit Variable Cost
24 €/MWh
VRE Source Variable Cost
2 €/MWh
Cost of Non-Supplied-Power
250 €/MWh
Thermal Ramp Rate
5% to 20% of Max. Capacity
Max. Demand
1000 MW
Max. VRE generation
500 MW
Max. Thermal Generation
1000 MW
The mathematical representation of the model can be found on Eqs. 1(a–f); the problem is an economic dispatch where we want to minimize the operational cost (\(C_{h}\)), which comes from the variable cost of the generators (\(\text{VC}_{g}\)) and the cost non-supplied power (CNSP). For the Mode 1 execution the cost function is represented in Eq. 1c in which the variables \(p_{g,h}\) and \(\text{nsp}_{h}\) correspond to energy produced by power plant \(g\) and non-supplied power respectively, both at time \(h\) and added over the whole year. \(D_{h}\) corresponds to the total system demand at time \(h\) while \(\text{RU}_{g}\) and \(\text{RD}_{g}\) are the maximum upward and downward ramp rates, defined only for the thermal generator, and \(\underline{P_{g}}\) and \(\overline{P_{g}}\) represent the minimum and maximum production of each generator \(g\); all of these constitute the input parameters of the model. The Mode 2 cost function is presented in Eq. 1b wean additional parameter \(W_{r}\) corresponds to the number of elements represented by centroid \(r\) (representative period) where \(r\in\{1{\ldots}n\}\) and \(n\) is the number of clusters; in this case, \(h\) is added over the specified length of the representative periods (e.g.: 6, 12, or 24 hours) hence the temporal aggregation of the model.
In this section we present the main results from the simulation; first, we describe the different marginal costs of the system and explain the situations under which those occur and then we move on to analyze the effect the ramping rate has over the marginal costs. In the second part, we analyze runs for simulations with different clustering configurations for the aggregated model and compare the OF values of the complete hourly model with aggregated one.
3.1 Marginal Cost Analysis – Mode 1
The marginal costs (MC) have been analyzed for the Mode 1 run of the ESM. Besides the variable costs of 24 and 2 €/MWh for thermal and wind respectively. The MC refers to the cost of energy if demand were to increase by one unit. In Fig. 2 we present at the MCs distribution in log-scale (left column) and the constrained periods (right column) under different ramping constraint. To calculate the log-scale we take the natural logarithm of the absolute frequency of each MC value. The number of constrained periods is calculated as the number of consecutive hours (x-axis) where either of the thermal generator’s ramping constraints, upward or downward) are binding (e.g.: \(p_{g,h}-p_{g,h-1}=\text{RU}_{g}\)).
Fig. 2
MC Distribution (log-scale) and number of interconnected hourly periods under different ramp rates
×
In Fig. 2, all MCs above 24 €/MWh result from an active ramping constraint. Since the solver optimizes each single time period under perfect foresight, it schedules generation knowing exactly how much energie will be required later. In this case, an increase in demand of one unit in a period where several previous periods were affected by the ramping constraint would result in the summation of all the required variable costs of the generating units needed to meet marginal demand; also the MC value of 250 corresponds to periods where an increase in demand by one unit would lead to Non-Supplied Power; in the 10% and 20% cases this happens less than ten times while the number increases considerably for the 5% because of the more exacting constraint.
3.2 Mode 2 Run – 10% Ramping
The aggregated ESM was run and analyzed based on predefined characteristics using a variable number of representative periods as well as different lengths of time slices (TS). In this section we analyze the results of the aggregated ESM considering a moderate ramp of the thermal unit of 10%.
To obtain a realistic representation of the ramping behavior of the system, the aggregated model was clustered by dividing the original time series into TS keeping the hourly resolution and clustering the resulting data. This ensures that the ramping behavior of the model is adequately represented, since the chronological order is kept even in the clustered time series.
In analyzing the clustering performance, we mainly study how the number of clusters, the length of TS, and the ramping rates affect the OF error. A clustering is considered to have a good performance if it results into a good approximation of the total cost of the original model while having a low computational demand. Also, we use median values because of outliers in the OF values of the aggregated model that generate variability in the results for some cases. So, the formula used to calculate the OF-Error presented in Fig. 3 and Fig. 4 is presented in Eq. 2, where OFComplete corresponds to the OF value of the complete (ground-truth) model and \(\text{OF\-Aggregated}_{ts,cl}\) to the median of the OF value for 1000 different runs of the aggregated model given a time slice length (ts) and a number of clusters (cl).
OF error vs. TS length – Medians and Avg. of Medians 10% Ramp Rate
Fig. 4
OF error as a function of the no. of clusters – 10% ramp rate
×
×
The following diagrams in Fig. 3 are intended to support the statement that, in the studied ESM with a ramp rate of 10%, a TS length of at least 12 and a number of at least six clusters is needed to achieve a well performing clustering algorithm.
Increasing the TS length from 12 to 24 time periods implies an increase of computational demand of 100% or a decrease of 50% if reduced from a length of 24 time periods to a length of 12 time periods. Still, the error only improves 0.4%, from an OF error of 3.1% (12 time periods) to 2.7% (24 time periods) when an average of all clusters is considered.
In terms of clustering performance as a function of the number of centroids, we analyzed the OF error for a variable number of clusters and a constant length of TS. In Fig. 4. In this figure, we can see that the error decreases for an increasing number of clusters regardless of the TS length. To select the correct number of clusters, the so-called elbow method is used [13] which indicates that around six clusters seem to be enough for the majority of TS analyzed.
The stability of a clustering algorithm determines the ability of an aggregated ESM to produce consistent results between different runs. If the clustering algorithm produces centroids which are very different from each other (e.g.: they are farther away in the hyper-dimensional space) the output of the aggregated ESM could result in a higher standard deviation in terms of the OF error and may even force the user to solve the aggregated ESM multiple times to get meaningful results.
To analyze the stability of the clustering results, we consider the actual output of the algorithms: the centroids for the input data of the aggregated model for wind capacity factor and system demand. The higher the similarity of the centroids, the lower the variance in the OF error of the aggregated model.
In Fig. 5 we see the calculated centroids of the 1000 iterations. As the clustering algorithm converges to slightly different centroids in both graphs, small data clouds begin to form. The smaller these data clouds are, the more stable and consistent the result.
Fig. 5
Computed centroids for TS of length 12, 6 and 7 clusters – 10% ramp rate
×
Comparing the 12 time period TS with the 24 time period TS of Figs. 5 and 6, we can see that the spanned area of data clouds for the 24 TS length is bigger than in the 12 TS length scenario. This as well leads to think that the 12 TS length delivers more constant and stable results. However, as can be seen in Fig. 5a with six clusters, the centroids span a smaller area.
Fig. 6
Computed centroids for TS of length 24, 6 and 7 clusters – 10% ramp rate
×
The same is to be observed for Fig. 7 where the inter-quartile range (IQR5) for the 12 TS length tends to be smaller compared to the 24 TS length and the same seems to hold true for TS with 24 time periods. As can be seen in Fig. 6, the centroids span a smaller surface for six clusters rather than seven clusters. Ideally, the clustering algorithm would converge to the same coordinates at each of the 1000 iterations.
Fig. 7
OF error comparison for 12 and 24 TS length – 10% ramp rate
×
Looking at the OF error for each of the iterations as a function of the number of clusters for a TS length of 12 and 24 in Fig. 7, we can see that in both cases the standard deviation of the OF is smaller for a system with six centroids rather than with seven centroids. In particular, for Fig. 7, the solution with seven centroids yields a larger IQR than the solution with six centroids. This supports the idea that for this ESM, the clustering set-up between six and seven centroids is enough to represent the system.
3.3 Mode 2 Run – 20% and 5% Ramping cases
We also ran the model using 20% and 5% ramp rates. For the 20% case we found that the aggregated model approximated the OF with an error between 0.5% and 3.6% depending on the configuration of the number of clusters and the length of the TS; we infer this is because the model is less restricted and the number of interconnected periods is lower. In this case however, the ideal number between six and seven clusters remained and it was the TS length which changed, as there was no clear improvement by increasing the length of the TS; this might be because the ramping constraint is not binding in many consecutive periods so the aggregated model doesn’t benefit much from having a higher granularity of time.
Regarding the 5% ramp rate we found that the aggregated model had errors between 7% and 15%, higher than in the other cases. We also found that the improvement vs. the number of centroids plateaued between 6 and 7 clusters but the rate was not as steep as the previous cases. The main finding for this configuration is related to the length of TS; for the 5% ramp rate was clear that a longer TS improved the performance of the model. This is to be expected as low ramp rate binds together more consecutive periods.
4 Case Study: A Stylized Power System
As a case study we used a stylized power system to evaluate the performance of different clustering setups and compare them with the complete hourly solution of the model. The components of the power system can be found in Table 2 and information about the demand used as input data can be found in Table 3. Again, we assumed that all elements are connected to the same point in the grid.
Table 2
Power System Descritpion
Technology
Cap. per Unit (MW)
Units
Var. Cost (€/MWh)
Combined Cycle Gas Turb.
500
7
72
Open Cycle Gas Turb.
400
4
67
Wind Turbine Type I Turb.
100
15
2
Wind Turbine Type II Turb.
100
8
5
Solar
100
20
0
Table 3
Input Demand
Metric
Value (MWh)
Max. Demand
5380
Min. Demand
2290
Avg. Demand
3617
Median Demand
3641
The distribution of the MC for the Mode 1 run of the stylized is presented in Fig. 8; from the sampled probability we see that the values tend to cluster around the variable cost of each technology and around two times the variable cost of the CCGT (160 €/MWh). The log-scale plot shows the pattern more clearly, though it also highlights that there are some MC values that lie between 100 and 160. According to this, we proceeded to run the ESM in Mode 2 considering a range between three and eight representative periods, clustering fifty times each time, and then compared the OF of each execution with the Mode 1 run.
Fig. 8
MC distribution – Mode 1 execution of stylized power system
×
In Table 4 the summary of the runs with respect to the OF value can be found. We can see that there is hardly any difference as all of the Mode 2 runs are within a 1% of the OF value for the Mode 1 model. However, regarding the energy production by source, the aggregation that better approximates energy production is the one that considers five representative periods.
Table 4
Summary of Mode 2 Executions
Number of RPsa
OF Mean Value (MM€)
Rel. Errorb
RP 3
1666.4
1.07%
RP 4
1666.4
1.07%
RP 5
1666.8
1.05%
RP 6
1667.1
1.03%
RP 7
1667.3
1.02%
RP 8
1667.6
1.00%
Mode 1
1684.5
aNumber of representative periods (clusters)
bWith the respect to the Mode 1 run
In all of the Mode 2 executions the model uses the available renewable energy. Regarding thermal generation, all of them underestimate the energy production from OCGT and overestimate CCGT as shown in Table 5. So from a cost-oriented perspective we can say that the aggregated models performance is roughly the same, but on the operational side using 5 representative periods is a better choice.
Table 5
Comparison of Energy Production – Mode 1 vs Mode 2
Wind
Solar
CCGT
OCGT
GWh
Error
GWh
Error
GWh
Error
GWh
Error
RP 3
6197.0
0.1%
3921.9
0.0%
12781.8
28.9%
8694.22
24.8%
RP 4
6197.0
0.1%
3921.9
0.0%
12603.2
27.1%
8872.77
23.3%
RP 5
6197.0
0.1%
3921.9
0.0%
12551.2
26.6%
8924.8
22.9%
RP 6
6197.0
0.1%
3921.9
0.0%
12778.1
28.9%
8697.88
24.8%
RP 7
6197.0
0.1%
3921.9
0.0%
12835.4
29.4%
8640.57
25.3%
RP 8
6197.0
0.1%
3921.9
0.0%
12973.9
30.8%
8502.10
26.5%
Mode 1
6188.59 GWh
3921.87 GWh
9915.37 GWh
11568.97 GWh
5 Conclusions and Future Work
Having aggregated ESM that accurately represent the behavior of complete ones is an active research problem of great importance to adequately represent VRE sources in models used for planning and policy making. So far no definite solution has been found to this problem and current approaches rely on exploring different clustering techniques and transformations in the input data. In this work we diverged from current research by exploring the relationship between the characteristics of the ESM and the hyperparameters of the aggregation procedure. In this process we gained insight about how the number of representative periods is related to the distribution of marginal costs; also, we found out the length of each representative period is linked to the interconnection between consecutive hours. In our simulations this situation was represented by the interconnected hours that arise from the ramp rate constraint of the thermal generation. These insights are important because they provide much needed advice to modellers onto choosing hyperparameter values for the data aggregation process.
Some of our findings from the basic ESM seem to hold in the case study, in which we found that an adequate number of clusters is related with the distribution of the marginal costs and that using more representative periods doesn’t ensure better performance and might even be problematic. Also we found out that considering only cost as a metric is misleading, especially if our objective is to accurately represent real-world operation. Nevertheless, despite the lowest errors regarding thermal generation corresponds to suggested number of clusters (five, with errors of 26.6% and 22.9% for CCGT and OCGT respectively), its values are high and lowering them is considered a topic of future research: how to better capture, via representative periods, the technical constraints of thermal generators.
Another aspects that we consider for future research are about the characteristics of different sets of interconnected hours and how to identify them from the input data and the characteristics of the power system being modelled, and the effect of short-term storage and network topology.
Conflict of interest
The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.
Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/.
Publisher’s Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
