Short-term wind speed forecasting by an adaptive network- based fuzzy inference system (ANFIS): an attempt towards an ensemble forecasting method

a Center for Advanced Mechatronics and Robotics Universiti Tenaga Nasional, Jalan IKRAM-UNITEN, Kajang, 43000, Selangor, Malaysia b Department of Software Engineering, Faculty of Computer Science and Information Technology, University of Malaya, Kuala Lumpur, Malaysia 1 moslem@uniten.edu.my*; 2 danial.hooshyar@gmail.com; 3 amir.ramezani.my@gmail.com, khairuls@uniten.edu.my, weria@uniten.edu.my, firas@uniten.edu.my


I. Introduction
Short-term wind speed forecasting is essential in achieving a robust integration of a wind farm into the power grid and ultimately having a stable electric power outcome.The power output of a wind farm is undoubtedly related to the wind speed, and given the volatile nature of wind, especially in the short-term consideration, it is a huge burden on the wind farms to predict their power output in shortterm and long term horizons.Some countries such as India and China have already regulated standards regarding wind-forecasting requirements of their wind farms.A forecast horizon of up to 4 hours with 15-min intervals and an error of at most 15% has been set by National Energy Administration of China as requirements for wind farms in this country [1] .Two of the major setbacks lay in the way of an accurate wind speed forecasting are firstly the volatile nature of the wind speed and its dependent on weather and topographic factors, and secondly random fluctuations in the wind speed data due to measurements errors [2] .Therefore, this unavoidable noisy behavior of a wind speed time series Accurate Wind speed forecasting has a vital role in efficient utilization of wind farms.Wind forecasting could be performed for long or short time horizons.Given the volatile nature of wind and its dependent on many geographical parameters, it is difficult for traditional methods to provide a reliable forecast of wind speed time series.In this study, an attempt is made to establish an efficient adaptive network-based fuzzy interference (ANFIS) for short-term wind speed forecasting.Using the available data sets in the literature, the ANFIS network is constructed, tested and the results are compared with that of a regular neural network, which has been forecasted the same set of dataset in previous studies.To avoid trial-and-error process for selection of the ANFIS input data, the results of autocorrelation factor (ACF) and partial auto correlation factor (PACF) on the historical wind speed data are employed.The available data set is divided into two parts.50% for training and 50% for testing and validation.The testing part of data set will be merely used for assessing the performance of the neural network which guarantees that only unseen data is used to evaluate the forecasting performance of the network.On the other hand, validation data could be used for parameter-setting of the network if required.The results indicate that ANFIS could not outperform ANN in shortterm wind speed forecasting though its results are competitive.The two methods are hybridized, though simply by weightage, and the hybrid methods shows slight improvement comparing to both ANN and ANFIS results.Therefore, the goal of future studies could be implementing ANFIS and ANNs in a more comprehensive ensemble method which could be ultimately more robust and accurate.should be taken into account before forecasting happens.Recently, the wavelet transform (WT) has been used for this task and for eliminating, or reducing to the least, of this noise.Yousefi et al [3] showed in a recent study that pretreatment of wind speed data series using wavelet transform and could improve the efficiency of a neural network in predicating the wind speed in short-term horizons) [4][5][6].For building a prediction model, the redundant highly auto-correlated wind speed data could be eliminated to improve the prediction accuracy as it has been previously shown in the literature that wind speed data has a high degree of autocorrelation.
The volatile nature of wind speed time series together with its dependence on many parameters such as weather temperature etc. makes it very difficult to develop a single robust and reliable model for wind forecasting.Therefore, it has been many attempts to develop hybrid or ensemble methods for wind speed forecasting [7][8][9][10][11].The main objective of this study was to develop a robust forecast model for the wind speed so to be able to use it in future ensemble methods.

A. Dataset
In this work the wind speed is forecasted for a 1-hour horizon using the available historical wind speed data.Therefore the output of the ANFIS or ANN is the 1-hour ahead wind speed while the input should be tuned in order of achieving the best outcome.The task of choosing the number of lags to be used to feed the network could be handled by using ACF and PACF.The performance of the proposed method is tested on a set of data from ABEI, Aberdeen, Idaho weather station available on the Internet.This station is chosen merely based on the availability of the data to the public.Hourly wind speed data of 1200 consecutive hours are used.The testing part of data set will be merely used for assessing the performance of the neural network which guarantees that only unseen data is used to evaluate the forecasting performance of the network.Fig. 1 and 2 show the wind speed time series and the histogram of wind speeds for our data set.

B. Artificial neural network
For the sake of comparison, a back propagation feed-forward neural network similar to the work of [3] is employed for constructing the artificial neural network and forecasting the 1-hour ahead wind speed data.Among supervised structures, ANN is the most commonly used one and has been successfully adopted for both short-and long-term forecasting of time series where normally a defined error function, which is typically mean square error, is minimized using a gradient descent method [12].A weight is being assigned to connect every two nodes.During the training phase, these weights have to be set in order of minimizing a predefined error function which is typically mean squared error (MSE).By setting the weights, the network is applicable to any unknown sample.ANN could be trained for any sort of problem including clustering, classification and regression.In this work, the trained ANN is used for prediction which lies in the regression category.Levenberg-Marquardt algorithm (LMA) is used for weight selection of the ANN structure.LMS has been used in many curve-fitting applications for finding the minimum though it is not a guarantee that it could find the global minimum.Other algorithms with a better success rate in finding the global minimum could be also applied, including the evolutionary algorithms like artificial bee colony [13] etc.However, as the primary concern of the study is the ANFIS performance in wind-speed forecasting and its comparison with a typical ANN, the enhancement in the neural network performance is not to be investigated.

C. Theory of ANFIS
ANFIS [14] has been employed in a variety of applications including stock market forecasting [15], project evaluation [16], time-series forecasting [17][18][19].Both artificial neural network and fuzzy logic are used in ANFIS architecture.ANFIS stands for adaptive-neuro-fuzzy inference system where a fuzzy inference system (FIS) is constructed and its membership function parameters are adjusted by a back propagation algorithm alone or in combination with a least squares type of method.This tuning ability makes the fuzzy system able to learn from the available data being used for modeling.
The structure of ANFIS is similar to that of neural network in the sense that it is a network of input and output layers which are connected through the hidden layers which are basically the input and output membership functions and other related parameters.Only for simplicity of illustration, a twoinput (assuming to be x and y), one-output (assuming to be z), five-layer ANFIS that uses nine if-then rules is shown in Fig. 3.The learning process of an ANFIS is basically the process of adjusting the membership functions.To facilitate the calculation of these parameters, a gradient vector is employed that could give us a measure of how the input and output data are mapped by the fuzzy inference system for any given data set.Basically, an optimization approach is employed to minimize an error measure by adjusting the parameters once the gradient vector is developed.This error measure is the mean squared difference between actual and desired outputs.ANFIS could either use back propagation or a combination of least squares estimation or back propagation for membership function parameter estimation.

D. Membership functions
Three lag and the current wind speed data are selected as the four inputs of the ANFIS and the sole output is the 1-hour ahead wind speed.On each of the four input, two generalized bell membership functions are provided, makes it eight altogether.And therefore, the generated FIS structure contains 16 fuzzy rules with 104 parameters.For a better ability of Considering input 1 to be the current wind speed, input 2 is lag 1, input 3 is lag 2 and input 4 to be lag 3 of the wind speed, the membership functions are set as the follows initially.Fig. 4 depicts the membership function used in this ANFIS model.

E. Network performance evaluation
The evaluation of the proposed model is evaluated using mean squared error (MSE) and root mean squared error (RMSE) calculated as (1) and (2).
Where, n is the number of observation, Ei is the wind speed prediction, and Mi is the actual measured wind speed value.The MSE gives information on short-term performance by allowing a term-by-term comparison of actual deviations between estimated and measured values.The lower the MSE, the more accurate is the estimate.

F. Selection of the input variables
The input variables are essential elements in the neural network and in the ANFIS since the selected features can affect prediction accuracy.Therefore, the selection of input variables will be carried out based on the Autocorrelation Function (ACF) and the Partial Autocorrelation Function (PACF).Both of these are statistic tools used for time series analysis.Autocorrelation is the cross-correlation of a signal with itself at different points in time.Informally, it is the similarity between observations as a function of the time lag between them.It is a mathematical tool for finding repeating patterns, such as the presence of a periodic signal obscured by noise.In time series analysis, the partial autocorrelation function (PACF) gives the partial correlation of a time series with its own lagged values, controlling for the values of the time series at all shorter lags.It contrasts with the autocorrelation function, which does not control for other lags.Fig. 5 and 6 show the results of ACF and PACF for 24 hours lag of time series.

A. Input data selection
As mentioned earlier, the input of either ANN or ANFIS is historical wind speed data and the output is the future wind speed based on the desired prediction horizon.To avoid a cumbersome trialand-error process to choose the input variables, autocorrelation (ACF) and partial autocorrelation (PACF) coefficients of time series with respect to their lags are analyzed.Based on this results, which could be seen for ACF and PACF in figures 3, 5 respectively, only three lag of the current time series are selected for predicting the 1-hour ahead value of time speed.Meaning x (t+1) is going to be forecasted using x (t), x (t-1), x (t-2) and x (t-3) considering x to be the time series and t to be the time index.Using the same number of lags for predicting a 1-hour ahead wind speed, a neural network is with 4 inputs, 5 hidden layer and 1 output layer is constructed.The same data set data set is used for training (500 sets) and testing and validation (500).As there is no recommendation for choosing the number of hidden layers, this task has been done by a trial-and-error process where it has been observed that 5 hidden layer would result in a lower mean squared error.(MSE), which is the indicator for accuracy of the predictor.The RMSE of the training data for ANN is 2.04 while it is 2.17 for test data.Also, different learning algorithms are considered for ANN to see how the model reacts.These training models include Levenberg-Marquardt, Bayesian regularization, Resilient Backpropagation and Scaled Conjugate Gradient which are provided in MATLAB neural network toolbox.However, a considerable improvement has not been noticed.Therefore the conventional Levenberg-Marquardt is considered for the training algorithm of ANN throughout this study.The structure of ANN model used in this study is shown in Fig. 7 while the training results are depicted in Fig. 8.

C. The ANFIS results
Three lags and the current wind speed data are selected as the four inputs of the ANFIS and the sole output is the 1-hour ahead wind speed.On each of the four input, two generalized bell membership functions are provided, makes it eight altogether.And therefore, the generated FIS structure contains 16 fuzzy rules with 104 parameters.Figure shows the root mean squared error (RMSE) of the ANFIS modeling on training and testing data set during the learning process of ANFIS.RMSE of ANFIS on both data sets is slightly higher than ANN results which shows the lower capability of ANFIS in predicting the 1-hour ahead wind speed from available historical data set.Fig. 9 shows the performance of ANFIS on both training and checking data sets.The future goal of this study is to establish a robust hybrid (ensemble) forecasting method using ANFIS, ANN and other available methods.In this section, a simple weighted hybrid of these two models is developed and tested on the checking data.The following weighted model is used in (3).
The value of w is changing in the range of 0 to 1 to find out the best possible outcome.Although ANN presents better prediction of wind speed time series comparing to ANFIS, the hybrid of these two methods, though it is based on a simple weightage, could outperform both ANN and ANFIS.This is in line with previous works stating the superiority of on ensemble algorithms in time series prediction.However, more efforts should be devoted in order of developing better performing models and eliminating the trial-and-error process in selecting the w parameters in (3).

IV. Conclusions
In this study an ANFIS structure is developed for short-term wind speed forecasting.It has been seen that ANN can outperform ANFIS in short-term forecasting of the wind speed time series when the same data sets are employed for training and testing purposes.This is in contradiction with some of the earlier studies on ANFIS and ANN.Autocorrelation function (ACF) and partial autocorrelation function (PACF) are implemented for facilitating the selection of input variables.These statistical tools are very helpful in guiding us for selecting an efficient number of time series lags for prediction process.The volatile nature of wind speed time series together with its dependence on many parameters such as weather temperature etc. makes it very difficult to develop a single robust and reliable model for wind forecasting.Therefore, it has been many attempts to develop hybrid or ensemble methods for wind speed forecasting.The main objective of this study was to develop a robust forecast model for the wind speed so to be able to use it in future ensemble methods.In that sense and in comparison with normal artificial neural network, it has been observed that ANFIS could greatly outperform ANN when they are both employed on a similar dataset.The ANFIS performance on forecasting wider time horizons, ranging between two to five hours, is also investigated.The results indicate that although the forecasting accuracy, reasonably, decreases with increasing the prediction horizon, still a low mean squared error is achieved in these cases that indicates the possibility of employing ANFIS for wider prediction horizons.
An attempt is made to construct a simple hybrid forecasting model consisting of ANN and ANFIS predictors by employing a weight factor to the outcomes of both ANN and ANFIS.Although ANN outperforms ANFIS in short-term wind speed forecasting, the hybrid method proves to be promising as it presents lower RMSE comparing to both ANFIS and ANN methods.Although this early attempt for constructing an ensemble method is successful, other approaches should be considered to find out whether or not more robust ensembles of ANN and ANFIS could result in better forecasting performances for short-term time-series prediction.The other area to be explored is the possibility of improving the performance of ANFIS itself, and whether using different structures of ANFIS could improve its performance.

Fig. 1 .
Fig. 1.Mean hourly wind speed values of data set

Fig. 3 .
Fig. 3.A typical ANFIS structure for two inputs and one output with nine if-then rules

Fig. 4 .
Fig. 4. The membership functions used in ANFIS model

Fig. 7 .
Fig. 7.The ANN structure used in this study

Fig. 9 .
Fig. 9.The performance of ANFIS on training and checking datasets The forecasting of ANFIS and its comparison with the actual time series for a period of 72 hours (3 days) together with the error of prediction in each hour is shown in Fig. 10.

Fig. 10 .Fig. 11 .
Fig. 10.ANFIS 1-hour ahead forecasting result and the associated errors for a 72-hour out-of-sample periodUsing the same time series and ANFIS structure (4 inputs and 1 output), an attempt is made to understand the performance of ANFIS in predicting a higher time horizons.Meaning, using three lags and the current wind speed value the wind speed value for two-hour up to 5-hour ahead is forecasted separately and the results are shown in Fig.11.The general trend is that the RMSE for both training and testing data sets increases when the forecasting time horizon is widened.Comparing ANFIS and ANN results, it has been observed that the results of ANN are slightly better than ANFIS for shortterm wind speed forecasting.