Neural network training could be made more efficient by performing certain preprocessing steps on the network inputs and targets. Network input processing functions transforms inputs into better form for the network use. The normalization process for the raw inputs has great effect on preparing the data to be suitable for the training. Without this normalization, training the neural networks would have been very slow. There are many types of data normalization. It can be used to scale the data in the same range of values for each input feature in order to minimize bias within the neural network for one feature to another. Normalized data can also speed up training time by starting the training process for each feature within the same scale. It is especially useful for modeling application where the inputs are generally on widely different scales. The sigmoid transfer function is sensitive to input values in the range -1 to +1, thus the input data should be scaled to match this range. That was done according to equation (3).
where Iscaled is the scaled input variable, Iactual is the value of the variable to be scaled, Imin is the minimum value of the variable used in the training set, Imax is the maximum value of the variable used in the training set. It should be noted that any new input data should be normalized before simulated by the network and the corresponding predicted values should be de-normalized before use.
Training and testing stage of NNs
The ANN was developed using the popular MATLAB software package (MATLAB R2015a). All the networks were trained using Levenberg–Marquardt algorithm, this algorithm is suitable for training small- and medium-sized problems. The training parameters of Levenberg–Marquardt algorithm include: the maximum number of iterations, target performance which specifies the tolerance between the neural network prediction and actual output, the maximum run time and the minimum allowed gradient.
The most popular technique used for training the network for complex non-linear relationships is a back – propagation algorithm (Jadid and Fairbairn, 1996). It is recommended when using this algorithm in MATLAB 2015a that the input data are divided into three sets, training, validation, and testing sets. The training set is used to reduce the ANN error. During the training process, it is important to monitor the error in the validation set. The validation and the training sets error will normally decline during the initial stage of training. However, when the network begins to over-fit the data, the error on the validation set will typically begin to rise. When the validation set error increases for a specified number of epochs, the training is stopped.
There are two terms inserted in the algorithm to accelerate its convergence, learning rate, and momentum factor. The learning rate is a constant used to adjust the size of the increments of weights, while the momentum factor encourages the algorithm to move in a fixed direction. Therefore, allowing it to rapidly move over flat spots in the error surface (Moreira and Fiesler, 1995).
The testing phase is used for checking the network employs test specimens from the database, which have not used in the training process. In the present work, training data set includes 201 data entries, and the remaining data entries 86 are equally divided between the validation and testing sets. The dividing process was done randomly between the three sets and each dataset has been examined manually to ensure that it covers the range of input parameter.