Backpropagation Neural Network

Here is a brief description of an assignment I did on Neural Networks.

Neural Network algorithm, devised in the mid 20th century is now popularly used in the field that we call machine learning. This technique tries to imitate the learning pattern of our brain, so that the machine could be made capable to produce acceptable outputs of unpredicted inputs. Such algorithms find a great use in various fields such as automobiles, internet advertisements, pattern recognitions etc.

There are various neural network algorithms, each having some different solving techniques. Such as Backpropagation Algorithm, Radial Basis function Algorithm, Kohenon Self Organising map, Linear Vector Quantization and many more.

The fundamental concept however remains same. We are given a set of inputs and sometimes the desired output of few of the inputs. We have to deploy fitness function between the two to generate the output with least possible error or error above a tolerance value. This fitness function performs various mathematical operations like multiplication, division and also uses some user defined function so that the output that we may have has error value less than the tolerance value. This is where most of the training of our neural network takes place. Some algorithms such as Backpropagation algorithm take this idea further and reroute the output data in the reverse direction.

Weights are scalars that are multiplied to the input so that we may get some desired result. The value of weights is given only once and after that, they update themselves automatically.

This mechanism can be easily related to the nervous system in our brain, from which it is derived.

Our nervous system also follows a similar pattern.

A much more interesting corollary can be drawn from the following. A child’s mind is also fresh and has no knowledge of anything similarly our machine is fresh and doesn’t have any knowledge of what sort of outputs should be there, there ranges etc. A child learns by the method of examples and is corrected by a teacher which could be his guardian whenever he commits a mistake. This process of error correction can go on till the child gets acceptably close to the answer. After that he/she is given some tests to find out whether he has actually learnt or not.

Similar method is adopted in our neural network algorithm technique.

Here we have used f(x)=x as my threshold function, thus derivative of f(x) is 1.

The choice of threshold function depends on the range of the inputs and outputs. If the value lies outside the range of the function, then we may get unreliable result.

We can set the weights of the layers as random or initialize to one or any other value other than 0.

Then we calculate the f(net) for first layer and using this output as input for next layer, we calculate the f(net) again and so on till we get the output from the last layer. This would be the experimental output. This was compared with the target output and error was calculated. If the error is below the tolerance level then we stop the iterations here and save these weights, else the weights are updated and same procedure is carried on.

The data provided to me by my project guide was of a machine output where the inputs comprised of following parameters: ‘run order’, ‘speed(rpm)’ and feed; and output was ‘surface finish value’ measured in micrometers.

There is a large variation in the input data.

’run order’ varies from 1-126,

‘speed(rpm)’ ranges between 2000-5000 and

the ‘feed’ from 1-8.

Also the output (‘surface finish value (micrometers)’)   ranges from 0.7-1.75.

MatLab could not operate on such varied data and would print NaN. Thus to solve this, I divided the speed by 1000 and run order by 100 (1000 and 100 were arbitrarily chosen).

This solved the problem and weights for a training set of 90 tuples out of 100 were computed. Remaining 10 were used to check whether the predicted data from the computed weights matched the target data.

Following is the graph comparing the target output is in green and the computed output in red.

Image

X axes denotes the serial number and Y axes the surface finish value or the output.

Target Output                         Computed Output

1.72

1.8194

1.61

1.5721

1.75

1.7256

1.59

1.7752

1.67

1.6861

1.55

1.6412

1.72

1.679

1.54

1.4742

1.66

1.5875

1.73

1.7268

1.59

1.5598

The data was then normalized. Input data sets and training output sets were divided by the highest value given for the entire 100 data sets (Assuming this to be the upper data limit of the range which the machine will have).

The weights computed in the training of 90 data sets were used on the normalized test data sets comprising of rest 10 tuples. The output matrix, since it was normalized, was multiplied by the highest output in the entire given data.

Following is the graph, comparing the target output in green and the computed output in red.

pic1

X axes denotes the serial number and Y axes the surface finish value or the output.

Target Output                         Computed Output

1.72

1.8322

1.61

1.4496

1.75

1.71

1.59

1.7941

1.67

1.6799

1.55

1.6038

1.72

1.6679

1.54

1.3573

1.66

1.5496

1.73

1.786

1.59

1.5395

Update: Found the code for the same while moving data from my old laptop. I have uploaded it on github.

Link: https://github.com/ChthonicCrusader/SemesterProject_Backpropagation

 

Thanks!!!

Advertisements

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s