The post Image Processing and Classification appeared first on dDev Tech Tutorials - Retopall.

]]>Therefore, this is done with complex algorithms that are based on the **convolutional operation**. This operation consists on editing a feature image with some **filters **(kernels) that will be applied to get a transformed image.

The input image will be an **RGB image** but it can also have one channel (black and white image). This will vary the depth of the RGB image and also the channel. Also RGB images can be resized with a depth of 1 depending on their amount of colour.

Each filter will be a three dimensional array (width, height, depth) with some values.

These values will vary depending on the purpose. These values can be asigned in two different ways depending on their used

**Predifined Filters****Learning filters**for Machine Learning (Convolutional Neural Networks)

Now we will analyze each of them

This type of filters have been used for a long time with respect to the modern Convolutional Neural Networks in machine learning. They are very useful to reconstruct and proccess and image with different patterns.

Here I will saw some examples of filters for differents finalities:

With this filter we can apply the convolution operation to get the most important pixels that represent the edges of the image.

This is a popular effect that create cool effects in images. It is usually combined with Artificial Intelligence algorithms to blur the background and not the front object or person. Also commonly known as Portrait Mode

The blur effects also have a predifined filter:

Finally we have the sharpenss effect that is the opposite to the blur:

Convolutional Neural Networks are widely used for complex problems of image classificiation and detection. With some layers we can get output values that will be the prediction of the network. This network contain also convolutional layers. This layer have also filters that will have been initialized randomly. However with the backpropagation training method we will update these filters to obtain better filters that simplify the images and remark the most important parts of them.

The post Image Processing and Classification appeared first on dDev Tech Tutorials - Retopall.

]]>The post Optimization Algorithms in Neural Networks appeared first on dDev Tech Tutorials - Retopall.

]]>When we train the network we can now if the gradient descent is working well or not analyzing the error.

The error is very important to analyze the performance of our neural model. We have two main formulas to calculate the error.

The error will be calculated with the sum of the output neuron values in different ways:

**Mean Squared Error (MSE)**

$$E(x)=\frac{1}{2}\sum_o{(x-d)^2}$$

Where $x$ is the obtained value and $d$ the desired value.

The square pow is only used to have only positive values. We can also you the absolute valor of the difference:

$$E(x)=\frac{1}{2}\sum_o{\mid x-d\mid}$$

**Cross Entropy**

$$E(x)=\sum_o{d\log{x}}$$

Also depending on the learning we apply to the model, we will have three posibilities:

**UNDERFITTING**: the model is not accurate and is very flexible. When we train with a dataset, the error is always high. This happens when the**learning rate**of the network is very low.**OVERFITTING**: this situation comes up when we try in a very accurate way. We learn with a high learning rate almost 1 so we will decrease the full gradient in one training set**DESIRED:**the best way to learn is the middle term between the two situations explained before. We want to be flexible with the training sets to learn from everyone but also to try to decrease the error remarkably.

Also we can understand the behavior of the network with the error function during the epochs. When we are training we will have a function that decreases (**training loss**). In the test of the neural network (**valid loss**) we will have another function. If they are very similar like the example here the network will have a good performance.

The fluctuations of the error usually mean a good factor of the performance of the network. Also the error should decrease during the time passed (epochs)

To get very similar result of the two functions valid loss and training loss we can use different optimization algorithms:

**Gradient Descent Variants****Gradient Descent Optimization Algorithms**

The first type were explained in the Feed-Forward Neural Network Tutorial where the mini-batch, batch and stochastic gradient descents were analyzed.

We have a great variety of optimization algorithms. Some of them increase the speed of approach to the minimum, other have a dynamic learning rate to reach the best minimum and other have both of them.

In this animation you can watch the different optimization algorithms and which ones reach the best minimum and the epochs it take to each one.

The gradient descent value calculated in the back-propagation will be $g_t$ that will be equivalent to all of the $\frac{\partial E}{\partial w}$ where $E$ is the error. $\theta$ will be the matrix with all the weights.

This optimization algorithm helps to accelerate the Stochastic Gradient Descent in the relevant direction, reducing the oscillations in the wrong directions and stimulating the direction to the minimum.

$$v_t=\lambda v_{t-1}+\mu g_t$$

Finally to update each weights:

$$\theta =\theta – v_t$$

The momentum term $\lambda$ will be 0.9 or a similar value.

With the momentum the descent is following the slope blindly. We want to use smarter movements so when the error is approaching to the minimum the speed will decrease.

$$v_t=\lambda v_{t-1}+\mu g_t(\theta-\lambda v_{t-1})$$

The post Optimization Algorithms in Neural Networks appeared first on dDev Tech Tutorials - Retopall.

]]>The post Activation Functions Neural Networks appeared first on dDev Tech Tutorials - Retopall.

]]>There are a lot of different activation functions that could be used for different purposes.

An activation function is used to separate active and inactive neurons depending on a rule or function. Depending on their status they will modify their values or not.

We will see a wide range of activation functions:

- Identity
- Binary Step
- Logistic (Soft Step)
- Hyperbolic Tangent
- ReLU
- ELU
- SoftPlus
- SoftMax

This activation function will not produce any change to the input value. The output will be the same.

$$f(x)=x$$

The derivative of the identity is:

$$f'(x)=1$$

It’s a simple fragmented function.

$$f(x)= \left\{ \begin{array}{lcc}

0 & if & x < 0 \\

\\ 1 & if & x \geq 0

\end{array}

\right.$$

This function will be a step function but with an inferior and superior limit. This function is widely used for **Feed-Forward Neural Networks**

$$f(x)=\frac{1}{1+e^{-x}}$$

The derivative of the logistic function is:

$$f'(x)=f(x)(1-f(x))$$

It’s very similar to the logistic function but it will tend to -1 in the $-\infty$ and to 1 in $\infty$ .

$$f(x)=\frac{2}{1+e^{-2x}}-1$$

The derivative of the hyperbolic tangent is:

$$f'(x)=1-f(x)^2 $$

This is used almost every network. In the negative X axis the result will be 0 and in the right positive X axis the value will be mantained

$$f(x)= \left\{ \begin{array}{lcc}

0 & if & x < 0 \\

\\ x & if & x \geq 0

\end{array}

\right. $$

We have also one variant of this function called **PReLU**:

$$f(x)= \left\{ \begin{array}{lcc}

\alpha x & if & x < 0 \\

\\ x & if & x \geq 0

\end{array}

\right.$$

Where $\alpha$ is a parameter to change the slope of the function

Similar to ReLU but with an exponential curve in the negative X axis.

$$f(x)= \left\{ \begin{array}{lcc}

\alpha(e^x-1) & if & x < 0 \\

\\ x & if & x \geq 0

\end{array}

\right. $$

Where $\alpha$ is a parameter to edit the curve. It’s recommended a value similar to 1

In this function the Y value always increases.

$$f(x)=\log_e{(1+e^x)}$$

Finally we have the softmax function that is mainly used for the output layers specially in the Convolutional Neural Network.

It’s explained in the Convolutional Neural Network tutorial in the Output layer section.

To get more information about activation functions I recommend this tutorial: https://towardsdatascience.com/activation-functions-neural-networks-1cbd9f8d91d6

The post Activation Functions Neural Networks appeared first on dDev Tech Tutorials - Retopall.

]]>The post Self Driving Cars 3D Simulation appeared first on dDev Tech Tutorials - Retopall.

]]>Apart from the effects of the simulation, this autonomous simulation has a lot of complex algorithms and neural networks behind it.

It’s developed in C# and Unity 3D and has some keys ideas to make this work. You can also learn some of the basic ideas and create a simple Self Driving Simulation in 3D here.

This are the most important key ideas for creating the project:

- Track System
- Car Sensors
- Neural Network
- Output movements (acceleration and rotation)
- Training: back-propagation and genetic algorithm
- Car DNA and JSON importer
- Environment and camera

The track of the demo is made in Blender with bezier curves. In Unity 3D is a mesh that has a collider component.

When the track is imported from Unity, some track points are added in game. These points will be in charge of localizing each car. They will be in the entire track.

Now we will understand how a car is tracked:

The track systems is similar to a GPS localization and will help to create navigation systems as a new feature of the application.

- We take the initial trackpoint where the car has been spawned. In this case denoted with the colour purple. With the next two points we will be able to trilaterate the position and calculate the point where the three points are with the same distance (center of a circunference).
- This centers will be calculated at the start of the simulation with the track points.

```
float v1 = x1 * x1 + y1 * y1;
float v2 = x2 * x2 + y2 * y2;
float v3 = x3 * x3 + y3 * y3;
float a = x1 * (y2 - y3) - y1 * (x2 - x3) + x2 * y3 - x3 * y2; ;
float b = v1 * (y3 - y2) + v2 * (y1 - y3) + v3 * (y2 - y1);
float c = v1 * (x2 - x3) + v2 * (x3 - x1) + v3 * (x1 - x2);
float xCenter= -b/(2f*a);
float yCenter = -c / (2f * a);
```

Where x and y are the positions of the track points selected.

When all the center points have been calculated, on each update of the simulation we will calculate the intersection point of the line that joins the car and the center with the line of the current track point and the next point.

```
public Vector2 intersection(float x1, float y1, float x2, float y2, float x3, float y3, float x4, float y4)
{
float pxN = (x1 * y2 - y1 * x2) * (x3 - x4) - (x1 - x2) * (x3 * y4 - y3 * x4);
float pyN = (x1 * y2 - y1 * x2) * (y3 - y4) - (y1 - y2) * (x3 * y4 - y3 * x4);
float pD = (x1-x2)*(y3-y4)-(y1-y2)*(x3-x4);
return new Vector2(pxN / pD, pyN / pD);
}
```

With the intersection calculated, the initial and current track point, we will calculate the distance.

The distance in the image will be the red path.

Each car will have some back and front sensors that will be the equivalent of the real LiDAR sensors. Each sensor will raycast the distance to the first object that could be a track or another car.

The output of the sensor will be the max distance of it divided by the collision point distance from the car. The output will be a value from [0-1]. If the sensor haven’t detected any object it will return 1.

This are the car sensors parameters in the editor:

Each car will have a Feed-Forward Neural Network. This network will have a lot of parameters to edit such as the architecture or the initialization range.

This architecture is similar to the neural network of a car. It will have as input the sensor data. Each sensor will have one neuron value if it only detects the track. However if the simulation also detect car collisions, the input data of a sensor will be two for each. The output will be always 2 because of the acceleration and rotation of each car. The hidden layers will vary and will be able to be customized by the user.

The architecture of the hidden layers can be edited:

```
[Header("Architecture Network")]
public int[] hiddenArchitecture;
```

Also the activation function can be selected from the sigmoid to the hyperbolic tanh and RELU:

```
[Header("Activation")]
public Activation activationFunction = Activation.Sigmoid;
```

The values of the weights, bias, neurons and error will be stored in different array:

```
//Arrays
private List<float[]> neurons;
private List<float[,]> weights;
private List<float[]> bias;
private List<float[]> deltas;
private List<float[,]> gradient;
private List<float[,]>previousVariation;
```

The Neural Network class will have the feed-forward update method that will be charge of calculating the outputs with the input data and the weights values.

This is the feed-forward algorithm implemented in code:

```
for (int i = 0; i < neurons[0].Length; i++)
{
neurons[0][i] = inputs[i];
}
for (int i = 0; i < getLayers() - 1; i++)
{
float[,] weightsLayer = weights[i];
float[] layerNeurons = neurons[i];
float[] layerNextNeurons = neurons[i + 1];
for (int j = 0; j < layerNextNeurons.Length; j++)
{
float sum = 0;
for (int k = 0; k < layerNeurons.Length; k++)
{
sum += (weightsLayer[k, j] * layerNeurons[k]+bias[i][k]);
}
layerNextNeurons[j] = applyActivationFunction(sum);
}
}
```

Each car depending on the outputs will accelerate and deccelerate and also steer. Each car will be able to move in every direction and with different speeds. Also cars will have a rigid body to trigger the collisions with other objects ingame.

Depending on the values from [0-1] of the output layer we will need to manipulate them to get right and left rotations and also have gradual speeds. The simulation also helps the cars to have a speed limit.

```
//Rotation
float rotation = outputs[0] * 2 - 1;
transform.Rotate(new Vector3(0, (netConstantRotation * rotation)*Time.deltaTime, 0));
//Acceleration
if (applyAcceleration)
{
float accelerate = outputs[1] * 2 - 1;
if ((speed) >= maxSpeed)
{
if (accelerate >= 0)
{
accelerate = 0;
}
}
else if ((speed) <= minSpeed)
{
if (accelerate <= 0)
{
accelerate = 0;
}
}
acceleration += netConstantAcceleration * accelerate;
}
```

Each car will have two different modes. The autonomous mode where the movements will be predicted by the neural network or the manual mode. In this mode the car will be steered by the WASD control of the keyboard. This will help the car to learn of the movements of the car. We will be seeing this in the Supervised Learning.

This is the most important part. When the neural network is initialized it will be completely random and the weights will need to be modified to get nice movements of the car and prevent the car from colliding.

There are different of artificial intelligence learning algorithms. In this simulation the Supervised Learning and the Reinforcment Learning are implemented.

One strategy for the car to learn is to use the Evolution Law of Darwin. You can get a deep explanation of this in this post of Genetic Algorithm. The overall idea is to get better individuals with different methods. The weights in this type or learning are called DNAs. This DNA will have all the important information of the car (the neural network learnable parameters).

In this image you can see the steps of the Genetic Algorithms to create new child cars from the parents (best cars):

The best cars must be selected with the accuracy and the diversity of each car. The accuracy will be proportional to the distance travelled calculated in the track system of the car. With the accuracy we will calculate the fitness of each car that will help us to calculate the best cars.

Then a DNA some operation will be executed to create new childs that share DNA of their parents and also have little mutation changes.

Here you can change the code of the selection of the cars:

```
GameObject[] highestCars = new GameObject[selectedCount];
if (controller.cars.Count > 0)
{
for (int a = 0; a < selectedCount; a++)
{
//Get highest fitness of the cars that are not selected
GameObject carMaxFitness = null;
for (int i = 0; i < controller.cars.Count; i++)
{
if (!controller.cars[i].GetComponent<CarController>().selected)
{
if (carMaxFitness != null)
{
//If a car has better fitness swap and set selected
if (controller.cars[i].GetComponent<CarController>().getFitness() > carMaxFitness.GetComponent<CarController>().getFitness())
{
carMaxFitness.GetComponent<CarController>().selected = false;
controller.cars[i].GetComponent<CarController>().selected = true;
carMaxFitness = controller.cars[i];
}
}
else
{
controller.cars[i].GetComponent<CarController>().selected = true;
carMaxFitness = controller.cars[i];
}
}
}
highestCars[a] = carMaxFitness;
}
for (int j = 0; j < highestCars.Length; j++) {
selectedCars[j] = highestCars[j];
}
for (int i = 0; i < controller.cars.Count; i++)
{
if (!controller.cars[i].GetComponent<CarController>().selected && !controller.cars[i].activeSelf)
{
Destroy(controller.cars[i]);
controller.cars.Remove(controller.cars[i]);
}
}
for (int i = 0; i < controller.cars.Count; i++)
{
controller.cars[i].GetComponent<CarController>().selected = false;
}
}
```

This is another way to learn. In this case with gradient descent algorithms. This is a much complex learning because it will try to learn with the movements of a human drived car. Because the Gradient Descent is a multidimensional problem, this learning won’t give us the assurance that it will give always nice trainings. It will also depend on the error we made with the control of the car.

The idea of the back-propagation algorithm is to edit the weights of the neural network to decrease the general error of the output data of the network.

This error will be calculated with the movements of the human controlled car. Then we will execute the Gradient Descent Algorithm to try to decrease this error.

Here you can see the calculation of the error signal:

```
//Output Layer error signal
float[] outputs = getOutputs();
for(int i = 0; i < outputs.Length; i++)
{
float delta = -(desired[i] - outputs[i])*applyDerivativeActivationFunction(outputs[i]);
deltas[getLayers() - 1][i] = delta;
}
//Hidden layers error signal
for (int i = getLayers() - 2; i >= 0; i--)
{
for (int j = 0; j < neurons[i].Length; j++)
{
float sumDelta = 0;
for (int k = 0; k < neurons[i + 1].Length; k++)
{
sumDelta += deltas[i + 1][k] * weights[i][j,k];
}
float delta = sumDelta * applyDerivativeActivationFunction(neurons[i][j]);
deltas[i][j] = delta;
}
}
//Calculate gradient with this error signals.
calcGradients();
```

The adjustment of the weights of the neural network with this error signal is implemented here:

```
public void adjustWeights()
{
//Adjust weights and biases
for (int i = 0; i < weights.Count ; i++)
{
for (int j = 0; j < neurons[i + 1].Length; j++)
{
//bias[i][j] -= biasLearningRate * deltas[i + 1][j];
for (int k = 0; k < neurons[i].Length; k++)
{
float variation = weightLearningRate * gradient[i][k, j];
weights[i][k, j] -=variation;
}
}
}
//Reset array of previous gradients
gradient = createGradientArray();
}
```

To improve the performance of this algorithms, I have implemented some optimization algorithms: minibatch gradient descent and momentum.

To get better results with this algorithm, the initialization of the weights and how are they has a lot of importance for the movement to a local minimum of the Gradient Descent. It’s important firstly to get cars that works well in the track with the Genetic Algorithm learning and then get accurate results of the movements with the backpropagation.

The DNA contains all the weight data of the neural network. This DNA can be written to a JSON file to analyze this weights and also the accuracy of the cars and other variables. Also it will help to use a DNA to create the cars in another simulation.

In this fragment of code you can see how to import and export the DNA:

```
public void exportJson(DNA dna)
{
string s = JsonUtility.ToJson(dna);
Debug.Log(s);
using (StreamWriter streamWriter = File.CreateText(Path.Combine(Application.persistentDataPath, fileName)))
{
streamWriter.Write(s);
}
}
public DNA importJson()
{
using (StreamReader streamReader = File.OpenText(Path.Combine(Application.persistentDataPath, fileName)))
{
string jsonString = streamReader.ReadToEnd();
return JsonUtility.FromJson<DNA>(jsonString);
}
}
```

View of DNA in chart:

The scene has different game objects apart from the cars: the track and the camera as the main objects.

For the visual effects, the simulation includes glow effects, post proccessing, lighting and shaders.

The camera follows the best car. You can turn this functionality off by locking the camera.

**INTRUCTIONS FOR DOWNLOAD AND RESEARCH**

The project will be able to download soon.

**KEYS**

**Tab **– Switch from autonomous mode to manual mode

**R/T** manage lighting

**WASD **manual car movement

**Shift**– Genetic algorithm generation

**Control **– Random generation

**C **– Lock/unlock camera

**Z**– Switch car

**F** – Save DNA in a JSON file in Appdata

The post Self Driving Cars 3D Simulation appeared first on dDev Tech Tutorials - Retopall.

]]>The post Autonomous Cars appeared first on dDev Tech Tutorials - Retopall.

]]>Self Driving Cars are a main focus area of companies to get advantages on Artificial Intelligence algorithms to their cars or other means of transport.

Nvidia, Google, Tesla and most of the automobile industries are investing in this new advances and creating autonomous prototypes that can reduce the amount of accidents and improve the comfort of the passengers and the conductor.

Self driving cars also should predict the behaviors of the human driving cars because they will be in the same environment.

A self driving car will need to have sophisticated systems in different areas.

Firstly, in the hardware part, cars should have sophisticated sensors and cameras to get the different agents that surround the car. These agents can be other cars, traffic lights, road lines…

This sensors can help the car to understand the environment and represent it in a 3D Computational space to understand better how it should move.

The car includes LiDAR sensors for raycasting distances at light speed, different HD cameras to detect objects, a 360 degress camera, a radar for long distances and a GPS for getting the current position and map data.

Also they will need to have a powerful CPU where all the algorithms are executed with a high speed and accuracy to have instant responses and prevent accidents.

This is the most complex part for self driving cars. With so many data obtained by the sensors, the CPU should manage it in an optimal way and give the car some outputs like the steering and acceleration of the car.

Now we will subdivide the software in different models and technologies

Neural Networks are the main tool for Artificial Intelligence. Because the amount of data the car must handle the neural networks tend to be insane and with different architectures. For object detection and line detection a Convolutional Neural Network must be implemented. It is widely used for image classification and it can reach a high accuracy. Also a car will have a Recurrent Neural Network to make better predictions . All of this models are based on the Feed-Forward Neural Network.

The entire network will be a complex Modular Network. This type of architecture wil have more simple Neural Networks with different layers: convolutional layers, pooling layers, fully connected layers…

They will also have sophisticated optimization algorithms to get better results.

The inputs of the Neural Network are all the images, distances, positions given by the sensors and the outputs all the possible movements and behaviors the car could do.

One important part of self driving cars is the computer vision. The are libraries for this specific tasks such as OpenCV. The car should detect everything around him and must represent it in a 3D Environment where the car is more comfortable.

In conclusion a lot of work must be done to get a complete autonomous car. A lot of different situations must be handled by the car and also the car should respond to imprediptible situations. Nowadayrs, comercial cars can drive in an autonomous way when the conditions are favorable but the conductor should be careful and be prepared to take the steering wheel.

You can create your own Self Driving 3D simulation in this tutorial series with Unity 3D.

The post Autonomous Cars appeared first on dDev Tech Tutorials - Retopall.

]]>The post Universe Simulation appeared first on dDev Tech Tutorials - Retopall.

]]>The simulation of a finite Universe built in Unity 3D can be watched here:

As the Universe is infinite and the distances are insane I have made the project in a specific space where every object is randomly generated. Also to make the simulation more interesting I have implemented some technological inventions that are currently being researched but they haven’t been used yet.

The project can be subdivided in 3 main areas:

- Random Universe generation: create different systems and environments
- Orbiting: planet, satellites orbiting around a star or a planet.
- Rotation: planets and other entities must move depending on the planet position.
- Space Elevator: technological invention to lift out of the atmosphere of a planet different objects fast

Firstly the simulation must generate different stars in the space that have minimum distance to each other. This start could be of different colours depending on their age.

Secondly an specific random amount of planets will be created with different textures, positions and radius to the respective star.

Finally, satellites and the space elevator will be initialized. Satellites will orbit around each planet with different 3D orbits.

In this images you will se how do satellites orbit around planets with different trayectories and planets around stars.

Planets also rotate around their own axis that will have a desviation with respect to the XYZ axis.

Finally there is implemented the space elevator to every planet. This lift rotates in a synchronous way with the planet and goes from the surface of the Earth to an specific height.

Thanks to the centripetal force of the Earth is it possible to create this incredible invention but the tension is also very high. The unique solution is to have a material that can support this tensions such as the Carbon Nanotubes. This structure is the best candidate for the cable of the space elevator

The post Universe Simulation appeared first on dDev Tech Tutorials - Retopall.

]]>The post AIEditor – Neural Network Editor appeared first on dDev Tech Tutorials - Retopall.

]]>Currently artificial intelligence is being used in major companies such as Amazon or Google and will soon be used to extend to our personal environment. Scientists from all over the world try to understand the functioning of our prodigious brains. Artificial intelligence is not easy to understand, and teaching people to use it is not a simple task. Therefore, in this project I have proposed to develop a simulator and at the same time editor of artificial intelligence with a graphical interface that facilitates to develop, to understand and to innovate in this technological sector.

**Keywords:**

Artificial intelligence, future, algorithms, neural network, autonomous learning, programming, AI, machine learning, deep learning, education

The post AIEditor – Neural Network Editor appeared first on dDev Tech Tutorials - Retopall.

]]>The post Autonomous Driving Tutorial appeared first on dDev Tech Tutorials - Retopall.

]]>In this tutorials you will learn how to create a self driving simulator where the cars learn to drive using Darwin’s law of genetic

The project is created in Unity 3D using C# as programming language. You will learn how to design the map in Unity and the algorithms we implement. Also you will be able to program this algorithms in C#

The project can be divided in different aspects we will must focus on.

- Neural Network: we will be creating a multilayer neural network
- Genetic Algorithm with its respectives crossover and mutation functions
- Visual simulator in 3D with Unity: create your track
- Lasers: input data of the neural network
- Car movement: output actions of the network
- Camera: viewing the simulation in an intelligent way
- Controller: handles all the executions and the algorithms

This video explains how all the components work. There are explained the Neural Network, Genetic Algorithm, Camera and Cars.

I explain the neccessary maths and algorithms for the simulator

Neural Networks are computing systems that are used to relate different stimulus with different actions or possible solutions. A neural network is organized in layers and each layer will have a certain amount of neurons. This neurons will be connected to others with weights. Every neuron and weights will store a value. The weight values will be static when the network is being updated in our project. However, the neurons values will be changing during the simulator because they will depend on the input layer and the weights of the network.

In out project each car will have its own neural network and it will be updated every time. This updated algorithm is called feed-forward. The cars will have as the input layer the distances to the walls that surround the cars. With this inputs and the weights we will get some ouputs that would be the rotation and the acceleration.

We will need to put this into practice and create a class in C# called Neural Network that will contain as variables the List of float[][] of weights and the List of neurons organized in layers. It will have two contructors. One for the first generation of cars that will be random weights and the second for assigning the DNA to the weights. It will also have the feed-forward algorithm implemented. You can learn how to program the Neural Network in the next tutorial:

For a deep explanation of neural networks see Neural Networks and Feed-Forward Neural Networks

**GENETIC ALGORITHM**

Instead of using backpropagation for the Neural Network to learn, we will be using reinforcement learning. We will use the Genetic Algorithm applied to every cars. As the Darwin’s law says the entity (car) with the best genes will survive and it will propagate the genes to the next generation. They will evolve in a better way and survive more time.

If we want to express this matematically for out cars we should then start by creating for each car is own list of genes. There will be as much genes as the quantity of weights in the neural network of each car. When we create the array we will start by creating it in a random way in the first generation.

We can divide the Genetic Algorithm in different sections:

- Random Initialization of DNAs
- Selection of best cars depending on the fitness function
- Crossover of the genes of the parents
- Mutation the new DNA

**INITIALIZATION**

The initialization of the DNAs in a random way will be executed only in the first generation when the Neural Networks of the cars are created. We will create the DNA as a List of length the amount of weights in Neural Network. We will create genes in the interval of [-1,1]

We will create different random DNAs for each car. Then we will assign this DNA to the weights of the Neural Networks of the cars.

**SELECTION**

When the simulator finished because all the cars have collided with a wall, we should choose which have the best genes. You can do this in a lot of different ways. Depending on the distance traveled, the average speed or the time driving. I choosed for this project the time of the car driving, so we will ge the genes for the first and second car that survive more time in the track.

All this factors could be joined to obtain a better selection of parents. It can depend also on the diversity we want to achieve.

We could create a function that depends on the factors mentioned before in order to get a specific score for each car called **fitness**

**CROSSOVER**

With the parents selected we will swap the genes to create a new DNA with a mix of both parents prefering random fragments from one parent and the other. This random value could be modificated depending if the parents have very different fitness or very similar. As they a similar fitness score they will have more similar probability to be chosen the genes.

You can see here and example of the genes being manipulated:

**MUTATION**

When we create for each new car for the new generation the mixture between both parents we will create different mutations. This is used because the parents are the best from the generation but not the best driving cars. We must create small mutations in the DNA of each new car to get better cars.

Fo more information about genetic algorithms see Genetic Algorithms

The tutorial of how to program the genetic algorithm and implement the genes in code is the next video:

Additionaly from the algorithms, we should create all the mesh from the cars, terrain, effects, track and camera.

Each car must be able to move and rotate. Each car will have its own lasers that raycast the distances to the track walls and output a value from 0 to 1.

The camera will behave as a drone that follows one car and will have a smooth movement when changing to other car. It will rotate the cars it is looking to

The terrain, track and effects can be choose by the creator. Nevertheless I have created a video explaining one simple terrain with a track and water:

The neural networks must have some input data to work and to handle the car movement and rotation. In this case we will store the distances of different rays that raycast to the walls of the track. It will have some parameters such us the angle of view and the number of laser sensors.

The outputs of the neural network will be the rotation and acceleration. We will need to update the car position and rotation depending on this values and the time passed in simulation. The car will have an Accelerated Linear Uniform Movement.

The camera will automatically move from one car to another. Depending on the cars that haven’t collided yet the camera will follow one of them. When a car collides and the camera is following it, it will change to another car in a fluent way. Also the camera will rotate slowly around the car.

We will finally need to join and interact with all the stuff created before. We will have a class called Controller that will be in charge of controlling every object and algorithm in the simulator.

The lasers will be the inputs of the Neural Network and will be assigned to the input layer of each car with its own lasers. Then we will need to apply the feed-forward algorithm and get the outputs of the Neural Network. These outputs will be the rotation and acceleration of each car that will be used in the physics of the car. This sequence will be updated every time for every car.

If it’s the first generation the DNA of every car will be random. In the next generations the weights will be adjusted by the genetic algorithm.

The next video will be the Car Controller that is the final video where you will be finished all the code and fix some bugs for the project to work. I will also explain how the parameters work and the Car Controller will be coded.

If you like the projects you can give a comment and if you have some questions tell it to me.

The code is available here: Github Repository

The post Autonomous Driving Tutorial appeared first on dDev Tech Tutorials - Retopall.

]]>The post Genetic Algorithm appeared first on dDev Tech Tutorials - Retopall.

]]>The idea of this algorithm is to create the best agents in an specific environment. Each agent will perform actions depending on its neural network configuration and the environment will analyze it and score it with the fitness function. If the score is high it will give better rewards to the agents in this case, transmit some parts of its DNA to the next generation agents.

The neural networks weights are the connections of the neurons to the next layer neurons. They store a decimal value that in this case all the ways constitue the DNA of the agent or entity.

The process of learning of the genetic algorithm can be subdivided in different subprocesses:

- Initialize all the neural networks of the new agents (weights – DNA). If it’s the first generation all the weights will be random [-1,1].
- Selection: with an specific fitness function we will select the best survivors of the environment depending on their behavior.
- Crossover: on each new agent we will crossover the DNAs of the selected survivors.
- Mutation: some values will be randomly updated to get new unexpected behaviors that could improve the behavior of the agent.

All the entities in a generation will be initialized. If the generation is the first, the neural networks will have random weights between [0-1] and also the DNA will be identical to this weights.

During the simulation the entities will take actions depending on its DNA (weights) when the neural networks is updated with the feed-forward function. When we want to start a new generation we will compute the accuracy of each agent depending on the distance travelled, time survived. We also need to calculate the diversity that will depend on the DNA differences and consequently in their behaviors.

Finally to get the fitness value of an entity we will need to calculate it depending on the diversity and accuracy with a selection rate of the importance of the diversity

fitness=k · diversity + (1-k) · accuracy

When we calculate all the agent’s fitness we will be able to get the entities with the best fitness and perform the DNA manipulation.

With the winner DNAs we will swap some fragments to create a new DNA that shares parts of each parent. Each child (new agent), will have different DNAs. The length of the fragments will be random and will depend on the crossover rate. It will give important to the order of the best fitnesses or not.

In the image the first DNA is the first winner (most fitness) and the second DNA the second winner. The third DNA is the crossovered DNA of a new agent of the next generation.

To create new agents that can have different behaviors from the parents we will do little mutations to the DNA. This probability of mutation will be inversily proportional to the total fitness of the previous generation.

The first DNA is the crossovered that we will be mutating. The second is the mutated DNA.

The idea is to create new generations when the previous has finished or when we want to evolve. When the generation count increases better entities will be created and the DNA will improve and the agents will survive more time.

If you want to put in practice this concepts you can watch my tutorial series of how to create a Self Driving Car Simulator

The post Genetic Algorithm appeared first on dDev Tech Tutorials - Retopall.

]]>The post Feed-Forward Neural Network appeared first on dDev Tech Tutorials - Retopall.

]]>However, if we want to have good predicted values we should train the neural network with lot of examples. When we finished the training the neural network will be able to give accurate results as outputs.

As we saw in the last post of Neural Networks we have different ways to learn and train the neural network. Depending on the problem we want to face we will use an specific way of learning. For image classification, supervised learning is a good idea. In this tutorial we use the learning algorithm.

The architecture of a feed forward neural network consists on some layers that contain neurons. This neurons are fully connected to the neurons of the next and previous layer.

Each neurons, weight and bias store a value. Depending on the position of the layer it will have a different name:

- Input layer: all the data we want to predict is stored on this input neurons
- Hidden layers: contain the neurons that are between the output and input layer. It improves the intelligent of the neural network
- Output layer: predicted values of the network.

For having a better neural network that gives accurate results we will need to train it. Because we are using supervised learning we will use the back-propagation algorithm. We will have labelled data so we will have the input values of the data and also the outputs it must give. With this outputs we will be able to calculate the error. The reduction of the error will be done with the gradient descent algorithm where we will try to go to the best local minimum error. This could be improved with optimization algorithms.

To train the model will be divided in:

- Initialization: all the weights will be randomly filled with values in the range [-1,1]
- Training loop: iterate through all the dataset
- Feed-Forward: updating
- Calculate error
- Back-Propagation: training
- Update weights and biases

Moreover, to update the network we will use the Feed-forward function. With some inputs in the input layer, the weights and the bias we will update the neurons of the hidden layers and the output layers.

\[o_m^{l+1}=\sigma(\sum{w_{m,n}^{l} \cdot o_n^l+b^l}) \]

We will start in the first layer and we will and go forward in the neural network. $m$ is the position of the neuron we want to calculate the value, $o$ is a neuron, $w$ is a weight that connects $m$ and $n$ neuron positions. $b$ is the bias in the layer position $l$.

When we finally sum up all the values we will get the $net$ value of the neuron if we want to get the final value $out$ we will need to apply the **activation function $\sigma(net)$**

Depending on the activation we want to do we will have:

**Linear **

\[ out = net$ \]

**RELU **

\[ out=\begin{cases} net, & net>0 \\ 0, & net<0 \end{cases} \]

**Sigmoid**

\[ out=\frac{1}{1+e^{-net}} \]

**Hyperbolic tanh **

\[ out=\frac{e^{net}-e^{-net}}{e^{net}+e^{-net}} \]

With this we will be updating the entire network.

The error should be calculated each iteration of the training loop:

\[ E = \frac{1}{2}\sum{(out_m^l-d_m^l)^2} \]

Where $l$ is the last layer output predictions, $d$ is the desired value (labelled data) and $out_m$ the outputs prediction of the network that must be improved.

To train the model we will need to reduce the error.

We will calculate the partial derivative of the error with repect to each weight. Applying the chain rule we will get:

\[\dfrac{\partial E_{total}}{\partial w_{m,n}}=\dfrac{\partial E_{total}}{\partial out(a_n^l)}\dfrac{\partial out(a_n^l)}{\partial{ net(a_n^l)}}\dfrac{\partial net(a_n^l)}{\partial w_{m,n}}\]

**Output layer back-propagation**

Solving the partial derivatives where $a_n^l$ is the $out$ activation, $\sigma’$ is the derivative of the activation function

The derivative of the error is:

$$\dfrac{\partial E_{total}}{\partial out(a_n^l)} = -(x_{n}^l-a_{n}^l) $$

Doing every derivative we have:

$$\dfrac{\partial E_{total}}{\partial w_{m,n}}=-(x_{n}^l-a_{n}^l) \sigma'(a_n^l)a^{l-1}_m=\delta_n^l a_m^{l-1} $$

If the activation is the sigmoid function its derivative will be:

$$\sigma'(x)=\sigma(x)(1-\sigma(x))$$

We should then create a new important term called **error signal** that will be the error we will propagate through the neural network in the opposite direction of the feed-forward (from the outputs to the inputs). It won’t depend on the previous neuron of the weight we want to calculate the error.

$$\delta_n^l=-(x_{n}^l-a_{n}^l) \sigma'(a_n^l)$$

**Hidden layer back-propagation**

Each new derivative now will depend on the previous neurons so the calculations will change.

$$\delta_n^{l+1}=\sum_{m=0}(\delta_m^l w_{m,n}^{l+1}) $$

Finally we will have:

$$\dfrac{\partial E_{total}}{\partial w_{m,n}}=\left(\sum_{m=0}(\delta_m^l w_{m,n}^{l+1})\right)\sigma’a^{l-1}_n$$

If we want to calculate the error with respect to each bias it won’t depend on the previous neuron value because it will be one so we will have the next formula:

$$\dfrac{\partial E_{total}}{\partial b_{m}^l}=\delta_m^l$$

**General Formula**

With all of this calculations we can conclude that we can obtain the variaton of each weight to decrease the error with the following formula:

$$\dfrac{\partial E_{total}}{\partial w_{m,n}}= \delta_n^l a_m^{l-1} $$

And for the bias:

$$\dfrac{\partial E_{total}}{\partial b_{n}}= \delta_n^l $$

Where the $\delta$ will be calculated different depending on the layer.

All the calculations and ideas are shown in the next video:

When we have every derivative calculated for every weight and bias we will need to use it for decreasing the error of the neural network.

We will define $\mu$ as the learning rate of the network. This learning rate is constant but with some optimization algorithms we will be able to change it during the train.

There are different ways to update all the weights and biases.

**Stochastic Gradient Descent**

It’s the normal way of updating the network. We update each all the network weights and biases on each training. It’s the fastest way and the less memory it takes but it’s not the most efficient way to update the network.

$$w_{m,n}^l= w_{m,n}^l-\eta\dfrac{\partial E_{total}}{\partial w_{m,n}} $$

**Batch Gradient Descent**

It’s takes much memory because it updates the network when all the training sets derivatives have been calculated. It can also produce overfitting of the network. It’s not recommendable.

**Mini-Batch Gradient Descent**

It’s a middle term between the other optimization gradient descents. It update all the elements of the network in a mini-batch of normally between 64-256 training data. It doens’t consume a lot of memory and it approaches fast to the minimum.

$$w_{m,n}^l= w_{m,n}^l-\eta\dfrac{1}{s}\sum_{s=0}^{s}\dfrac{\partial E_{total}}{\partial w_{m,n}} $$

Gradient descent algorithms are explained in the next video:

If you have any question you can add a comment here.

The post Feed-Forward Neural Network appeared first on dDev Tech Tutorials - Retopall.

]]>