Autonomous Driving Tutorial

A Self Driving car simulation will be made in this post. You will understand how Genetic Algorithms and Neural Networks work in a practical way. The environment will be a 3D terrain with a track where the cars should learn how to drive.

INTRODUCTION

In this tutorials you will learn how to create a self driving simulator where the cars learn to drive using Darwin’s law of genetic

The project is created in Unity 3D using C# as programming language. You will learn how to design the map in Unity and the algorithms we implement. Also you will be able to program this algorithms in C#

The project can be divided in different aspects we will must focus on.

• Neural Network: we will be creating a multilayer neural network
• Genetic Algorithm with its respectives crossover and mutation functions
• Visual simulator in 3D with Unity: create your track
• Lasers: input data of the neural network
• Car movement: output actions of the network
• Camera: viewing the simulation in an intelligent way
• Controller: handles all the executions and the algorithms

UNDERSTANDING THE PROJECT

This video explains how all the components work. There are explained the Neural Network, Genetic Algorithm, Camera and Cars.
I explain the neccessary maths and algorithms for the simulator

NEURAL NETWORK

Neural Networks are computing systems that are used to relate different stimulus with different actions or possible solutions. A neural network is organized in layers and each layer will have a certain amount of neurons. This neurons will be connected to others with weights. Every neuron and weights will store a value. The weight values will be static when the network is being updated in our project. However, the neurons values will be changing during the simulator because they will depend on the input layer and the weights of the network.

In out project each car will have its own neural network and it will be updated every time. This updated algorithm is called feed-forward. The cars will have as the input layer the distances to the walls that surround the cars. With this inputs and the weights we will get some ouputs that would be the rotation and the acceleration.

We will need to put this into practice and create a class in C# called Neural Network that will contain as variables the List of float[][] of weights and the List of neurons organized in layers. It will have two contructors. One for the first generation of cars that will be random weights and the second for assigning the DNA to the weights. It will also have the feed-forward algorithm implemented. You can learn how to program the Neural Network in the next tutorial:

For a deep explanation of neural networks see Neural Networks and Feed-Forward Neural Networks

GENETIC ALGORITHM

Instead of using backpropagation for the Neural Network to learn, we will be using reinforcement learning. We will use the Genetic Algorithm applied to every cars. As the Darwin’s law says the entity (car) with the best genes will survive and it will propagate the genes to the next generation. They will evolve in a better way and survive more time.

If we want to express this matematically for out cars we should then start by creating for each car is own list of genes. There will be as much genes as the quantity of weights in the neural network of each car. When we create the array we will start by creating it in a random way in the first generation. 
We can divide the Genetic Algorithm in different sections:

• Random Initialization of DNAs
• Selection of best cars depending on the fitness function
• Crossover of the genes of the parents
• Mutation the new DNA

• INITIALIZATION

The initialization of the DNAs in a random way will be executed only in the first generation when the Neural Networks of the cars are created. We will create the DNA as a List of length the amount of weights in Neural Network. We will create genes in the interval of [-1,1]

We will create different random DNAs for each car. Then we will assign this DNA to the weights of the Neural Networks of the cars.

• SELECTION

When the simulator finished because all the cars have collided with a wall, we should choose which have the best genes. You can do this in a lot of different ways. Depending on the distance traveled, the average speed or the time driving. I choosed for this project the time of the car driving, so we will ge the genes for the first and second car that survive more time in the track. 
All this factors could be joined to obtain a better selection of parents. It can depend also on the diversity we want to achieve. 
We could create a function that depends on the factors mentioned before in order to get a specific score for each car called fitness

• CROSSOVER

With the parents selected we will swap the genes to create a new DNA with a mix of both parents prefering random fragments from one parent and the other. This random value could be modificated depending if the parents have very different fitness or very similar. As they a similar fitness score they will have more similar probability to be chosen the genes.

You can see here and example of the genes being manipulated:

• MUTATION

When we create for each new car for the new generation the mixture between both parents we will create different mutations. This is used because the parents are the best from the generation but not the best driving cars. We must create small mutations in the DNA of each new car to get better cars.

Fo more information about genetic algorithms see Genetic Algorithms

The tutorial of how to program the genetic algorithm and implement the genes in code is the next video:

VISUAL 3D SIMULATOR

Additionaly from the algorithms, we should create all the mesh from the cars, terrain, effects, track and camera.

Each car must be able to move and rotate. Each car will have its own lasers that raycast the distances to the track walls and output a value from 0 to 1.

The camera will behave as a drone that follows one car and will have a smooth movement when changing to other car. It will rotate the cars it is looking to

The terrain, track and effects can be choose by the creator. Nevertheless I have created a video explaining one simple terrain with a track and water:

LASERS

The neural networks must have some input data to work and to handle the car movement and rotation. In this case we will store the distances of different rays that raycast to the walls of the track. It will have some parameters such us the angle of view and the number of laser sensors.

CAR MOVEMENT

The outputs of the neural network will be the rotation and acceleration. We will need to update the car position and rotation depending on this values and the time passed in simulation. The car will have an Accelerated Linear Uniform Movement.

CAMERA

The camera will automatically move from one car to another. Depending on the cars that haven’t collided yet the camera will follow one of them. When a car collides and the camera is following it, it will change to another car in a fluent way. Also the camera will rotate slowly around the car.

CONTROLLER

We will finally need to join and interact with all the stuff created before. We will have a class called Controller that will be in charge of controlling every object and algorithm in the simulator.

The lasers will be the inputs of the Neural Network and will be assigned to the input layer of each car with its own lasers. Then we will need to apply the feed-forward algorithm and get the outputs of the Neural Network. These outputs will be the rotation and acceleration of each car that will be used in the physics of the car. This sequence will be updated every time for every car.

If it’s the first generation the DNA of every car will be random. In the next generations the weights will be adjusted by the genetic algorithm.

The next video will be the Car Controller that is the final video where you will be finished all the code and fix some bugs for the project to work. I will also explain how the parameters work and the Car Controller will be coded.

If you like the projects you can give a comment and if you have some questions tell it to me.

The code is available here: Github Repository