Simple Genetic Algorithm

What it is?

What it is NOT.  Let look at the word "genetic" and "algorithm".

First word "genetic", genetic is the science of understanding how gene of living things work.  Genes transmit character of parents to their offsprings.  Genes compose of DNA, DNA has double helix structure. In DNA, the four basic alleles : C,G,T,A encode the instructions to produce all kinds of protein in living body.  The real living gene is very complex.  Genes in parents can be carried to their offsprings
by various methods including : recombination and mutation.  The process of genetic in animals is very complex, many of which is still unknown to us.  Second word "algorithm" means the finite step of procedure for a computer to carry out to solve some problem.

GA is not about genetic.  The natural genetic is far more complex than what we can hope to find algorithm to imitate it.  Instead, GA has been INSPIRE by natural genetic.  GA uses randomization and genetic inspired method to search for the problem space.  GA uses basic genetic operations : reproduction, crossover and mutation to improve the solution and use "fitness" function to guide the search.

GA can be regarded as an optimization technique.  GA searches in problem space to find the optimal solution.  Optimization is a mathematical method to solve for a solution with the constraints and to minimize or maximize objective function.  For example, solve for x, y where
        x + y < 10
with objective function
        min  5x + 2y
 

Optimization problem

To solve optimization problems, the problem is usually formed in term of mathtimatical equation (differential equation), most of the techniques employ "gradient" search, i.e. searching along the gradient to the optimal solution.  Therefore, the derivative of the problem must be available. There are many techniques such as : steepest descent, greedy. Some technique employs randomness in searching, i.e. stochastic methods, like simulated annealing.  SA use the probabilistic procedure to search for the optimal solution.  The "randomness" will gradually be reduced toward the convergence of the solution.
 

GA vs traditional optimization techniques

GA doesn't use the "knowledge" about the problem, such as the second derivative.  It doesn't rely on the "nice" property to the problem to search for the optimality.