Adam Optimizer

  Optimizers 1) Optimizers are algorithms or methods used to minimize an error function or to maximize the efficiency of production. 2) These are mathematical functions which are dependent on model's learnable parameters that is weights & biases. Advantages: 1) Highly adaptive, adjust the learning rate for each network weight individually. 2) Faster Convergence  3) Easy to Implement 4) Computationally Efficient 5) Little memory requirements 6) Requires less tuning 7) Works good with sparse data. Disadvantages: 1) Overfitting 2) Lack of Control 3) Memory Requirements 4) Not suitable for non-convex Functions. 5) Focuses too much on computation speed Applications: 1) Used in ANN, RNN & CNN 2) Used in any real time applications for building the model.