Choosing the Learning Rate

Choosing the right learning rate and possibly adjusting it during training is critical for the model optimization.

The following blog gives good guidelines on how to set the learning parameters:

Estimating an Optimal Learning Rate For a Deep Neural NetworkMedium