Learning rate functions determines how to update the learning rate of the network while learning.