New Study Explains Optimal Settings for Adam Optimizer’s β1 and β2

Researchers find Adam’s β1 = 0.9 and β2 = 0.999 work, but the β1 = √β2 rule is only optimal for certain settings; training benefits from adjusting momentum parameters. getnews.me/new-study-explains-optim... #adamoptimizer #hyperparameters