SGD, Momentum, RMSProp and Adam
How do SGD, momentum, RMSProp, and Adam update, and what role do the first and second exponential estimates play in Adam?
Answer as you would in an interview
Answer aloud or write down a few key points. The text field is optional; the review opens after your attempt.