preprint · Research Square
Abstract In this paper, we present a new gradient descent optimization method that overcomes some weaknesses of existing gradient descentoptimization strategies. The proposed optimization technique is basedon the variation of simple gradients and regulates the learning rate inresponse to the need for an efficient learning adjustment mechanism.The memory and hyper-parameter requirements of our algorithm are reduced compared to other algorithms. We provide a convergence analysisof the algorithm. In order to evaluate our algorithm we perform classification tests on the MNIST data collection, IMDB movie review data andCIFAR-10 data, to explore how our proposed method performs againststate-of-the-art optimizers, such as SGD(M), Adam, Adamax, NadamRMSProp, Adagrad and AdaDelta. Our algorithm is computationallyefficient and effective when compared to existing methods.
This page summarises published work. The authoritative version sits with the publisher.
DOI: 10.21203/rs.3.rs-2772139/v1
Is something wrong with this record? Report it or request removal.
Discussion
Have you built on this work, tried to replicate it, or seen it applied in practice? Share what you know. Verified researchers and MARATTO™ domain experts can open a discussion, and any member can reply. Contributions are reviewed before they appear.
No discussion yet. Open the first thread.
New to MARATTO™? Create a free account.
