MyJournals Home  

RSS FeedsEntropy, Vol. 23, Pages 1629: Gradient Regularization as Approximate Variational Inference (Entropy)

 
 

3 december 2021 21:57:04

 
Entropy, Vol. 23, Pages 1629: Gradient Regularization as Approximate Variational Inference (Entropy)
 


We developed Variational Laplace for Bayesian neural networks (BNNs), which exploits a local approximation of the curvature of the likelihood to estimate the ELBO without the need for stochastic sampling of the neural-network weights. The Variational Laplace objective is simple to evaluate, as it is the log-likelihood plus weight-decay, plus a squared-gradient regularizer. Variational Laplace gave better test performance and expected calibration errors than maximum a posteriori inference and standard sampling-based variational inference, despite using the same variational approximate posterior. Finally, we emphasize the care needed in benchmarking standard VI, as there is a risk of stopping before the variance parameters have converged. We show that early-stopping can be avoided by increasing the learning rate for the variance parameters.


 
68 viewsCategory: Informatics, Physics
 
Entropy, Vol. 23, Pages 1630: Theory of Non-Equilibrium Heat Transport in Anharmonic Multiprobe Systems at High Temperatures (Entropy)
Entropy, Vol. 23, Pages 1632: Effect of Ti on the Structure and Mechanical Properties of TixZr2.5-xTa Alloys (Entropy)
 
 
blog comments powered by Disqus


MyJournals.org
The latest issues of all your favorite science journals on one page

Username:
Password:

Register | Retrieve

Search:

Physics


Copyright © 2008 - 2022 Indigonet Services B.V.. Contact: Tim Hulsen. Read here our privacy notice.
Other websites of Indigonet Services B.V.: Nieuws Vacatures News Tweets Nachrichten