MyJournals Home  

RSS FeedsEntropy, Vol. 24, Pages 1117: Learnability of the Boolean Innerproduct in Deep Neural Networks (Entropy)


13 august 2022 11:54:27

Entropy, Vol. 24, Pages 1117: Learnability of the Boolean Innerproduct in Deep Neural Networks (Entropy)

In this paper, we study the learnability of the Boolean innerproduct by a systematic simulation study. The family of the Boolean innerproduct function is known to be representable by neural networks of threshold neurons of depth 3 with only 2n+1 units (n the input dimension)—whereas an exact representation by a depth 2 network cannot possibly be of polynomial size. This result can be seen as a strong argument for deep neural network architectures. In our study, we found that this depth 3 architecture of the Boolean innerproduct is difficult to train, much harder than the depth 2 network, at least for the small input size scenarios n≤16. Nonetheless, the accuracy of the deep architecture increased with the dimension of the input space to 94% on average, which means that multiple restarts are needed to find the compact depth 3 architecture. Replacing the fully connected first layer by a partially connected layer (a kind of convolutional layer sparsely connected with weight sharing) can significantly improve the learning performance up to 99% accuracy in simulations. Another way to improve the learnability of the compact depth 3 representation of the innerproduct could be achieved by adding just a few additional units into the first hidden layer.

99 viewsCategory: Informatics, Physics
Entropy, Vol. 24, Pages 1114: Quantum Teleportation and Dense Coding in Multiple Bosonic Reservoirs (Entropy)
Entropy, Vol. 24, Pages 1115: Causality Analysis for COVID-19 among Countries Using Effective Transfer Entropy (Entropy)
blog comments powered by Disqus
The latest issues of all your favorite science journals on one page


Register | Retrieve



Copyright © 2008 - 2023 Indigonet Services B.V.. Contact: Tim Hulsen. Read here our privacy notice.
Other websites of Indigonet Services B.V.: Nieuws Vacatures News Tweets Nachrichten