MyJournals Home  

RSS FeedsEntropy, Vol. 25, Pages 899: Analysis of Deep Convolutional Neural Networks Using Tensor Kernels and Matrix-Based Entropy (Entropy)

 
 

3 june 2023 14:43:09

 
Entropy, Vol. 25, Pages 899: Analysis of Deep Convolutional Neural Networks Using Tensor Kernels and Matrix-Based Entropy (Entropy)
 


Analyzing deep neural networks (DNNs) via information plane (IP) theory has gained tremendous attention recently to gain insight into, among others, DNNs’ generalization ability. However, it is by no means obvious how to estimate the mutual information (MI) between each hidden layer and the input/desired output to construct the IP. For instance, hidden layers with many neurons require MI estimators with robustness toward the high dimensionality associated with such layers. MI estimators should also be able to handle convolutional layers while at the same time being computationally tractable to scale to large networks. Existing IP methods have not been able to study truly deep convolutional neural networks (CNNs). We propose an IP analysis using the new matrix-based Rényi’s entropy coupled with tensor kernels, leveraging the power of kernel methods to represent properties of the probability distribution independently of the dimensionality of the data. Our results shed new light on previous studies concerning small-scale DNNs using a completely new approach. We provide a comprehensive IP analysis of large-scale CNNs, investigating the different training phases and providing new insights into the training dynamics of large-scale neural networks.


 
79 viewsCategory: Informatics, Physics
 
Entropy, Vol. 25, Pages 898: A Multiple-Medical-Image Encryption Method Based on SHA-256 and DNA Encoding (Entropy)
Entropy, Vol. 25, Pages 900: Adaptive Resilient Neural Control of Uncertain Time-Delay Nonlinear CPSs with Full-State Constraints under Deception Attacks (Entropy)
 
 
blog comments powered by Disqus


MyJournals.org
The latest issues of all your favorite science journals on one page

Username:
Password:

Register | Retrieve

Search:

Physics


Copyright © 2008 - 2024 Indigonet Services B.V.. Contact: Tim Hulsen. Read here our privacy notice.
Other websites of Indigonet Services B.V.: Nieuws Vacatures News Tweets Nachrichten