Discover and read the best of Twitter Threads about #geometricdeeplearning
Most recents (4)
Q. What does Noether’s theorem tell us about the “geometry of deep learning dynamics”?
A. We derive Noether’s Learning Dynamics and show:
”SGD+momentum+BatchNorm+weight decay” = “RMSProp" due to symmetry breaking!
w/ @KuninDaniel
#NeurIPS2021 Paper: bit.ly/3pAEYdk
1/
A. We derive Noether’s Learning Dynamics and show:
”SGD+momentum+BatchNorm+weight decay” = “RMSProp" due to symmetry breaking!
w/ @KuninDaniel
#NeurIPS2021 Paper: bit.ly/3pAEYdk
1/
@KuninDaniel Geometry of data & representations has been central in the design of modern deepnets.
e.g., #GeometricDeepLearning arxiv.org/abs/2104.13478 by @mmbronstein, @joanbruna, @TacoCohen, @PetarV_93
What are the geometric design principles for “learning dynamics in parameter space”?
2/
e.g., #GeometricDeepLearning arxiv.org/abs/2104.13478 by @mmbronstein, @joanbruna, @TacoCohen, @PetarV_93
What are the geometric design principles for “learning dynamics in parameter space”?
2/
We develop Lagrangian mechanics of learning by modeling it as the motion of a particle in high-dimensional parameter space. Just like physical dynamics, we can model the trajectory of discrete learning dynamics by continuous-time differential equations.
3/
3/
Very glad to share our latest review on #geometricdeeplearning in molecular sciences, with special emphasis on drug discovery, quantum chemistry and synthesis prediction. 1/6
We put strong focus on the various existing molecular representations and their individual symmetries and advantages for different modeling applications. 2/6 
We discuss the major network architectures, such as RNNs, GNNs, 3D CNNs and Transformers, their applications on molecular systems, and the relevant equivariances they incorporate. 3/6
The recording of my #inaugural talk at @imperialcollege is now available online
This geometric view on deep learning is the convergence of many old and recent research threads and joint work with @joanbruna @PetarV_93 and @TacoCohen
This geometric view on deep learning is the convergence of many old and recent research threads and joint work with @joanbruna @PetarV_93 and @TacoCohen
Joan's 2014 paper on spectral #graphneuralnetworks was my inspiration to get into this field and write our paper with him, @ylecun @trekkinglemon and Arthur Szlam that popularized the term #geometricdeeplearning
Some of the works I mention are due to my amazing students and colleagues @jonathanmasci Davide Boscaini @EmanueleRodola @befcorreia Pablo Gainza @FreyrSverrisson @emaros96 @ffabffrasca @gbouritsas @frederickmonti @b_p_chamberlain @aittalam Kirill Veselkov Guadalupe Gonzalez
In summer 2020, I started writing a blog on #GraphML in @TDataScience
towardsdatascience.com/graph-deep-lea…
It was a new and rewarding experience, from which I learned a lot. I was surprised that such a technical topic would attract >200K views and >6K claps in less than half a year.
towardsdatascience.com/graph-deep-lea…
It was a new and rewarding experience, from which I learned a lot. I was surprised that such a technical topic would attract >200K views and >6K claps in less than half a year.
I am grateful to all the readers and colleagues whom I used as "guinea pigs" to proof-read my posts especially @frederickmonti @emaros96 @b_p_chamberlain @ffabffrasca @gbouritsas @__lucab @fhuszar and to @kmborgwardt @thomaskipf @guennemann whose comments helped improve them
The first post was based on a seminar I gave for the @IEEEsps on the successes and challenges of #GraphML and #geometricdeeplearning
towardsdatascience.com/deep-learning-…
towardsdatascience.com/deep-learning-…
