Discover and read the best of Twitter Threads about #NeurIPS2020
Most recents (18)
Favorite #NeurIPS2020 presentations and posters this year
PS: heavily biased by what I happened to catch and whom I happened to talk to
PPS: still catching up on talks so the list is rather incomplete and I'd hope to grow
PPPS: with contributions from @ml_collective members
PS: heavily biased by what I happened to catch and whom I happened to talk to
PPS: still catching up on talks so the list is rather incomplete and I'd hope to grow
PPPS: with contributions from @ml_collective members
[Talk] No. 1 has to go to -- keynote talk by @isbellHFh @mlittmancs et al simply brilliant ππ
slideslive.com/38935825/you-cβ¦
slideslive.com/38935825/you-cβ¦
[Talk] "Where is machine learning going" by @BachFrancis slideslive.com/38938273/where⦠appeared at preregister.science
Spoiler:
Spoiler:
In typical space-cowboy style, @ylecun, donning no slides, but only a whiteboard on Zoom, explains how all the various self-supervised models can be unified under an Energy Based view. #NeurIPS #SSL workshop 
Together with @FeryalMP , @synapticlee , @sarazann2 , @criticalneuro, @somnirons , @tyrell_turing & Doina Precup, we are excited to inform you that the Biological and Artificial Reinforcement Learning workshop at #NeurIPS2020 is happening tomorrow!
Our amazing lineup of invited speakers are @shakir_za , Claudia Clopath, @neuro_kim , George Konidaris, Ishita Dasgupta, @hartleylabnyu & @yael_niv!
They will be joining the panel session together with Peter Dayan moderated by @neurograce ! You can submit your questions for the panel here:
app.sli.do/event/cknrqkmj
app.sli.do/event/cknrqkmj
an awesome workshop on ML for molecules at #NeurIPS2020 neurips.cc/virtual/2020/pβ¦
dying to watch all the talks here!
dying to watch all the talks here!
For #NeurIPS2020, we spoke with @wojczarnecki about Spinning Tops, advice he wish he received as a student, and his goals for next year! #PeopleBehindThePapers
AI has been extremely successful in real world games (GO, DOTA, StarCraft) with results coming from relatively simple multi-agent algorithms. In this paper, we hypothesise that they share a common geometry - Spinning Tops. Learn more: bit.ly/3qI8RrD #NeurIPS2020
Iβve always loved biology. During my masters I decided to take a handful of neurophysiology courses - which I found to be interesting. But eventually I realised that my true strengths were in mathematical sciences. A career in ML and AI became a natural way to combine the two.
New insights about the limitations of density-based anomaly detection!
With @laurent_dinh, we show that perfect density model *cannot* guarantee anomaly detection.
πPaper: arxiv.org/abs/2012.03808
(1/8)
With @laurent_dinh, we show that perfect density model *cannot* guarantee anomaly detection.
πPaper: arxiv.org/abs/2012.03808
(1/8)
We demonstrate that the widespread intuition that an anomaly should have low density does not quite apply in general, irrespective of the dimension of the data or the inductive bias of the model.
(2/8)
(2/8)
We deconstruct this expectation through the lens of invertible reparametrizations (e.g., Cartesian to polar coordinates). While density changes with the underlying representation, the status of inlier/outlier *does not*.
(3/8)
(3/8)
I'm concerned that my students haven't been getting the proper NeurIPS experience, since #NeurIPS2020 is online. Post a GIF that you think captures the authentic NeurIPS experience. ;) I'll start.
A deeply interesting tutorial by @fchollet @MelMitchell1 @ChrSzegedy at #NeurIPS2020
"#Abstraction & #Reasoning in #AI systems: Modern Perspectives "
or "What are abstraction, generalization and analogic reasoning?"
1/
πππ
πΊneurips.cc/virtual/2020/pβ¦
"#Abstraction & #Reasoning in #AI systems: Modern Perspectives "
or "What are abstraction, generalization and analogic reasoning?"
1/
πππ
πΊneurips.cc/virtual/2020/pβ¦
@fchollet begins by laying some foundations in notation and background.
E.g., Generalization is a spectrum (and we are looking at lower bands in #ML nowadays)
2/
E.g., Generalization is a spectrum (and we are looking at lower bands in #ML nowadays)
2/
And Abstraction? It is the engine behind generalization!
And of course it comes in different flavors:
- program-centric: that is akin to high-level reasoning (inducing & merging programs)
- value-centric (interpolating examples) π essentially what #DL does and excels at!
3/
And of course it comes in different flavors:
- program-centric: that is akin to high-level reasoning (inducing & merging programs)
- value-centric (interpolating examples) π essentially what #DL does and excels at!
3/
Snippet 1 from the #NeurIPS2020 tutorial: @balajiln What do we mean by uncertainty and out-of-distribution robustness? nips.cc/virtual/2020/tβ¦
Snippet 2: On even simple benchmarks, neural networks are not only poor at generalizing to OOD but also degrade in their uncertainty estimates.
Snippet 3: There are a number of applications where uncertain & robust methods are already being used. It's at the heart of many AI & ML areas.
Tomorrow @latentjasper @balajiln and I present a #NeurIPS2020 tutorial on "Practical Uncertainty Estimation and Out-of-Distribution Robustness in Deep Learning". Whether you're new to the area or an expert, there is critically useful info! 8-10:30a PT nips.cc/virtual/2020/tβ¦
The talk is split into three sections: 1. Why Uncertainty & Robustness; 2. Foundations; and 3. Recent Advances.
Tutorials do _not_ need registration to attend!
Very excited to co-organize the #NeurIPS2020 Muslims in ML (MusiML.org) Workshop with @shakir_za @mo_norouzi and @2ayasalama!
Focus on potential for good & harm to those who religiously identify, culturally associate, or are classified by proximity, as βmuslimβ.
Focus on potential for good & harm to those who religiously identify, culturally associate, or are classified by proximity, as βmuslimβ.
Workshop will run on Tuesday, December 8, 2020 with talks by Mona Diab @NShafeiEN @dtbyler @SeeTedTalk @AnverEmon @samhaaElBeltagy @abidlabsand a policy panel with @RoyaPak @DiaKayyali !
Also, *nothing* would have been possible without the amazing project managment by @TasmieSarker
Our #NeurIPS2020 oral: NeuMiss networks: differentiable programming for supervised learning with missing values
neurips.cc/virtual/2020/pβ¦
with Marine Le Morvan @JulieJosseStat @tomamoral @ErwanScornet, visiting @Mila_Quebec
Goal: predict with various missing mechanisms
Thread 1/5
neurips.cc/virtual/2020/pβ¦
with Marine Le Morvan @JulieJosseStat @tomamoral @ErwanScornet, visiting @Mila_Quebec
Goal: predict with various missing mechanisms
Thread 1/5
The intuition: as features go missing, the best predictor must use covariances between features to compensate on the slope of observed features.
Classic approach: fitting with EM a probabilistic model.
Its limitations: requires model of missing mechanism & intractable with p
2/n
Classic approach: fitting with EM a probabilistic model.
Its limitations: requires model of missing mechanism & intractable with p
2/n
Our approach: write the optimal predictor under various assumptions, approximate with a differentiable function composition: a Neural Network.
This theory leads to introduce a new non-linearity: the multiplication by the missingness mask at each layer
3/5
This theory leads to introduce a new non-linearity: the multiplication by the missingness mask at each layer
3/5
I finally watched all the talks I wanted to, ended up importing 56 papers to my bib, and now present to you:
π My 13 favorite papers (sorted alphabetically) at #EMNLP2020! π₯
[1/15]
π My 13 favorite papers (sorted alphabetically) at #EMNLP2020! π₯
[1/15]
#EMNLP2020 recommendation:
"Attention is Not Only a Weight: Analyzing Transformers with Vector Norms"
@goro_koba, @ttk_kuribayashi, @sho_yokoi_, Kentaro Inui
Small vectors with high attention still have small impact!
aclweb.org/anthology/2020β¦
[2/15]
"Attention is Not Only a Weight: Analyzing Transformers with Vector Norms"
@goro_koba, @ttk_kuribayashi, @sho_yokoi_, Kentaro Inui
Small vectors with high attention still have small impact!
aclweb.org/anthology/2020β¦
[2/15]
#EMNLP2020 recommendation:
"BLEU might be Guilty but References are not Innocent"
@markuseful, David Grangier, @iseeaswell
Translationese references reward the wrong systems!
aclweb.org/anthology/2020β¦
[3/15]
"BLEU might be Guilty but References are not Innocent"
@markuseful, David Grangier, @iseeaswell
Translationese references reward the wrong systems!
aclweb.org/anthology/2020β¦
[3/15]
Excited to share our new #neurips2020 paper /Deep learning versus kernel learning: an empirical study of loss landscape geometry and the time evolution of the Neural Tangent Kernel/ (arxiv.org/abs/2010.15110) with @KDziugaite, Mansheej, @SKharaghani, @roydanroy, @SuryaGanguli 1/6
We Taylor-expand Deep Neural Network logits with respect to their weights at different stages of training & study how well a linearized network trains based on at which epoch it was expanded. Early expansions train poorly, but even slightly into training they do very well! 2/6
Linearized DNNs underperform compared to even low learning rate trained nonlinear networks, but only for expansions /very early/ in training. We call this the *nonlinear advantage* and show that it disappears quickly into training. 3/6
We can greatly simplify Hamiltonian and Lagrangian neural nets by working in Cartesian coordinates with explicit constraints, leading to dramatic performance improvements! Our #NeurIPS2020 paper: arxiv.org/abs/2010.13581
with @m_finzi, @KAlexanderWang. 1/5
with @m_finzi, @KAlexanderWang. 1/5
Complex dynamics can be described more simply with higher levels of abstraction. For example, a trajectory can be found by solving a differential equation. The differential equation can in turn be derived by a simpler Hamiltonian or Lagrangian, which is easier to model. 2/5
We can move further up the hierarchy of abstraction by working in Cartesian coordinates and explicitly representing constraints with Lagrange multipliers, for constrained Hamiltonian and Lagrangian neural networks (CHNNs and CLNNs) that face a much easier learning problem. 3/5
Twitter might seem like a not-so-kind place especially if you are a young student who just had your paper rejected by #NeurIPS2020. You might be seeing all your peers/professors talking about their paper acceptances. Let me shed some light on the reality of the situation [1/N]
Twitter (and generally social media) paints a biased view of a lot of situations including this one thechicagoschool.edu/insight/from-tβ¦. Looking at your twitter feed, you might be feeling that everyone else seems to have gotten their papers accepted except for you. That is so not true! [2/N]
#NeurIPS2020 has an acceptance rate of around 20% which means an overwhelming majority of the papers (80%) have been rejected. Also, a lot of the accepted papers might have already faced rejection(s) at other venues before being accepted at #NeurIPS2020. [3/N]
[Thread on reviewing for #MachineLearning confs, after receiving the reminder from @iclr_conf (#ICLR)]
Posting response to @iclr_conf's request for reviewing here in the hope (again) that we can change the reviewing structure of ML conferences to promote better science.
1/
Posting response to @iclr_conf's request for reviewing here in the hope (again) that we can change the reviewing structure of ML conferences to promote better science.
1/
Whether it is @NeurIPSConf (#neurips), @icmlconf (#icml), @iclr_conf (#ICLR), @RealAAAI (#aaai) or any other crowded ML conf, the reviewing structure that involves a fixed review window, multiple assigned papers, unlimited supplemental material, etc., promotes the following:
2/
2/
a) Reviews are handed off to inexperienced reviewers, directly or indirectly, due to the time pressure involved.
b) It leaves reviewers with little time to digest the technical details, leading them to potentially miss subtle errors or undervalue the merits of the work.
3/
b) It leaves reviewers with little time to digest the technical details, leading them to potentially miss subtle errors or undervalue the merits of the work.
3/
#NeurIPS2020 Authors! Wondering what a broader impact statement is and how to write one? We realize this may be a challenge, especially with no precedents to follow from past years. Looking on the bright side, you'll be a part of history! Here are some resources and tips. 1/5
First, the impact statement should not be long or overly speculative - we are looking for a paragraph or two, and not all papers need to include even that much. If your paper is more foundational - e.g. core ML methods or theory - then you may write that it is not applicable. 2/5
Next, the blogpost below (from Oxford Governance of AI) has advice and examples, from experts on the future and impact of ML. Their framework helps situate research with an 'impact stack'. shorturl.at/ilsvF Another valuable post from @bhecht : tiny.cc/5crnpz 3/5
