Discover and read the best of Twitter Threads about #NoisyStudent
Most recents (1)
What's the difference between recent works in semi-supervised learning? Key: consistency training (UDA, FixMatch, ReMixMatch, ICT, etc.) and self-training (#NoisyStudent), see photo below and my recent talk (bit.ly/thangluong-tal… with bonus slides on #MeenaBot!). 
Our UDA work (arxiv.org/abs/1904.12848) proposes the use of strong augmentation (RandAugment) which subsequent works (FixMatch, NoisyStudent) follow. UDA uses soft pseudo-labels whereas FixMatch uses hard ones after "weak" augmentation in consistency training.
It is also important to note that adding noise to the student, training equal-or-larger students, and iterative self-training is a novel combination that defines the success of #NoisyStudent on ImageNet (arxiv.org/abs/1911.04252, to appear in #CVPR).
