Discover and read the best of Twitter Threads about #Backpropagation
Most recents (2)
🧵1/9 A deep dive into the history of #Backpropagation: A key technique in training multilayer architectures for neural networks. This powerful method revolutionized the way we train AI systems, leading to major breakthroughs in various domains. 🤖#DataScience #DeepLearning #AI 
🧵2/9 #Backpropagation is based on a simple concept: use gradient descent to optimize multilayer networks. By applying the chain rule for derivatives, it computes gradients efficiently, leading to optimized weight configurations in each layer of the network. #DataScience #AI
🧵3/9 The shift to Rectified Linear Units (ReLU) accelerated learning in deep networks, allowing training without unsupervised pre-training. This non-linear activation function proved more effective than its smoother predecessors like tanh(z) or 1/(1+exp(−z)). #ReLU #DataScience
"Training Machines to Learn the Way Humans Do: an Alternative to #Backpropagation"
Today's SFI Seminar by Sanjukta Krishnagopal
(@UCBerkeley & @UCLA)
Starting now — follow this 🧵 for highlights:
Today's SFI Seminar by Sanjukta Krishnagopal
(@UCBerkeley & @UCLA)
Starting now — follow this 🧵 for highlights:
"When we learn something new, we look for relationships with things we know already."
"I don't just forget Calculus because I learned something else."
"We automatically know what a 'cat-dog' would look like, if it were to exist."
"We learn by training on very few examples."
"I don't just forget Calculus because I learned something else."
"We automatically know what a 'cat-dog' would look like, if it were to exist."
"We learn by training on very few examples."
1, 2) "[#MachineLearning] is fundamentally different from the way humans learn things."
3) Re: #FeedForward #NeuralNetworks
"You choose some loss function...maybe I'm learning the wrong weights. So I define some goal and then I want to learn these weights, these thetas."
3) Re: #FeedForward #NeuralNetworks
"You choose some loss function...maybe I'm learning the wrong weights. So I define some goal and then I want to learn these weights, these thetas."
