NoProp: Training Neural Networks Without Back-Propagation or Forward-Propagation
an alternative training method to backprop that does local layer learning
Notes tagged MachineLearning
-
-
Temperature Scaling
a parameter that controls how confident Softmax predictions are
-
Scaled-Dot Product Attention
The specific self-attention formulation from the Transformer paper, distinguished by scaling scores by the square root of the attention dimension.
-
Neural Codec Language Models are Zero-Shot Text-to-Speech Synthesizers
VALL-E can generate speech in anyone's voice with only a 3-second sample of the speaker and some text
-
Self-Attention
A method of computing a token representation that includes the context of surrounding tokens.
-
Overfit First
the first step to training a neural network successfully
-
Residual Vector Quantisation
A tokeniser for audio
-
Snake Activation Function
An activation function for modelling data with periodicity (repeating patterns)
-
Leaky ReLU
An activation function that outputs a small value for negative numbers
-
