Machine Learning ≈ Looking for Function
- Speech Recognition :
f(vocal.wav) = "text" - Image Recognition :
f(animal.png) = "animal" - Playing Go :
f(Current Situation) = "Next Move"
Different types of Functions
-
Regression : The function outputs a scalar .
- Predict PM2.5 tomorrow
-
Classification : Given options (classes) , the function outputs the correct one.
- Spam filtering
- Playing Go
-
Structured Learning
- create something with structure (image/document)
Progress
-
Function with Unknown Parameters
For example , You want to predict the views on your YouTube .
Model(A Function with Unknown Parameter) : $$y = b + wx_1$$
-
Define Loss form Training Data
Loss is a function of parameters : $$L(b,w)$$
Loss indicates how good a set of value is .
On this picture , red refers to large L ,blue refers to small L .
-
Optimization
$$ w^*,b^* = arg min L $$ Gradient Descent :
$$w^1=w^0 – \eta \left. \frac{\partial L}{\partial w} \right|_{w=w^0} \eta : learning rate (typerparameters)$$
Attention : Global minima ≠ Local minima , but it does not matter!
How to make Loss smaller?
To reduce the loss, one approach is to increase the dimensionality of the input data.
We can use a new Function : $$y = b + \sum_{j=1}^{7} w_j x_j$$.It receives the data from the previous seven days.
Linear models have severe limitation : Model Bias.We need a more flexable model!
All Piecewise Linear Curves can be fitted!
How to represent the blue function(Hard Sigmoid)?
Every time we renew the Parameters is called Update .
