Advice for Applying Machine Learning

1.2 Regularized linear regression cost function

1
J = 1/(2*m)*sum( (X*theta - y).^2 ) + lambda/(2*m)*sum(theta(2:end).^2);

1.3 Regularized linear regression gradient


1
2
r = theta; r(1) = 0;
grad = 1/m*(X'*(X*theta-y)) + lambda/m*r;

2.1 Learning curves

1
2
3
4
5
6
7
for i = 1:m
xi = X(1:i,:); yi = y(1:i);

theta = trainLinearReg(xi, yi, lambda);
error_train(i) = linearRegCostFunction(xi, yi, theta, 0);
error_val(i) = linearRegCostFunction(Xval, yval, theta, 0);
end

3 Polynomial regression

$$ h_{\theta }(x) = \theta _{0} + \theta _{1}(waterLevel) + \theta _{2}(waterLevel)^{2} + … + \theta _{p}*(waterLevel)^{p} $$

1
X_poly = X.^[1:1:p];

3.3 Selecting λ using a cross validation set

1
2
3
4
5
6
7
for i = 1:length(lambda_vec)
lambda = lambda_vec(i);

theta = trainLinearReg(X, y, lambda);
error_train(i) = linearRegCostFunction(X, y, theta, 0);
error_val(i) = linearRegCostFunction(Xval, yval, theta, 0);
end


1.2 Regularized linear regression cost function

1
J = 1/(2*m)*sum( (X*theta - y).^2 ) + lambda/(2*m)*sum(theta(2:end).^2);

1.3 Regularized linear regression gradient


1
2
r = theta; r(1) = 0;
grad = 1/m*(X'*(X*theta-y)) + lambda/m*r;

2.1 Learning curves

1
2
3
4
5
6
7
for i = 1:m
xi = X(1:i,:); yi = y(1:i);

theta = trainLinearReg(xi, yi, lambda);
error_train(i) = linearRegCostFunction(xi, yi, theta, 0);
error_val(i) = linearRegCostFunction(Xval, yval, theta, 0);
end

3 Polynomial regression

$$ h_{\theta }(x) = \theta _{0} + \theta _{1}(waterLevel) + \theta _{2}(waterLevel)^{2} + … + \theta _{p}*(waterLevel)^{p} $$

1
X_poly = X.^[1:1:p];

3.3 Selecting λ using a cross validation set

1
2
3
4
5
6
7
for i = 1:length(lambda_vec)
lambda = lambda_vec(i);

theta = trainLinearReg(X, y, lambda);
error_train(i) = linearRegCostFunction(X, y, theta, 0);
error_val(i) = linearRegCostFunction(Xval, yval, theta, 0);
end

Translated by gpt-3.5-turbo