Neural Networks Representation

Posted on Edited on Disqus: Word count in article: 1.3k Reading time ≈ 1 mins.

Neural Networks Representation

1.3 Vectorizing Logistic Regression

All right, I have already used the vectorized approach without any loops in the previous exercise.

1.4 One-vs-all Classification

1
2
3
4
5
6
7
initial_theta = zeros(n + 1, 1);
options = optimset('GradObj', 'on', 'MaxIter', 50);

for i = 1:num_labels
    [theta] = fmincg(@(t)(lrCostFunction(t, X, (y == i), lambda)), initial_theta, options);

all_theta(i,:) = [theta];

The key points here are:

  • The for loop is needed here to iterate from 1 to num_labels
  • all_theta should be assigned with (i,:), otherwise it will be a one-dimensional vector.

1.4.1 One-vs-all Prediction

1
2
A = sigmoid(X * all_theta');
[m, p] = max(A, [], 2);

The max function will return a two-dimensional vector. m is the value, and p is the index of the maximum value in each row.

2 Neural Networks

Finally, I touched on Neural Networks with multiple classes. 😳

The and are trained for prediction.

Prior to this, I was wondering about the prediction process with a trained model. Is it the same as the training process?

1
2
3
4
5
6
7
a0x = ones(m, 1);

A1 = [a0x, X];
A2 = [a0x, sigmoid(A1 * Theta1')];
A3 = sigmoid(A2 * Theta2');

[v, p] =  max(A3, [], 2);

Amazing, right? 🤩

First, I forgot to apply the sigmoid function to the hidden layer and output layer. The accuracy shows:

Training Set Accuracy: 69.62

And it will get the same accuracy if the output layer has no sigmoid. This also happened in the previous One-vs-all Prediction.

Therefore, I still have a question. 🧐

It’s time to continue with the training. 💪

Translated by gpt-3.5-turbo