bryantpq / ai_classifiers Goto Github PK

Within

def train(self, X, y,
              learning_rate=1e-3, learning_rate_decay=0.95,
              reg=5e-6, num_iters=100,
              batch_size=200, verbose=False):
        """
        Trains the model with stochastic gradient descent
        """
        num_train = X.shape[0]
        iterations_per_epoch = max(num_train / batch_size, 1)


        for it in range(num_iters):
            X_batch = None
            y_batch = None

            r_indices = np.random.choice(num_train, batch_size)

1. The typical flow is for each epoch (what you call a num_iters), you iterate through the entire dataset once (you typically shuffle the indices and iterate through that instead of doing a random choice each time).

Effectively, if you have a dataset of size N, and your num_iters is of size m, you train on N x m samples.

What you're doing is effectively training on m x batch_size if that makes sense.

2. instead of calling random.choice you should generate an indexing array of length of dataset, then permute that array to pass through your dataset - it's more efficient and much less expensive. Also, you don't run into the issue of accidentally selecting the same index again and again as can happen above