After last weeks blog I continued to train the Neural Network that we had created, when I realized my model was running an accuracy of 1 on the training data but only a .8 or so on the validation data. This told me that my model had begun overfitting my data so I decided to add a few more Dropout layers to my network, along with some extra Batch Normalization to help speed it up as seen below. Also to continue training after your model has finished running all of your epochs rerun the cell with "history" in it and it will continue another set after the last epoch.
model.add(Conv2D(16, kernel_size = (3, 3), activation='relu', input_shape=(image_size, image_size, 3)))
model.add(MaxPooling2D(pool_size=(2,2)))
model.add(BatchNormalization())
model.add(Conv2D(32, kernel_size=(3,3), activation='relu'))
model.add(MaxPooling2D(pool_size=(2,2)))
model.add(BatchNormalization())
model.add(Conv2D(64, kernel_size=(3,3), activation='relu'))
model.add(MaxPooling2D(pool_size=(2,2)))
model.add(BatchNormalization())
model.add(Dropout(0.2))
model.add(Conv2D(168, kernel_size=(3,3), activation='relu'))
model.add(MaxPooling2D(pool_size=(2,2)))
model.add(BatchNormalization())
model.add(Dropout(0.2))
model.add(Flatten())
model.add(Dense(512, activation='relu'))
model.add(BatchNormalization())
model.add(Dropout(0.6))
model.add(Dense(128, activation='relu'))
model.add(BatchNormalization())
model.add(Dropout(0.4))
model.add(Dense(64, activation='relu'))
model.add(BatchNormalization())
model.add(Dropout(0.2))
model.add(Dense(1, activation = 'sigmoid'))
New Hyperparameters
Now that we have a saved model I want to show you a couple of awesome Hyperparameters that can save you a lot of time in training your model. These will be the callbacks EarlyStopping and Model Checkpoint.
from keras.callbacks import EarlyStopping, ModelCheckpoint
Early Stopping allows your model to stop itself when the model begins overfitting or just stops improving. If you have restore_best_weights set to true it will return the model to the weights where it was preforming the best before the epochs ended. Then lastly you can also set a tolerance on how many epochs you want your model to try to find better weights before the model stops by setting the patience setting. So overall the Early Stopping allows you to set your model to run a large amount of epochs without having to worry about your model running forever or overfitting.
es = EarlyStopping(monitor='val_accuracy', patience=10,
restore_best_weights=True)
Then next we have Model Checkpoint, this feature is used for saving the progress of the training. So it is very similar to the model.save that we used in the last blog except that instead of just saving at the end it will save your model while it is running. Also similar to the restore_best_weights of the Early Stopping, Model Checkpoint has a feature called save_best_only that allows you to make sure it is not overwriting a model that is performing better.
checkpoint_cb = ModelCheckpoint("pneu_model2.h5",
save_best_only=True)
So between these two features you can run your model with a large amount of epochs and save yourself the fear of saving over your best model, overfitting or even worse having your computer crash while you do not have a saved model.
*With a saved model you can load it in with model = load_model('pneu_model2.h5')
epochs = 50
steps_per_epoch = train_generator.n // batch_size
validation_steps = test_generator.n // batch_size
es = EarlyStopping(monitor='val_accuracy', patience=10, restore_best_weights=True)
checkpoint_cb = ModelCheckpoint("pneu_model2.h5",
save_best_only=True)
cb_list = [es,checkpoint_cb]
history = model.fit(train_generator,
steps_per_epoch=steps_per_epoch,
epochs=epochs,
validation_data=test_generator,
validation_steps=validation_steps,
class_weight=class_weight,
callbacks=cb_list)
Evaluating our model
Testing your metrics on your training data is very simple with Keras, all you need to do is pull choose which metrics you would like to access and call them like such: loss, acc, rec = model.evaluate(test_generator).
After that is I usually enjoy creating a small confusion matrix so I can have a visual of how my model is preforming as well. I find the easiest way to do this is to create a list of all of the predictions, round them all to 0 or 1 because the neural network actually works by creating a sort of confidence interval, and then use the confusion_matrix through Sklearns metrics package.
predictions = []
for x in model.predict(test_generator):
for z in x:
predictions.append(np.round(z))
metrics.confusion_matrix(test_generator.classes,predictions)
Conclusion
With our goal of receiving a high recall score it seems as though we have done very well with a score of .98, meaning that we will have a very low chance of diagnosing someone that has pneumonia as being healthy. Though we did well on this our overall accuracy of .86 could use some improvement and I believe the best way to get this done would be to collect some more images images in general, especially some more images of healthy people. This way our neural network would have more balanced data to work with. Though I am happy with this model for this tutorial I would urge you to try changing and adding some of the layers and see what you get.
Note that for these tutorials I used the testing data as validation data, normally this is not what you should use. I did this because the point of these blogs was showing how neural networks are setup but, if you are creating your own it is best to create a validation set out of your training data with a train_test split.
Top comments (0)