How to make your model training stops when reached a threshold?

Sam Oz — Sat, 24 Oct 2020 14:42:51 +0000

Sometimes you need your model training phase stops if you reach a certain threshold. Maybe this way you can avoid overfitting or you have a pipeline and you want it deal with these kind of things automatically. Whatever your reasons are, there is a solutions: Callbacks. According to wikipedia, callback is:

"In computer programming, a callback, also known as a "call-after" function, is any executable code that is passed as an argument to other code; that other code is expected to call back (execute) the argument at a given time. This execution may be immediate as in a synchronous callback, or it might happen at a later time as in an asynchronous callback. Programming languages support callbacks in different ways, often implementing them with subroutines, lambda expressions, blocks, or function pointers."

Here is a small chunk of code that you can use in Tensorflow-Keras.

def train_mnist():
class myCallback(tf.keras.callbacks.Callback):
def on_epoch_end(self, epoch, logs={}):
if(logs.get('acc')>0.99):
print("\nReached 99% accuracy so cancelling training!")
self.model.stop_training = True

mnist = tf.keras.datasets.mnist

(x_train, y_train),(x_test, y_test) = mnist.load_data()
x_train, x_test = x_train / 255.0, x_test / 255.0
callbacks = myCallback()
model = tf.keras.models.Sequential([

  tf.keras.layers.Flatten(input_shape=(28, 28)),
  tf.keras.layers.Dense(512, activation=tf.nn.relu),
  tf.keras.layers.Dense(10, activation=tf.nn.softmax)

])

model.compile(optimizer='adam',
              loss='sparse_categorical_crossentropy',
              metrics=['accuracy'])

# model fitting
history = model.fit(x_train, y_train, epochs=10, callbacks=[callbacks])
# model fitting
return history.epoch, history.history['acc'][-1]

References:
https://en.wikipedia.org/wiki/Callback_(computer_programming)
https://www.coursera.org/learn/introduction-tensorflow/

Laying the Foundation for Deep Learning

Sam Oz — Sat, 20 Jun 2020 11:08:01 +0000

This post is for those are lost along the way just like me.

Last week I have started a study marathon and the reason to do that I've realized I am a bit rusty about the foundations of the domain. I tried to find a study partner but since I couldn't find a suitable partner I will be using this platform as a journal to keep me in line and hopefully help couple of you along the way. About me; I am working as a data analyst at PwC(when I started writing this post) so I know statistics and machine learning as a practitioner. I am not coming from CS or Math background, I have studied Economics therefore I had some math knowledge kept me going for a while. I am also writing my thesis which is on natural language processing for that reason alone I must know the mathematical background of neural nets and modern architectures used in NLP such as transformers etc. to explain in academic lingo. Other than that NLP is rapidly growing and improving field and you should be able to read recent academic papers to stay up-to-date on state of the art methodologies. Additionally I should mention I know intermediate level Python and R programming so it won't be in my curriculum.
Last warning: As I mentioned above, I know most of the topics but need a refresher for foundations. So this is not a beginner-friendly walkthrough.

So enough of me, I will demonstrate my way of getting this done and I hope this helps people with similar background. Here is the path I am and will be walking on for a while:

1) Linear Algebra > 2) Multivariable Calculus > 3) Neural Networks > 4) Hyperparameter tuning, Regularization and Optimization > 5) Basic Architectures(like RNN,ConvNets, LSTM) > 6) NLP Basics > 7) Attention Models

Here are the resources for this steps:
1) LINEAR ALGEBRA
First part of the course is Multivariable Calculus
Math for Economists by Jason Kronewetter Ph.D.
http://ocw.uci.edu/courses/math_4_math_for_economists.html

Alternative:--You can also go for Gilbert Strang's Linear Algebra course on MIT open courseware.--
https://ocw.mit.edu/courses/mathematics/18-06-linear-algebra-spring-2010/

2) MULTIVARIABLE CALCULUS
Second part of the course is Multivariable Calculus
Math for Economists by Jason Kronewetter Ph.D.
http://ocw.uci.edu/courses/math_4_math_for_economists.html

Alternative:--MIT ocw is here for you again.-- https://ocw.mit.edu/courses/mathematics/18-02sc-multivariable-calculus-fall-2010/1.-vectors-and-matrices

3-4-5) DEEP LEARNING-OPTIMIZATION_ARCHITECTURES etc.
Andrew NG's legendary course will be enough for the theory.

https://www.coursera.org/specializations/deep-learning#courses

6-7) NATURAL LANGUAGE PROCESSING AND ATTENTION MODELS
Recently deeplearning.ai has released a great course for NLP practioners.
https://www.coursera.org/specializations/natural-language-processing?

Alternative:--Stanford ocw has released a course while ago but it can be math&programming heavy for some of you. So this is my alternative.
https://www.youtube.com/playlist?list=PLoROMvodv4rOhcuXMZkNm7j3fVwBBY42z

I will be sharing my updates about study in the next part.

___________________UPDATES______________________
I have finished Linear Algebra part in 8 days. I also took notes I am planning to share.

I started Multivariable Calculus part. Seems not so hard for now.

DEV Community: Sam Oz

How to make your model training stops when reached a threshold?

Laying the Foundation for Deep Learning