DEV Community: Akash Tandon

Kaggle Learn review: there is a deep learning track and it is worth your time

Akash Tandon — Sun, 28 Jan 2018 05:50:26 +0000

Right from my undergrad days when I was starting out with machine learning to this date, my admiration for Kaggle continues to grow. In addition to being synonymous with and popularizing data science competitions, the platform has served as a launching pad and breeding ground for countless data science and machine learning practitioners around the world, including yours truly. In fact, skills I'd picked up from the platform are part of the reason that I recently got to join SocialCops, a company I'd admired for years. However, I hadn't been on the platform in 2017 as much as I would have liked. So when I saw Ben Hamner's tweet launching Kaggle Learn, a set of interactive data science tutorials, I made up my mind to give it a shot.

Ben Hamner

@benhamner

Excited to launch Kaggle Learn - interactive tutorials on machine learning, deep learning, R, and data visualization kaggle.com/learn

22:26 PM - 16 Jan 2018

Zeroing in on deep learning

Learn currently hosts tutorials about 4 topics - introductory machine learning, R programming, data visualisation and deep learning. I'd stumbled across machine learning for the first time in the form of neural networks (NN) more than 3 years back. Since then, I'd studied the theoretical details of NN at various points of time but somewhat ironically, I'd never got into practical deep learning except for a few tutorials. Hence, I decided to get started with the deep learning track.
The reason I mentioned my past experience with ML and NN was to point out the fact that I was not a complete beginner when I had gotten started with this track and if you are, start with the machine learning track instead.

Getting started

If you are unfamiliar with neural networks or haven't come across them recently, it would be a good idea to get some theoretical foundation before starting with hands-on tutorials. There are a number of introductory resources out there, both text and video. I used an excellent video by 3Blue1Brown, a YouTube channel, as a refresher.

Choice of framework

The track uses the high-level Keras API and a Tensorflow backend. Even with numerous frameworks out there, this combination seems to find favor as a beginner-friendly choice among a large portion of the deep learning community. Personally, I admire Keras for being well-designed, user-friendly and playing a big role in democratizing access to deep learning methods.

The track

The deep learning track is currently comprised of six sections. They are:

Intro to Deep Learning and Computer Vision: Starting off with a computer vision example is a great way to get acquainted with machine learning. This is the application which had put deep learning in the limelight and the data (images) is something most of us deal with on an everyday basis. The accompanying exercise allows you to play around with basic convolutions and images.
Building Models from Convolutions: Convolutional neural networks (ConvNet) have received wide praise and coverage for being extremely successful with image recognition tasks. The basics of ConvNets are discussed and the stage is set up for their implementation.
Programming in Tensorflow and Keras: You get to see TF+Keras in action for the first time and you'll be amazed at the ease with which you can get up and running. There's a lot of hand-holding here so getting the code to work alone won't be very useful. Try to understand the code, including helper functions, as much as possible.

Related - Big deep learning news: Tensorflow chooses Keras
Transfer Learning: It was a great decision by Dan Becker to include this, and it is my favorite part of the tutorial. Prior to this, my perception of transfer learning was as an advanced topic which would require a decent amount of know-how to even get started. I am delighted to tell you that that I couldn't have been more wrong. Even if all you know are the very basics of NN, the idea of transfer learning itself is fascinating and I've decided to spend some time in near future to research about the topic. Prior to starting this section, I'd gone through the following video by the one and only Andrew Ng.

Data Augmentation: Simply put, data augmentation is a handy technique which results in increased number of data points for your machine learning algorithm. This section discusses the technique as well as its implementation in Keras.

Related - What you need to know about data augmentation for machine learning
A Deeper Understanding of Deep Learning: The code used in the previous sections, particularly the various parameters, are discussed in more detail. Also, stochastic gradient descent and backpropogation are briefly discussed.

Conclusion and additional resources

When learning a new topic, I've always found it best to start with a high-level overview. That's precisely what this track aims to offer and for most part, delivers. For a considerable amount of time, setting up deep learning frameworks used to be a roadblock to getting started with the topic. To that end, Kaggle leverages its platform's capabilities to host the code and while doing so, showcases its potential for being useful for collaboration. All that being said, this topic only scratches the surface, even if in a better manner than most tutorials out there. You can plan out your path from here on. If it helps, below are some of the resources I plan to dive into or explore over the next few weeks.

Neural networks and deep learning by Michael Nielsen (e-book)
Neural networks and deep learning by Andrew Ng and deeplearning.ai team (MOOC)
Practical deep learning for coders by Jeremy Howard and fast.ai team (MOOC)

If there's any other useful resource you can think of, feel free to mention it in the comments below.

If you read and liked the article, sharing it would be a good next step.
Additionally, you can check out some of my open source projects on Github.
Drop me a mail, or hit me up on Twitter or LinkedIn in case you want to get in touch.

This post was originally published on Tech and Mortals.

Open Source: The itch, the hustle and the merge

Akash Tandon — Thu, 18 May 2017 10:49:48 +0000

DISCLAIMER: Before you begin, be aware that the protagonist of any fictional account that this write-up might contain would be a male and would be referred to as a â€˜he’ or at times, simply as â€˜coder’. That’s not because the author is a sexist prick (hopefully) or a lazy writer; it’s so because the writer is a male, is identified by simpler minds as a â€˜coder’.

XKCD: Open Source

Background

There are two things that might be useful to know before we really dive in: who’s the nut-job who wrote this and why did he do it?

You can skip this part entirely if you want to. I won’t hate you for that. Or maybe I will. It doesn’t matter anyway, right?

Dev Philosophy

In context of this post, he is someone who likes to code and sporadically write a blog or two. He believes that what he writes, code or letter, can make an impact. Who doesn’t?

Why he wrote this is more important. The immediate reason is that he recently contributed to an amazing library, felt pretty darn good about it and would like his peers to do the same more often.

A deeper reason still is his belief in the philosophy of Free/Libre and Open Source Software (FLOSS). There are different camps within this movement. You are free to either do your research and align yourself with one of them, or stay neutral. The deeper philosophical debate isn’t in this post’s scope.

Below are quotes by two dudes who are faces of this movement. They might not be on the same page at times but have done more for the world than most will ever know:

Linus likes to fuck around every now and then. No pun intended.

The Contribution

Watch closely

“Every great Open Source contribution consists of three parts or acts. The first part is called “The Itch”. Perhaps there’s a bug which cropped up when the coder was working with a library, or maybe there’s some feature which the world needs but hasn’t been implemented yet. Better still, he just wants to â€˜contribute’.

The second act is called “The Hustle”. The coder pulls his socks up, puts on his hackathon t-shirt and dives into the source code and documentation. Now, he’s looking for the solution… but he won’t find it, not soon anyway, because of course he’s overwhelmed. Eventually he will write the required code, update the documentation and send a PR. The world will see his contribution now. Well, not yet. Because writing the code in your branch isn’t enough.

That’s why every contribution has a third act, the critical part, the part we call the “The Merge”.”

The Itch

Reasons why the itch might crop up have been detailed in the previous section. It might appear after a week or month-long buildup. Then again, it can also hit you out of nowhere one sunny afternoon as you are crouched in your air-conditioned office trying to implement a shiny new feature for your company’s â€˜world-changing’ product or at midnight when you are working on a college assignment due next morning.

It can hit you because you want to make an impact, or because someone told you that Open Source can help you land your dream job. Simpler reasons could be to attend conferences or a desire to get your Github tiles to turn green.

Whether you are a noble technologist or a simple prick, commits and pushes don’t judge. Pull requests might.

If you know the library you want to work with, what bug you want to fix and the new feature that you need, that’s great! Get the library’s source code and documentation on your local system. Maybe open an issue, file a bug or feature request while you’re at it.

If you’re one of those who just want to â€˜contribute’ but don’t where to start, pat yourself on the back for taking the first step. In a nutshell, you will need to zero in on the source code (library/package) and what you want to do with/in it. Maybe you want to help the folks at Mozilla or maybe you want to contribute to the popular machine learning library by Google.

These links will help you get started with scratching the itch:

Author’s itch

As for author’s most recent brush with Open Source, he wanted to contribute to Pandas, an amazing Python library for data analysis that he had been using for some time now.

He went to Panda’s GitHub repository, filtered open issues with the tag â€˜difficulty novice’ and found a bug which seemed interesting.

The Hustle

You are done searching and deciding the higher-level details. You know which project to contribute to and what bug-to-solve/feature-to-implement.

Buckle up because you’re just getting started. More often than not when working with an OS project, a large code-base or tons of documentation won’t be your first hurdle. Setting up the project or building it and running the tests can be a real pain though. Being on a UNIX-based system (Linux or Mac-OS) and being adept with the command line will come in extremely handy when doing this. Sorry Windows users, that’s just the way things are!

Most important tool you would ever use when developing Open Source software is git, a tool for version control. Version control enables multiple developers, hundreds or thousands at times, to work together on the same code-base. If you haven’t already, learn git.

Since tests have already been mentioned, it’s essential that you know this: they are important! In fact, tests are one of the most important components of any decently-sized software project. Think of it this way. In absence of reliable tests, we could end up breaking old features whenever we make changes. The developers would keep fixing bugs which could have been easily avoided and will eventually give up altogether. Many popular OS projects use a test-driven development (TDD) style and Continuous Integration (CI). Read up about CI and testing frameworks for your language of choice.

The drudge work has been done and the project has been set up on your machine. Ensure that you go through the project’s developer documentation (example). Once that’s done, we are all set to start writing code!

The coding can take time but it’s simple in principle. Write tests, if needed, and the required code.

At this point, the code has been written. It’s time to open a Pull Request and start working towards the third act.

Author’s Hustle

After identifying the bug, he forked the Pandas repo and cloned it on his local machine. He went through the â€˜Contributing to pandas’ docs which detailed how to work with the code-base, installed dependencies, created a new branch, wrote the required tests and code, pushed to his fork on Github, opened a Pull Request and waited for a maintainer to review his PR.

The Merge

At times, this will be a smooth ride. You would have done everything correctly and your PR will be merged without much fuss. Many a times, that won’t be the case; especially when you are starting out.

If the project uses CI, you will have to make sure that the required tests and checks pass. If the changes made by you aren’t up to the mark, you will be told by those reviewing the PR about what changes to make. This might happen multiple times but your hard work will pay off once the needful has been done and your PR will get approved.

This will be followed by the moment which you had been looking forward to all this while. Your contribution will get merged into the project. Hooray!

Author’s Merge

Changes had to be made to his PR multiple times after being opened. Getting all the checks to pass took couple of iterations of the process described previously. Then he was asked by the maintainer to add a few more tests just to be safe since a tricky case was involved. After a week of opening the PR, it got approved, the changes got merged and the author slept peacefully that night.

The Merge: Initiation

The Merge: Conculsion

Epilogue

You might feel overwhelmed at this point and that’s all right. It’s a lot of information to process and Open Source might seem like a lot of work. But this is how great software gets built. Also, this is a great way to make an impact through your code and meet awesome people in the process. Hang on and you will have the time of your life, mate.

You can follow my work on Github.

If you read and liked the article, sharing it would be a good next step.

Drop me a mail, or hit me up on Twitter or Quora in case you want to get in touch.