DEV Community: Kaylee Lubick

Animated Python: For Each Loops and Palindromes

Kaylee Lubick — Tue, 18 Oct 2022 11:24:28 +0000

Loops are one of the trickiest concepts for beginning programmers.

To help, the team at Concepts Illuminated started a series of animations that walk through how Python code executes for loops (and conditionals, variables, and functions).

The first animation features code that detects palindromes, a sample of which is available below.

The full animation, along with some commentary explaining the various steps, is available for free on YouTube [8 minute watch].

The animation was created using Manim, which allowed us to write Python code to explain Python code. I find that amusing (and also very helpful).

Building a ML Transformer in a Spreadsheet

Kaylee Lubick — Sun, 13 Mar 2022 14:47:24 +0000

Attention Is All You Need introduced Transformer models, which have been wildly effective in solving various Machine Learning problems. However, the 10 page paper is incredibly dense. There are so many details, it is difficult to gain high-level insights about how they work and why they are effective.

After several people-months of reading other blog posts about them, the team at Concepts Illuminated understood them well enough to create a Transformer in a spreadsheet and made a video walking through it.

At a high level, Transformers are effective because they convert the data in a way that can make it easier to find patterns. They build on ideas from Convolutional Neural Networks and Recurrent Neural Networks (Focus and Memory), combining them in something called self-attention.

The video covers these ideas in more details and this is the link to the spreadsheet with the implemented Transformer. Skip to the "Appendix" sheet if you want to see a layer with all the bells and whistles, including multi-headed attention and residual connections.

Implementing the Transformer really helped me understand all the components. I'm especially proud of our metaphor of "scoring points" for explaining self-attention.

Other resources I found useful when researching transformers:

More of our work:

Building a traditional neural network in a spreadsheet to learn AND and XOR (Part 1, Part 2)
Building a convolutional neural network in a spreadsheet to recognize letters (Part 1, Part 2)
Building a recurrent neural network in a spreadsheet to emulate autocomplete

Recurrent Neural Network In a Spreadsheet

Kaylee Lubick — Mon, 25 Oct 2021 11:27:49 +0000

Truly understanding Recurrent Neural Networks was hard for me. Sure, I read Kaparthy's oft-cited RNN article and looked at a diagrams like:

But that didn't resonate in my brain. How do numbers "remember"? What details are lurking in the simplicity of that diagram?

To understand them better, the team at Concepts Illuminated built one in a spreadsheet. It was not as straightforward as our previous attempts to build neural networks this way, mostly because we had to discover novel ways to visualize what was going on:

Those visualizations really helped RNNs click for me, and we were able then to implement one and figure out the weights to make it work. Walk through that entire process with us in this YouTube video.

The completed spreadsheet is here.

More of my work:

Building a traditional neural network in a spreadsheet to learn AND and XOR (Part 1, Part 2)
Building a convolutional neural network in a spreadsheet to recognize letters (Part 1, Part 2)

Building an Image Recognizing Spreadsheet

Kaylee Lubick — Wed, 22 Sep 2021 01:42:10 +0000

The team at Concepts Illuminated built an image recognizing spreadsheet to better understand Convolutional Neural Networks. The sheet can recognize the letters X, T, and L. Creating it really illuminates the details, especially how a deep convolution layer can build off the previous layers.

There is a two-part series that walks through building that spreadsheet.

Part 1 - https://www.youtube.com/watch?v=8QfFtUcuYCU
Part 2 - https://www.youtube.com/watch?v=pH0s0e62GT8

The fully implemented sheet is at https://bit.ly/CNNSheet3 if you just want to jump to the end.

If you are just getting into neural networks, maybe look at our previous series on building and training a traditional neural network in a spreadsheet:

Training a deep neural network in a spreadsheet

Kaylee Lubick — Sun, 22 Aug 2021 19:57:07 +0000

The second and final part of the Spreadsheet neural network video series is up. As promised in the part 1 writeup, this video demonstrates how to make and train a deep neural network; one with a single hidden layer.

Creating and implementing the neural network algorithms in a spreadsheet genuinely helped the entire Concepts Illuminated team understand them. Hopefully they help you too.

Training a simple Neural Network in a Spreadsheet

Kaylee Lubick — Sat, 21 Aug 2021 20:20:59 +0000

For SOME1, the team at Concepts Illuminated made a video series that walks the viewer through making a few different neural networks in a spreadsheet.

Why a spreadsheet? I personally find it easier to reason about how all the steps and equations fit together when you can see them all at once and a spreadsheet is a good mechanism for that.

The functions learned aren't particularly complex: the series covers the AND function, the XOR function, and then a 1 bit half-adder. However, these functions are easy to visualize in 3D (and Bubble charts) and can help explain why hidden layers are necessary.

Part 1 is available now. Part 2, which covers deep neural networks, should be out soon.