DEV Community: Andrew Garcia

GPT-4 Literally Doubles Your Learning Speed ... (According to GPT-4)

Andrew Garcia — Tue, 14 May 2024 16:35:16 +0000

I used GPT-4 to estimate how much it can boost your learning and development in programming and AI. The results? A whopping 2x increase in learning speed and productivity! Here’s a breakdown of the impact GPT-4 can have compared to traditional methods.

Advantages of Using GPT-4

Instant Feedback and Support: GPT-4 can provide immediate answers to your questions, explain complex concepts, and debug your code. This reduces the time spent searching for information or troubleshooting issues.
Code Generation: GPT-4 can help you write scripts by generating code snippets based on your descriptions. This can speed up the development process and help you learn new programming techniques.
Personalized Learning: GPT-4 can tailor explanations and examples to your skill level, helping you understand concepts more quickly.
Comprehensive Resources: GPT-4 can suggest relevant resources, such as documentation, tutorials, and best practices, making your learning process more efficient.
24/7 Availability: Unlike human tutors, GPT-4 is available around the clock, allowing you to learn and develop at your own pace.

Speed of Progress with GPT-4

By integrating GPT-4 into your learning routine, you can significantly boost your progress:

Learning New Concepts: A regular person can learn new programming concepts faster with GPT-4 because it can provide clear, concise explanations and examples on demand.
Developing Scripts: With GPT-4's assistance, you can develop scripts more quickly by leveraging its code generation capabilities and debugging support. This can reduce the time it takes to go from idea to implementation.
Problem Solving: GPT-4 can help you solve problems more efficiently by offering different approaches and solutions, which can lead to faster mastery of complex topics.

Example Scenarios

Learning a New Programming Language:
- Without GPT-4: You might spend hours reading tutorials, searching for answers, and experimenting with code.
- With GPT-4: You can ask specific questions, get instant explanations, and receive code examples that illustrate key concepts.
Building a Project:
- Without GPT-4: You might encounter roadblocks that require extensive research to overcome.
- With GPT-4: You can describe the problem to GPT-4 and get immediate guidance or code snippets that help you move forward.
Debugging Code:
- Without GPT-4: Debugging can be time-consuming and frustrating.
- With GPT-4: You can paste your code and get suggestions for fixing errors, along with explanations of why the errors occurred.

Estimating Time Savings

While the exact time savings can vary depending on the individual and the complexity of the tasks, here’s a rough estimate:

Learning Curve Reduction: You might reduce the learning curve by 30-50% because of the immediate, targeted assistance GPT-4 provides.
Development Speed: Script development could be 2-3 times faster with GPT-4's code generation and debugging help.
Overall Productivity: You can expect to achieve in weeks what might have taken months without GPT-4.

Quantifying the Impact of GPT-4

To illustrate the impact of GPT-4, here's a comparison of experience gains with and without using this tool. These estimates are based on the enhanced support and resources GPT-4 offers.

Normalized Experience Gains

Category	Time Investment (Without GPT-4)	Experience Gain (Without GPT-4)	Time Investment (With GPT-4)	Experience Gain (With GPT-4)
Learning New Programming Languages or Frameworks	6-12 months	1.0	3-6 months	2.0
Developing Advanced Projects	3-6 months	1.0	1-3 months	2.0
Staying Current with AI Tools and Integrations	Continuous learning, 5-10 hours/week	1.0 (1 year per year of learning)	Continuous learning, 3-5 hours/week	1.5-2.0 (1.5-2 years per year of learning)

Raw Estimates (Normalized)

Experience Category	Theoretical Job Experience Gain (Without GPT-4)	Experience Gain (With GPT-4)
Overall Gain	1.0	2.0-3.0
Learning New Tools/Technologies	1.0	2.0
Developing Projects	1.0	2.0
Continuous Learning	1.0 (1 year per year of learning)	1.5-2.0 (1.5-2 years per year of learning)

Conclusion

GPT-4 can significantly enhance your learning and development, helping you gain more experience in less time. Whether you're learning new programming languages, developing projects, or staying current with AI tools, GPT-4 provides valuable support to boost your productivity and skills.

Start integrating GPT-4 into your learning routine and experience the quantifiable difference it can make!

Disclaimer: The estimates and comparisons provided in this article are generated using GPT-4 and are based on general observations. Results may vary based on individual backgrounds, learning styles, and other factors.

I Wrote a Book on Publishing Python Packages: Here's Why It Matters

Andrew Garcia — Thu, 07 Dec 2023 00:11:11 +0000

I'm thrilled to present my book, Python Package Publishing: From Code to Open Source Success
. This guide aims to empower Python developers by shedding light on the intricacies of package creation and distribution.

The Idea

My journey as a Python developer revealed a common challenge among peers: the complexity of package development and distribution. This book is a response to that challenge, aiming to simplify and clarify the process for developers at all levels.

Why Publishing Python Packages Matters

Bridging the Knowledge Gap

The Python ecosystem is rich and diverse, thriving on open-source contributions. However, a gap exists in understanding how to effectively create and distribute packages. This book aims to bridge this gap, offering a comprehensive guide on package structure, documentation, licensing, and distribution on platforms like PyPI.

Empowering Machine Learning Development

Neural networks, particularly those developed using frameworks like TensorFlow and PyTorch, typically follow an ordered directory structure similar to Python packages. This structured approach is not only crucial for developing these networks but also for their deployment. While the book doesn't delve into machine learning specifics, it teaches how to structure code in an ordered directory format, which is useful for both package deployment and neural network development. (I am in the process of writing a more specialized program on Neural Network Development, which will also be up for sale in my Gumroad page)

Encouraging Open Source Contributions

By demystifying package development, the book encourages more developers to contribute to the open-source community. This enhances the Python ecosystem and fosters a culture of collaboration and knowledge sharing.

Professional Growth and Recognition

Publishing a Python package showcases your expertise and contributes to professional growth. Also, it can be quite fun!

Why This Program Stands Out

Comprehensive Learning for All Levels

This program is meticulously crafted to serve a wide audience. Whether you're new to Python or have years of experience under your belt, you’ll find valuable insights and techniques to enhance your package development skills.

Bilingual and Practical

Recognizing the diverse Python community, the eBook is offered in both English and Spanish, making the material accessible to a wider audience. It's not just about theories; the program is enriched with practical examples and step-by-step guides to solidify your learning.

Practical Tools for Immediate Application

Included are a comprehensive package example framed around a Virtual Pet game (Python code) and a documentation template (Read The Docs Sphinx/TeX code), providing practical tools for immediate application. These resources are designed to jumpstart your development journey and give you a tangible head start.

Optional Personalized Coaching

For those seeking a more personalized learning experience, the program offers an optional coaching option. This provides direct access to experienced mentors who can offer tailored guidance, enhancing your learning curve and proficiency in Python package development.

Inside the Program

Here’s a snapshot of what you’ll explore:

Introduction: Setting the stage for your Python package development journey.
Development Environment: Essential tools and setups for efficient development.
Project Development: Turning ideas into reality with Python.
Packaging Techniques: Best practices for creating distributable Python packages.
Deployment Strategies: Methods for successfully launching your package.
Documentation Skills: Crafting clear and effective user guides.
Conclusion: Reflecting on the journey and looking ahead.
Appendix: Handy Git Cheat Codes for efficient version control.

Conclusion

Python Package Publishing is more than a book; it's a tool for empowering developers across various disciplines. Whether you're a novice or an experienced developer, this book has valuable insights for everyone. By mastering the art of Python package development, you can significantly contribute to the growth and evolution of the Python community and beyond.

Interested in starting your journey in Python package development? Check out the book. Let's engage in a robust discussion in the comments below!

Recursion

Andrew Garcia — Fri, 09 Sep 2022 21:08:48 +0000

When reduced to a concise example, recursion can be quite a simple concept to grasp.

The below recursive function:

f(x,count)=10+x+f(x,count−1)f(x,\mathrm{count}) = 10 + x + f(x,\mathrm{count - 1})

Takes a summand $x$ float and a $count\mathrm{count}$ index which defines the number of recursions. The function decreases this index with every recursion, and terminates the function with a "0" when the $count\mathrm{count}$ value becomes 0.
Defining this function in code:

#include <bits/stdc++.h>
using namespace std;

double recurAdd(double x, int count)
{
    if (count == 0){return 0;} // termination
    return 10 + x + recurAdd(x, count - 1);
}

// Driver code
int main()
{
    cout << recurAdd(0.5, 3);
    return 0;
}

This small program will output 31.5 as a final value. We can represent the complete recursion in the following way:

f_1 = 10 + 0.5 + ( 10 + 0.5 + ( 10 + 0.5 (0) ) )

Where the recursion can be broken down in the following way, from its first to last input iteration:

f_1 = 10 + 0.5 + f_2 \\ f_2 = 10 + 0.5 + f_3 \\ f_3 = 10 + 0.5 + f_4 \\ f_4 = 0

After the last input function $f_4$ has been terminated, the running of operations can be visualized to occur from the deepest $f_4$ to the outermost $f_1$ nest. I visualize this process with the following schematic:

Though not all recursion algorithms may follow this pattern, this visual has helped me simplify my understanding of recursion. I hope my visual helps you too.

-Andrew

Tensor performance benchmarks for Python and C++

Andrew Garcia — Sat, 25 Jun 2022 00:06:54 +0000

What are tensors in computing?

Perhaps this requires a longer discussion but in computing, tensors are a mathematical abstraction. You may find the typical definition of tensors is "multidimensional arrays", and this may be interpreted as a collection of arrays within an array (I will refer to these manifestations as nested vectors in the table below). Tensors, however, are vectorized ("flattened") tensors at the lowest level of code and thus, are typically fastest in this primitive form.

Tensor performance: axes swaps

The image on the cover is a graphical representation of a tensor with 64 elements from 1 to 64. A tensor of any side length $L$ can be easily build as a vector in Python and C++:

# Python 
vector = [i for i in range(L*L*L)]

# C++
std::vector<int> vector;
for (int i=1; i<= L*L*L; i++) vector.push_back(i);

For the benchmarks the tensor is a 100 x 100 x 100 tensor with 1 million elements from 1 to 1 million. This is a huge tensor; making a dictionary with all its coordinates results in a 12 MB dictionary file!

Here's a graphical zoomed-in representation of this tensor to compare with the significantly smaller one in the cover photo:

The operation to benchmark has been chosen as the swapping of this huge tensor's 0th axes with its 2nd axes, with index reversal of all indices (that is, the first element becomes the last, the second the penultimate, and so on). This operation has been chosen as transposition has a high time complexity of $O(L3)\mathcal{O}(L^3)$ .
The graphical representation of this transposed tensor is presented below. It's also zoomed-in because they are large data and hard to move (even when using WebGL which uses the GPU):

The benchmarked operation was run with different computing methodologies and tensor representations. As was evident, vectorized tensors are superior in performance to nested vectors. Not so evidently, this trend reverses in Python when a JIT-compiler is used.

Language	Tensor Init. Container	JIT Compiling	Operation time	Timer
Python	numpy.zeros((N,N,N)); Nested Vectors	-	2,520 ms ± 229 ns	inline %timeit
Python	numpy.zeros(NNN); Vectorized	-	613 ms ± 68.4 ms	inline %timeit
Python	numpy.zeros((N,N,N)); Nested Vectors	Numba	1.38 ms ± 63,600 ms	inline %timeit
Python	numpy.zeros(NNN); Vectorized	Numba	5.33 ms ± 186,000 ms	inline %timeit
C++	vector<vector<vector<int>>>; Nested Vectors	-	10.5501ms	chrono::high_resolution_clock
C++	vector<int>; Vectorized	-	0.015105 ms	chrono::high_resolution_clock

The operations under JIT compilers show a large spread in operation time because they initially take some time to initialize and compile the scripts into machine code. Interestingly enough, this suggests there may be a more succint way to write machine code for multidimensional arrays compared to vectorized tensors when programming in Python.

Finally, C++ vectorized beats all. It beats JIT-compiled Python code by $O(2)\mathcal{O}(2)$ (i.e. 100x faster) and conventional Python executions by $O(4−5)\mathcal{O}(4-5)$ orders of magnitude.

-Andrew

Git Cheat Codes: TLDR useful Git commands

Andrew Garcia — Sat, 05 Feb 2022 16:44:33 +0000

I recommend printing this post and/or having it handy on another device while working on Git on your PC.

Project Dev. Sequence

This is a useful and common command sequence used to pass changes to a Github/Git project:

git status
git diff 
git add .   
git commit -m "changes to all files" 
git push

Short explanations to each of them:

git status inspect / check the files that have been changed in the working directory
git diff [filename] check the specific changes made on the selected, changed file. Useful for writing the commit message.
git add [filename] stage changes to the selected file. This command adds the change to the staging area
git add . stage changes to all changed files
git commit -m "[commit message]" pass changes to the local repository (i.e. to the folder in your local computer)
git push push these passed changes to the server (the Github page / internet)

Create a Repository

Locally

In computer:

Make a folder, name it with your desired repository name, add stuff to it.
Open terminal in that folder and type git init.
Keep the terminal open.

In Github:

Click on "+" sign on top-right corner of / New Repository...
Add info on that page and then click on "Create Repository" green button.
Copy-paste manual guidelines on next page (starts with "echo README" I think) to open terminal from above.
git push. You're done!

Easier way

In Github:

Click on "+" sign on top-right corner of / New Repository...
Add info on that page and then click on "Create Repository" green button.
... Ignore next page and go to the next option.
While in new repository, click on green "Code" button and copy https/ssh address (I typically copy the ssh one).

In computer:

Open terminal and type git clone [copied-address-from-GitHub]. You're done!

Branches

Basic commands

# create a new branch
git checkout -b [your-new-branch-name]

# check the names of all your branches
git branch

# go to a particular branch
git checkout [your-branch-name]

# transfer all information from a "source" branch to a "sink" branch
git checkout [sink-branch-name]
git merge [source-branch-name]

# delete branch locally (computer)
git branch -d [your-branch-name]

# delete branch remotely ("on GitHub")
git push origin --delete [your-branch-name]

Rename branch

# checkout to master branch
git checkout main
# local 
git branch -m [old-branch-name] [new-branch-name]
# remote (Github)
git push origin [new-branch-name]
git push origin --delete [old-branch-name]

Add all content from a fork to a larger repository

To add all the content from your GitHub fork to the larger repository, you can follow these steps:

# Clone the larger library repository to your local machine using the git clone command
git clone https://github.com/larger-repo.git

# Add the fork as a remote to your local clone of the larger library repository using the git remote add command.
git remote add fork https://github.com/[your-username]/[forked-repo-name].git

# Fetch the changes from the fork and switch to a new local branch that will track the forked repository's changes
git fetch fork
git switch -c branch-fork fork/main

# Checkout to the main branch of the larger repo
git checkout main 

# Merge the changes from the fork into your local clone of the larger library repository using the git merge command with the --allow-unrelated-histories flag 
git merge --allow-unrelated-histories branch-fork

# Commit the changes
git commit -m "Merge contents from fork to larger repo"

# Push the changes to the main branch of the larger repository
git push origin main

Undoing Stuff

Revert modified files not yet staged for commit

git restore [filename]

Let's say you have a file with code that you had previously pushed and you made changes to the code; this command does an UNDO on those changes made.

Undo a commit

On your Github repository, Click on the commits counter below the Code button

Let's say I want to undo the first commit, then I would copy its hash (95f0e33) and run the following command:

git revert 95f0e33

Temporarily revert to previous commit

git checkout [commit_ID]  #go to the former code commit / version
git checkout master #go back to current version

Remove files listed on `.gitignore` which have been previously committed (i.e. "not gitignored")

git rm -r --cached .;
git add .;
git commit -m "gitignored files deleted from remote!"

Make Git Remember Your Username and Password

Watch this video around the 3:29 timestamp
Generating an SSH Key 3:29

Adapt SSH key to repositories

'''new repository'''
git clone git@github.com:[user]/[repository].git

'''or if already created repository'''
cat .git/config  #displays local information of repository
git remote set-url origin git@github.com:[user]/[repository].git  #this is the command which sets  the key to your local repository.

Find all folders which are linked to Github repositories in a specific folder (Linux users)

While in folder to search, you may open a Terminal and type the following:

find -name ".git"  #simple
'''or'''
find -name ".git" -print0 | sort -z | xargs -0 cat #in alphabetical order (may take long to compile IF several git repositories synced)

Running computational tasks indefinitely on the cloud

Andrew Garcia — Sun, 13 Jun 2021 04:11:23 +0000

I have been running my most resource-intensive computations on the cloud (that is, on commercial external servers) rather than on using the CPUs from my personal computer. This is advantageous for many reasons such as liberating your computer from resources and in doing so, avoiding lagging or other undesired effects. These 2 reasons are most unique to the application:

Computational tasks can be kept running in the cloud even after your computer is turned off.
If more cloud resources are needed, you can upgrade your resources while keeping your same virtual environment at all times (i.e. programs / applications installed on the cloud space)

I have outlined the process to sending your calculations to the cloud with DigitalOcean as well as other useful features you may use for cloud computing below.

Outline

Sign-up to DigitalOcean
Python Users
Installing Git on droplet
Transferring files between droplet and computer
Run processes in the cloud even after turning off your computer
How to take a snapshot of your created droplet
Machine Learning: Installing Tensorflow

Let's begin.

Sign-up to DigitalOcean

Referral link: Sign up with a $100 credit over 60 days through here

I have used DigitalOcean virtual machines known as droplets for this cloud computing activity. I consider the DigitalOcean platform quite user-friendly and their pricing of droplets is also very reasonable and flexible; the cheapest droplet, for instance, will only cost about $5 a month and if you decide to only use it for 2 days, the price would be around $(2/30) x 5 = 33 cents TOTAL and not $5 (as long as the droplet is destroyed after the 2 days, see more on this below) You may click on the link above to sign up

Python users: Installing Anaconda python on droplet

https://www.digitalocean.com/community/tutorials/how-to-install-the-anaconda-python-distribution-on-ubuntu-20-04

Once you have the droplet, I recommend installing the Anaconda suite which comes with several python modules pre-packaged. The tutorial above allows you to do so with command lines alone.

Installing Git on droplet

https://www.digitalocean.com/community/tutorials/how-to-install-git-on-ubuntu-20-04#setting-up-git

Git may already be installed in your DigitalOcean droplet, but above is the tutorial nonetheless. Git can be used to communicate with your Github repository, so that Python programs developed in your computer can be pulled to your droplet, and viceversa. (P.S. You may also need a Github account)

Transferring files between droplet and computer

https://docs.digitalocean.com/products/droplets/how-to/transfer-files/

This approach uses Filezilla to do this. I use this in the case my programs generate files and I need to transfer them from droplet to computer.

Use screen to run process on droplet even after turning off your computer or exiting ssh

https://www.digitalocean.com/community/tutorials/how-to-install-and-use-screen-on-an-ubuntu-cloud-server

Screen is a program which allows you to do just that. You may need to install it INSIDE your droplet (i.e. ssh protocol) and then call "screen". After calling screen you can run your program (e.g. "python myprogram.py") and then press "Ctrl+A", followed by letting go off the keys, followed by pressing "D" to detach the screen.

Then you may exit the droplet or even turn off the computer and the python process will keep on running! To access again, access the droplet, then type "screen -r" to re-attach the screen and see how far your process has been running on the server.

How to take a snapshot of created droplet

https://docs.digitalocean.com/products/images/snapshots/how-to/snapshot-droplets/

In case you have been using an expensive droplet and you do not want to be charged for it for a period of time, you may need to destroy it. No worries! If you want to "revive" your droplet later on preserving all its installed programs and environment*, you may do so by taking a snapshot. Click on the link above for this option.

How to install TensorFlow on droplet

https://www.digitalocean.com/community/tutorial_collections/how-to-install-and-use-tensorflow

If you're ready to take your droplet to the next level, machine learning, then this tutorial may be just for you. The current recommendations are to install TensorFlow in a droplet with at least 4GB RAM.

Let's say you created your whole development environment on a smaller droplet. That's ok. You may take a snapshot of the smaller droplet and then re-execute it on a larger memory droplet. This strategy may also be useful if you would want to take time setting up your development platform on a smaller droplet to spend less money before moving to the larger one.

Conclusion

DigitalOcean droplets are designed to be suitable for developers and small applications. Their application to cloud computing goes beyond python, as it can be extended to as many programming languages as can be run on a computer.

A reduced explanation on neural networks

Andrew Garcia — Thu, 25 Mar 2021 19:26:57 +0000

In a linear fit, data is fitted to a linear equation, typically to the slope-intercept form:

y = w_1 x + w_0

A linear fit (red) of the data (in blue) with the above equation. Such equation is better known as y = m x + b, but it has defined as above for context.

Likewise, for a multi-linear equation with a total number of $D$ independent variables, the expression is:

w_1 x_1 + w_2 x_2 + \cdots + w_D x_D + w_0

Either of the above equations can be fitted by changing the $w_i$ weights in a way that minimizes the error between the data points and the theoretical points generated by the equation. This error metric is typically known as a loss function or cost function.

In the case of deep learning, the equation is more complex. The multilayer perceptron (MLP) model is used as an example here for simplicity, as it may be one of the simplest neural networks to explain. Its architecture is shown below:

A multilayer perceptron (MLP) neural network.

In here, each column is a layer of vertical nodes, where the first layer from the left has nodes containing the input $x_i$ variables containing the information of each $i$ pixel in this example. The layers residing between the first and last layers in the network are known as hidden layers as the flow of operations in these sectors is typically not disclosed by the machine and is thus hidden to the user. The nodes in these hidden layers are also typically known as neurons as they handle most of the $w_i$ coefficients adjustments during the training phase. Finally, the last layer contains the output nodes, which give rise to the dependent variables of the model.

It is helpful to get a notion of the connectivity of this topology to understand how the network operates. Here, the connections to 2 different nodes of the above network will be explored. First, let's look at the first node of the first hidden layer:

The connections of the 4 input variables to the first node of the first hidden layer is a linear combination of the first four variables to such node, and can be expressed as:

n_1 l_1 = w_{1,n_1 l_1}x_1 + w_{2,n_1 l_1}x_2 + w_{3,n_1 l_1}x_3 + w_{4,n_1 l_1}x_4

Now let's look at the connections for the following node in the second hidden layer:

The connections of the 6 hidden nodes from the first hidden layer to the first node of the second hidden layer is a linear combination of the latter 6 hidden nodes. This new node can then be defined by the below equation. The node previously defined in the above equation has been bolded for context.

n1l2=wn1,n1l2n1l1+wn2,n1l2n2l1+wn3,n1l2n3l1+wn4,n1l2n4l1+wn5,n1l2n5l1+wn6,n1l2n6l1n_1 l_2 = w_{n1,n_1 l_2} \bm{n_1 l_1} + w_{n2,n_1 l_2} n_2 l_1 + w_{n3,n_1 l_2} n_3 l_1 + w_{n4,n_1 l_2} n_4 l_1 + w_{n5,n_1 l_2} n_5 l_1 + w_{n6,n_1 l_2} n_6 l_1

”MLP neural networks are linear combinations of recursively-nested linear combinations.”

The number of nested operations depend on the depth [the number of layers of the network] and the number of variables and nodes in the system.

In a similar manner as with the linear equation fit, deep learning can be seen as the process done to minimize the cost or loss function of a deep learning neural network given the inputs and the shape of the network.

Images

Image classification and image recognition problems are widely taught in deep learning. The popular image recognition problem of "cat and dog" identification from a library of these animals occurs through a process analogous to the following heuristic.

For this example, let's use the following figure of a puppy as a "shiba.png" file (right-click Save Image As...):

Left-click above image for a graphical representation of the "dog" array

The .png image may be imported to Python in the following way:

import numpy as np
import cv2 as vision       #import computer vision module 
#read color image and declare it as (LxWx(rgb_color_channels(3))) array 
dog = vision.imload("shiba.png")   
# we can also decrease the complexity of the image by looking at the pixel intensities of the first channel, the "red" channel: 
dog = dog[:,:,0] 
dog = dog.reshape((*dog.shape[:2]))  # L X W array

The resulting image is now a 180 x 180 pixels array. To correlate the image in an analogous manner to the linear regressions made with the slope intercept form equation, each of these pixels must be made dependent to the inputs in the neural network. This can be done by vectorizing the above image array with the "flatten()" operation:

X = dog.flatten()

The $X\mathbf{X}$ variable above is thus a 1-D vector with a length of 32,400 pixels.

Click above image for a graphical representation of the vectorized "dog" array

After this transformation, each pixel can be more easily assigned to each of the nodes on the dense layer(s) of the neural network, which is fitted with a weight $w_i$ . The vector $W\mathbf{W}$ containing these weights must be of the same size as the number of pixels of this vectorized image. As an example, let's make these weights very small random numbers:

W = np.random.random(dog.shape[0])/10000

The value of a single $y$ data point is then the dot product between this vector and the vectorized image:

y = np.sum(W*X) 
print(y)

This value is analogous to calculating a singular $y$ value with the multi-linear equation shown at the beginning of this post with a random set of $w_i$ weights. As this value is only a single data point, it is evident that more than a single image is needed to make a proper fit for a cat-dog identification model.

Having the network properly identify this image as the drawing of a dog or cat will require passing the neural network through many images containing dogs as well as cats, allowing the network to adjust the weights associated with each pixel in the images to generate an adequate $y$ variable . In order to classify qualitatively, a logistic function (a "sigmoid") is typically used with the regression.

This is a reduced explanation on a simple neural network model. For a deeper understanding on this concept, here are some good online references:

Ian Goodfellow et. al. Deep Learning
Michael Nielsen Neural Networks and Deep Learning
Adrian Rosebrock pyimagesearch

-Andrew

My transition and journey from a Windows operating system to Linux (Ubuntu)

Andrew Garcia — Sun, 15 Nov 2020 04:39:26 +0000

A Linux 🐧 operating system (OS) can be a perfectly lean and open-source (essentially "free") alternative to Windows. As a matter of fact, this post [and all others] are being written using a 2015 HP Spectre x360 laptop with a fully-integrated Linux 🐧 OS (Ubuntu).

The perfect metaphor for going from a Windows to a Linux 🐧 operating system (OS) is getting rid of your training wheels; it is liberating and allows you to go farther without the burden of the small wheels at no additional expense, but it requires a little bit of work at the beginning to do so.

Not to worry, this coming from a relatively decent computer-savvy individual : it's not that bad with Ubuntu.

What is Ubuntu ?

Just like Windows has had several OS distributions (e.g. Windows 98, Windows XP, Windows.. something else) Linux 🐧 has a diverse variety of distributions you can choose from, each of which comes with pros and cons in terms of performance and learning curves. Among these distributions is Ubuntu, which is a fork [or a "child" so to speak] of the older Linux 🐧 distribution Debian

My Windows --> Ubuntu Switch

Dual-boot (Windows / Ubuntu hybrid computer) installation

I initially went with this option as I was initially worried about the extra work I may have to do if I were to need some functionality only offered to Windows.

I found instructions on YouTube on how to install Ubuntu alongside Windows to begin with due to these concerns.

I liked it a lot. When I booted with Ubuntu, the startup was instantaneous, all my programs ran faster and I could surf the internet with ease and no notorious lags. What I did not like so much was the fact that I had to allocate my computer's memory between the Windows and the Ubuntu boot.

Heads up: Keep in mind that once you go with the initial Windows / Ubuntu memory partition, you cannot re-partition the memory again [or with that much ease anyways], so be wise about how many GB you want to give Windows and how many you want to give Ubuntu

The way I saw it, it was a limitation on performance on the Ubuntu side.

Full Ubuntu installation

As I got more familiarized with Ubuntu, I found ALMOST every one of the tasks I used to perform with Windows I could do with Ubuntu. This along with the statement on the previous section was what motivated me to go with a full Ubuntu installation. You can easily go from dual-boot to full Ubuntu installation by learning from the same YouTube video(make sure to back up your data always)

Here are my Ubuntu "work-arounds" to Windows OS software

Microsoft Office --> (many; see below)

I am a graduate student so a lot of my work uses a software suite for office work (or what used to be Microsoft Office for me). There may be a million ways Microsoft Office programs can be replaced, but I'll go with mine below:

Microsoft Word --> TeX / LaTeX

Linux's open-source version of an office work-related software suite is LibreOffice, which may come pre-packaged with your Linux 🐧 installation. This suite has programs with functionalities very similar to those of Microsoft Word, Excel, and Powerpoint. However, some of these are not as mature as the Microsoft Office versions for my needs.

As far as word-processing software goes, LaTeX has become my best option. In contrast to how it may be for Windows OS's, it is very easy to install TeX and all packages thereof in Ubuntu. The syntax for writing documents is a whole programming language as a whole, but this can be made simpler by just starting with LaTeX templates and adapting them to your needs with ease.

If you would rather not put effort with LaTeX, I would go with Google Docs.

Microsoft Excel --> LibreOffice Calc

This is a part of LibreOffice which shines in my opinion. Not only is the quality of Calc as similar as that of Excel, but it also has some nice helper features, such as converting .csv files to spreadsheets and viceversa with ease by detecting these formats when you load them with Calc.

Microsoft PowerPoint --> Google Slides

This is the best option for me. LibreOffice version of presentation software does not cut it for me. You can use Google Slides with your google account.

EMAIL [Microsoft Outlook --> Thunderbird Mail]

Thunderbird Mail will come pre-packaged with your Linux 🐧 installation and is a great replacement option. It can do many practical things like tagging your emails, creating subfolders for emails and synchronizing all changes thereof with your email account. It can also be used to create custom signatures to your email accounts.

Mounting Drives / Network Drives

Not only is it very possible, it can also be faster than doing it with Windows. You just need to Google a bit
https://askubuntu.com/questions/46183/how-to-map-a-network-drive

Video Games

This is where Ubuntu may not be so great. However, if you like Steam games you may be in luck. A Steam app can be installed with Ubuntu and some Steam games can be run through an emulator/adapter such as Proton which will basically give comparable gaming quality as playing Steam from Windows. I have even been able to make it possible to run Minecraft through Steam on my Ubuntu system

Be aware that this may not apply to all Steam games, so thread with caution.

Conclusions

There is much more subject matter to cover than what I've given here. I may or may not make a sequel to this post later on. Nonetheless, I think this was a pretty good overview of what Ubuntu has to offer with Windows as a reference point. And I hope it makes you more confident on making the switch if you need to [or want to]

You have to let it all go, Neo. Fear, doubt, and disbelief.
😎

-Andrew

A very simple tutorial of the Crystal programming language

Andrew Garcia — Thu, 05 Nov 2020 22:08:01 +0000

Want to learn a pretty new, cool and easy-to-use programming language? Crystal is a compiled language with the syntax ease of ruby and computational efficiency of C. Yes that's right, C.

Installation

right click "Open Link in New Tab" here

For Ubuntu:

On a Terminal, Copy-paste the following (3) lines:

curl -sL "https://keybase.io/crystal/pgp_keys.asc" | sudo apt-key add -
echo "deb https://dist.crystal-lang.org/apt crystal main" | sudo tee /etc/apt/sources.list.d/crystal.list
sudo apt-get update

This will declare the files to your apt install. Then type:

sudo apt install crystal

Also copy-paste these to install additional dependencies that you will most likely need later on to build projects:

sudo apt install libssl-dev      # for using OpenSSL
sudo apt install libxml2-dev     # for using XML
sudo apt install libyaml-dev     # for using YAML
sudo apt install libgmp-dev      # for using Big numbers
sudo apt install libz-dev        # for using crystal play

To upgrade is as simple as writing these 2 lines:

sudo apt update
sudo apt install crystal

Hello World

open a Notepad or Text Editor and Type the following in it:

puts "Hello World!"

Save the file / give it a name. You can name it helloworld.cr .cr is the extension for crystal.

Open a Terminal on the same path where you have helloworld.cr saved. If on Ubuntu and your helloworld.cr file is on your Desktop, your path would be ~/Desktop

Then while on such path type:

crystal run helloworld.cr

This shall print Hello World!

Make a project which uses shards!

This is an example I've developed. You may fork my Github repository which shows the most updated version of this example if found useful for learning purposes.

There are basically 2 types of projects you can create with Crystal: an application (app) or a library (lib). We will make an app. For that, go to the folder or loc where you want to create the project and on a Terminal type the following:

crystal init app firstapp

This should basically create a folder with all required files for your project. Open the created firstapp folder and you'll find something like this:

We are going to use num.cr as the example shard. Num is a core shard used for scientific computing in Crystal

If you had already clicked on num.cr shard links, you'll see that you need to add the following lines to your shard.yml file to make it work:

dependencies:
   num:
      github: crystal-data/num.cr

Do it and save:

So now you need to install those declared dependencies to your project. Simply cd to your project on the terminal (cd firstapp) and then while inside firstapp, type:

shards install

If you get a red warning, you may need to install clang:

sudo apt install clang

Now go to the src folder and open the .cr file inside it.
In the first line of the firstapp.cr file you're gonna type the following:

require "num"

This calls/imports the num.cr shard which you've declared in the shards.yml file and built with the shards install command.

After that, you can add to the firstapp.cr file some interesting matrix objects and operations, and a lot more which the num.cr shard has to offer by going over its well-written documentation.

Happy coding 🦄 🌈

-Andrew

DEV Community: Andrew Garcia

GPT-4 Literally Doubles Your Learning Speed ... (According to GPT-4)

Advantages of Using GPT-4

Speed of Progress with GPT-4

Example Scenarios

Estimating Time Savings

Quantifying the Impact of GPT-4

Normalized Experience Gains

Raw Estimates (Normalized)

Conclusion

I Wrote a Book on Publishing Python Packages: Here's Why It Matters

The Idea

Why Publishing Python Packages Matters

Bridging the Knowledge Gap

Empowering Machine Learning Development

Encouraging Open Source Contributions

Professional Growth and Recognition

Why This Program Stands Out

Comprehensive Learning for All Levels

Bilingual and Practical

Practical Tools for Immediate Application

Optional Personalized Coaching

Inside the Program

Conclusion

Recursion

Tensor performance benchmarks for Python and C++

What are tensors in computing?

Tensor performance: axes swaps

Git Cheat Codes: TLDR useful Git commands

Project Dev. Sequence

Create a Repository

Locally

Easier way

Branches

Basic commands

Rename branch

Add all content from a fork to a larger repository

Undoing Stuff

Revert modified files not yet staged for commit

Undo a commit

Temporarily revert to previous commit

Remove files listed on .gitignore which have been previously committed (i.e. "not gitignored")

Make Git Remember Your Username and Password

Adapt SSH key to repositories

Find all folders which are linked to Github repositories in a specific folder (Linux users)

Running computational tasks indefinitely on the cloud

Outline

Sign-up to DigitalOcean

Python users: Installing Anaconda python on droplet

Installing Git on droplet

Transferring files between droplet and computer

Use screen to run process on droplet even after turning off your computer or exiting ssh

How to take a snapshot of created droplet

How to install TensorFlow on droplet

Conclusion

A reduced explanation on neural networks

Images

My transition and journey from a Windows operating system to Linux (Ubuntu)

What is Ubuntu ?

My Windows --> Ubuntu Switch

Dual-boot (Windows / Ubuntu hybrid computer) installation

Full Ubuntu installation

Microsoft Office --> (many; see below)

Microsoft Word --> TeX / LaTeX

Microsoft Excel --> LibreOffice Calc

Microsoft PowerPoint --> Google Slides

EMAIL [Microsoft Outlook --> Thunderbird Mail]

Mounting Drives / Network Drives

Video Games

Conclusions

A very simple tutorial of the Crystal programming language

Installation

For Ubuntu:

To upgrade is as simple as writing these 2 lines:

Hello World

Make a project which uses shards!

Remove files listed on `.gitignore` which have been previously committed (i.e. "not gitignored")