DEV Community: Jevon

Teaching Yourself to Code (VII)

Jevon — Thu, 11 Apr 2019 05:44:12 +0000

Pandas is rather outstanding! After around 18 hours of exerted efforts, the annual budget chart will now automatically update any new amounts. With respect to the spending about the type of purchase made

or how many of the type of purchase was made.

As well as with respect to the spending at the merchant visited

or how many times a merchant held the highest spending per data point

or even the type of merchant

All while allowing for analysis of spending amounts via linear regression

Now all that would be left is to tie up a few loose ends, like adding the geographical coordinate vectors so the geolocation isn't just a floating accessory, and then I can move on to working on packaging.

And all of this starting from the data in a spreadsheet.

With a little help from Google's APIs, the documentation, and a fantastic slack community.

Teaching Yourself to Code (VI)

Jevon — Wed, 10 Apr 2019 13:01:43 +0000

Update.

Just a quick update, mainly to keep track of where I am with this and to get a feel for what other areas might be useful.

Added a portion for users to gain information on stores around them within a 3.4km (~2.1mi) radius.

Turns out, there's a package called geocoder which seems to locate the user's IP and returns their geographical coordinates. Which was super handy for the above functions by passing each coordinate value

Two main drawbacks here: it doesn't seem to work very well in a mall. That is, the majority of the stores it located were outside of the mall where I was testing it. It's possible it has something to do with how the stores within the mall itself are set up, maybe inside of a specific directory so not immediately listed as a store but the mall itself, or it could just be a bug that I will have to fix later.

Plus, as I've recently found out, if one of the authentication servers goes down, the entire analytics system goes down; which, in turn, actually is not a euphemism for refreshing my API keys. Instead, it means I'll have to either find a solution that's more reliable than Google in order to keep the application up and running, thereby keeping the users happy, or find some means of hosting things independently.

The primary drawback to hosting things independently and not using an open source like Google means that my code would probably then become proprietary and I'd be unable to share any major updates.

Now, the real question that I still have yet to answer remains: How is this useful for finding stores around the area where the user has spent the most money and are thus likely to spend again?

Along those lines, do horizontal bars look better than vertical ones?

compared to

Teaching Yourself To Code (V)

Jevon — Tue, 09 Apr 2019 15:11:05 +0000

Personal project updates: To those who don't already know or follow my GitHub projects, I'm making a budget program that's directly employing specific methods to determine your overall budget and possible spending increases, or decreases. Or, if you prefer buzzwords, I'm using machine learning algorithms.

Nevertheless, thanks to the existing APIs and their amazing functionality, I can perform the bulk of the actual calculations and progression outside of the program itself and have the resulting data imported for the analysis. Which, in my opinion, is much handier than performing both calculations and analysis at runtime.

From just some data in a spreadsheet,

The information is then imported and analysed by purchase to develop a moving regression line.

That is, both the range and the regression update automatically with each new non-zero data point. Granted, this could probably make it interesting if you don't spend anything over the course of a few days, however, that would also mean that your spending would at least not have increased over those few days.

Beyond that, the expenditures can be grouped by category of the store visited,

by category of the item purchased,

or by the name of the store itself.

Granted, none of the above reveals how much was spent at each merchant as a singular dollar value; just determining the comparison of the amounts being spent. Clearly, there are stores in existence which are not on this list. The cause of that would be because I either don't yet know about them or I have never been to these places.

However, that by no means dictates that anyone should feel neglected due to my lack of information. Simply perform a quick google search
and review the top 5 results to see the website of the merchant of interest.

Thus far, all of these items have been made using the freely available APIs provided by Google via the developer console.

There are other features available, such as using GooglePlaces to load a map and locate places within a default 3.2km radius, which can be altered to anywhere between 0 and 5km, however, I've currently exceeded my quota during the testing period. I will update this post once it resets.

Update:
Returning a list of nearby restaurants

And returning a list of nearby cafes.

Including duplicates for places with multiple locations.

Teaching Yourself to Code (IV)

Jevon — Sat, 06 Apr 2019 22:51:22 +0000

Today I learned a very valuable lesson; reading the tests in a suite will often prevent much heartache. I say that for a few reasons. While my primary (and thus actually preferred) programming language is Matlab (totally should be read as "Octave"), I often tell people that it's Python. Why would I do this? That is because, as I mentioned in my previous post, the population of people who understand that language is sparse and I'd rather not have to deal with the slightly awkward expressions offered upon hearing this. Why are the subsequent expressions awkward? I don't actually know. I could make guesses, but since I'm unlikely to receive a truthful answer, then a "guess" is all they would amount to, at best. In other words, without plopping someone into an environment and evaluating their emotional state (how would that be done exactly?) as they go through the language, it's almost impossible to verify or place any validity in an answer received, outside of "I understand it" or "I do not understand it". That being said, it's often easier to choose the lesser of two evils and choose a language with which the audience is at least somewhat familiar.

What does this mean on my end? It means that, for everything I've ever written in Octave/Matlab (I'm going to be using the names interchangeably henceforth due to the extensive similarity in their code), I will be rewriting an edition in Python. Does this mean that I know Python and consider myself an expert? Not in the least, however since Python seems to be the official language of the sciences, and it's used by one of my all-time favourite entities, CERN, then I'm certain that Python can handle almost anything that I throw in its way. The downside is that this also means a lot of reformatting and that there may be some computational errors along the way. Probably induced by the data structures themselves but I don't actually know how Matlab stores matrices, much less how it allocates memory for these things. I mean, I know it has data structures, but I'm pretty sure it's unique in its processing, since it handles the mathematical computations exceedingly well.

Conversely, I'm honestly having trouble getting the same function to work well in Python.
Like, it ends up being a much smaller amount of code but these suggest that something is going to have to be handled differently if I want to maintain functionality even in this context. Granted, the Python edition doesn't have the surface plot encoded or even the contour plot, but if it is honestly having trouble with the initial portion of the descent then I probably shouldn't worry about either of those things as yet.

Nevertheless, I'm again proud of this little accomplishment of getting my budget written over and working with some sample data.
In this case, the linear regression plot works as well, but I ended up having to use numpy's polyfit feature instead of gradient descent like I wanted to. Either that or severely limit the range being used in the analysis...but that honestly seems a bit contradictory since gradient descent is hailed as a machine learning algorithm designed to find the local (or global) optima and I feel that if the range becomes too small then its ability to do that would be heavily stifled.

I suppose, therein lies the challenge; finding the happy medium satisfying a sufficient range and a runtime that doesn't last a lifetime. Either that or find something else that performs much the same task. Nonetheless, I'm still pleased with even this minor bit of progress.

Teaching Yourself to Code (III)

Jevon — Sat, 06 Apr 2019 13:38:12 +0000

So, as the tone of this journey undergoes a metamorphosis, we will hold a discussion on object-oriented programming and function declarations. However, since this isn't an overview on specifically how to do any of this, then that portion of the post will be brief; although I will still link any resources I've used or found useful in my educational exploration. That is to say, across both languages of Python and C#, you can easily declare classes, constructors, and objects. Tutorials Point actually has a very nice, and what I consider to be a comprehensive, overview of how to do exactly these things in Python; titled Python - Object Oriented. As for functions and their declarations, Tutorials Point also has a section on that as well, titled Python - Functions.

Now then, as for how this particularly ties together with Clean Code and the previous post; since, very clearly, there wasn't any mention of coding practices in that one. That connection would be the subject of meaningful names. Whether in the form of declaring variables, arrays, lists, functions, objects, classes, et cetera, the name of the declared item should be sufficiently explanatory of what that item represents. That is to say, "If a name requires a comment, then the name does not reveal the intent."

Now, let's look at an example from some code with which I've been working. In particular, a data frame. Which is like the Python version of a temporarily held database object and they're usually prefixed by "df_". So, suppose you have a name such as "df_HR". While it may be possible to infer the meaning of such a name given the context of the data being analyzed, it slightly violates the tenet of "literate" mentioned in my first post. This is so since it makes the assumption that the person reading the code after the fact is inherently familiar with the subject matter or the initial data set. Now, I will admit that I did take some liberties and chose "df_humRes", but even that wouldn't hurt to be written fully as "df_humanResources" or "df_human_resources" if I wasn't being lazy or writing a template I intend to change at a later date. So, if you're keeping up with my GitHub projects, then yes, I'm going to admit now that I'll probably change that for legibility in a later refactoring event.

Moreover, within your descriptive nomenclature, it is highly important to avoid misinformation. That is, while it is imperative to avoid leaving false clues and obscuring your code's intent, it is equivalently imperative to be mindful of spelling. To state it explicitly, "Spelling similar concepts similarly is information", then anything else, including but limited to inconsistent spellings however unintentional, is by definition misinformation. Thankfully, most IDEs now have code completion features which help to mitigate these unintentional misspellings.

Personally, I've become a fan of Droid Edit, Atom, VSCode, PyDroid 3, Spyder, and most recently VIM. Now, I'm sure you've noticed that these aren't all on a personal computer, mac, or laptop; for a while, I had to write code on mobile so I needed an editor for that too.

That being said, I often find myself writing these things as if they were an essay or otherwise a really long paper. In the sense that, there are often several drafts before a final copy is made. In the mathematics related coursework, the ones that have direct ties to any programming languages, there is often a disconnect between writing something to be used by many and writing something to find a solution so you can move on to the next stage. Much the same is true for many of the other hard science courses; physics, chemistry. While biology is a wet science, I haven't done much computational work in that area (read: any computational work) so I am loathed to comment on their practices. That is, much of the "programs" aren't what programmers would call "programs", or "production ready". Therefore, throughout the learning process of the students in these majors, although they may gain a deep understanding and respect of the scientific material and thus the world around them as it relates to their subject area, they tend to complete undergraduate study and find out that either they are exceedingly specialized in a language only used in the scientific community or find themselves ostracized from their assumed peer intellectuals in the workforce. You know, unless said peers were also in the same area of study.

I would say that this has hindered my growth a bit and stunted my progress in the way of the self-taught developer. As an example, much in the same way that there are few who understand or use Matlab in the industry (that I've encountered), there is a lot that the computer science students have studied and gone through that, as one who performed active research in chemistry and mathematics, I simply did not encounter. So as much as their struggles are foreign to me, mine are equivalently foreign. Doesn't help that there's an attached stigma along the lines of "everything relates to math". Ignoring that things that are "based on math" or "based on calculus" often have long since lost their meanings and hold a stronger relationship with their directly computater-related counterpart than with the parent science.

Then, in the same research experiences, although I wrote things to accomplish the tasks I desired, comparatively they were little more than scripts to aggregate and analyze data to a readable form rather than an outright program intended for mass consumption. That is, the level of work output for writing something to be used on information produced by an atomic force microscope, ignoring all work output towards the experiment, the analysis, the aggregation of results, the calibration, the presentation, or anything involved with the inorganic chemistry itself, is much less. Since writing the script is just something that was done because a minor portion of the overall experiment was hindering progress in the larger picture. Much the same goes for mathematics as well. That is, as it relates to differential equations, algorithms, or even machine learning, if we ignore any of the analytical work done to first find some calculable relationship between species of interest, if we ignore all of the time spent in the initial stages of the foundation, then the writing of the scripts is far less intense than it would be were you to solely spend time on writing something that would be overall beneficial to the larger populace.

Honestly, though, this should be no surprise. If you consider the overall population of a university, then the sheer number of students pursuing any of the hard sciences as their explicit area of study is almost inconsequential. It might even be considered a highly egregious sin to make the request that others come to understand you, subsequently anything about you, before making any assessments when your own experiences are inconsequential by population. If indeed you should choose to consider.

Consequently, it is then little to no surprise that an undergraduate has to make a decision to either forfeit the years of struggle and self-sacrifice or to return to academia from whence they came. Where the options become increasingly limited to either disappearing almost permanently from public view unless necessary or returning to academia may yet mean to never leave the halls; scarcely to be seen attending a non-mandatory social function outside of a laboratory, library, classroom, or their office. Either case dictates a scarcity of presence, only punctuated by the minor social circle with which one would choose to be surrounded. Which, in many cases, I have found that many humans are either intimidated by things they don't understand or would rather expend the energy to believe such things do not exist; conformity be damned. Which, again, should be no surprise given our collectively colourful histories.

That being said, I would choose neither to forfeit nor to ignore my years of sacrifice and instead to make the effort to build on them. That is, throughout the years in pursuit of the hard sciences, there are many soft skills which are cultivated; not unlike those taught in a martial arts dojo. An actual dojo, not one that just teaches you to beat people up or how to become a talented bully. Skills such as perseverance, the value of hard work, discipline, self-confidence, and, most importantly, how to learn. Yes, fellow humans, I would argue that learning is a developed skill which can be cultivated but cannot be taught. I would choose to build on these skills and use them to spur explosive growth.

Now, neither will I say that I am some savant who masters everything in a day nor that I am a saint. However, I will say that I will diligently study as I learn my limits and exceed them in each regard as I am able. Grey hairs gained purely from biological age don't have very interesting stories.

From Weak to Proficient

Jevon — Sat, 06 Apr 2019 04:26:38 +0000

Job hunting is difficult. Almost dishearteningly difficult. As an example, one day, an interviewer might say that I should learn a language for their prospective company, despite my protests of pointing out that I am at best a junior developer, and thus would need to spend some time adapting, or at worst have done an undergraduate tutorial course in it. Inevitably, because I'm still actively attempting to start my career in code, I agree to their request. So I study and subsequently do their assessment, all the while remaining aware of my actual level; wary of overconfidence. Predictably, imagine my lack of surprise upon the response that they would rather have someone with some experience working on a production level application in that language.

Nevertheless, I still say to keep chugging and keep working with the idea that these sleepless nights will one day pay off. In my case, things are probably going to get even worse before they get any better; recently-ish left my place of employment due to certain conflicts of interest. I'd rather not go into detail but for those who know me in real life, I thank you for your continued support and encouragement. I would like to say that you guys have brought me a long way, I tell you as much fairly continuously, but I'm also going to take some credit for the effort output.

A special thank you goes to one person in particular who believes in me and has continued faith in my progress. If you ever read this, < super emotional words here> ...sorry, I still can't bring myself to do that on even a public platform.

Enough digression. All of that being said, one thing that these continuous examinations have allowed me to see is my own growth in familiarity with a variety of languages; even if that involves spending a day learning something I've only seen in passing (or seen many years in the past). These things, these small victories are what reminds me that I started doing this for a reason. Even though many times, small victories are the only ones to be had.

In these cases, I like to remind myself of one little fact; in general, life is a sequence of these things. Whether or not they happen in succession or are sparse in their distribution, it becomes increasingly important to acknowledge when they happen; lest we become overwhelmed.

So even though I've not yet signed anything suggesting permanent employment, even though companies seem to all want formally trained developers or otherwise see an aspect of "computer science" on a resume, even though the population of people who use/understand Matlab or Octave is almost infinitesimal by comparison, I will accept that I've at least increased my level from when this all started not even two months ago.

If you're still on this journey with me, then you as well keep studying, keep growing, and do not forget why we began. Progress may be slow in comparison, but it's still progress.

Teaching Yourself to Code (II)

Jevon — Tue, 26 Mar 2019 02:07:46 +0000

Before we begin, let's take a quick review of some of the basics. As we go along, will list a few things for C# as they may or may not relate to Python accordingly. In this way, I hope to build a functional relationship between classes, objects, methods, variables, and other features of the languages.

Now, you may say that this would actually be quite difficult since python and C# are fundamentally different, and I would normally be inclined to agree (showing my inexperience). However, if we first assume that the line, "It is the programmer that makes the [language] seem simple" to be true, then surely there would be at least relatable topics/areas/ideas between the two languages. Thus, by induction, we may extend that idea of relatability to any other two language comparisons. In this case, we will hold the language of Python as constant, and perform comparisons thereof.

That being said, things like variables and variable assignments are much the same across the two languages. This may simply be a product of how I'm progressing through learning this language (SoloLearn is wonderful for quick tutorials), however the differences I began to note started at the commands and assignment operators. Notably, whereas in C#, and a few other languages, making a statement like "x++" is fine, such shorthand is unacceptable in Python; you must make a sentence such as "x+=1" or "x=x+1" even for unit incrementation.
Else, much remains the same. That is to say, even though the commands are clearly different, Console.Writeline("This is a string"); versus print("This is a string"), there are certain similarities. Before I continue, I do want to take a moment to observe Python's lack of semicolon. Yes. As I was saying, the structure of the languages is largely similar. Although there are certain differences.

Examples being, in both languages it is common to see things such as "for" and "while", it would be highly uncommon (and ineffective) to see "do { } while { }" or "foreach" sections in Python; it has no need of them. To implement these, it becomes a matter of "while: ... if: ..." or "for ... in ..." instead. Equivalently considerable, Python doesn't have a default "switch" case comparison. To accomplish this, you might have to employ a dictionary, where we would map each section to their respective functions, or you could define a particular class (or classes) where the specific method gets chosen at runtime. The following examples were initially listed on data-flair.

Using a dictionary:

Which, if I request the information to be printed,

Results in:

Then, using classes:

Which, when printed,

Gives us a similar result:

Two things I'd like to make a quick note of here, I use a blank " " to insert a space, purely for output formatting purposes, and these would result in the same if you were to make the function calls from the command line instead.

Additionally, the simple case use of conditions is largely the same; x > 0, y <= a, even the logical operators such as "x==y", "x!=y", "x&&y", "x||y", are relatively interchangeable. However, for Python, many times these logical operations can be written literally as "x and y" or "x or y", although in certain cases, a few remain the same, such as "x==y", "x!=y". As for how to interpret the logical operations, the traditional arguments made of any boolean statements remain. That is to say, observe the following:

"x and y" is True if both x and y are true.
"x or y" is True if either x or y is true.
"x and y" is False if either x or y is false.
"x != y", along with "x == y", when appropriately read as "x not equal to y", "x exactly equal to y" respectively, retain their logical truthfulness as you'd naturally expect.

Similarly, for some constant a, or for some function g; where g can be evaluated to some constant value; any comparative statement made, "x > a", "y < g", follow the expectations of value comparison that you'd expect.

Teaching Yourself to Code (I)

Jevon — Sun, 24 Mar 2019 03:49:20 +0000

Chronicling the things I've learned while teaching myself to do various things in life has shown me that sharing the information with others is a good way to avoid information overload and serves as a great way to order thoughts and notes on the subject matters at hand. Therefore, take a walk with me and see some of the things I encounter as I become a full-fledged software data scientist and software engineer. The first step, self-assigned given my lack of knowledge and skill in coding, is to become a software developer. Gotta walk before you can run, right?

Over the past few months, I've begun to teach myself to become a developer. Although, "re-teach" might be a more applicable term.

To give a bit of background on myself, this is something I'd once started in high school over a decade ago, however, due to certain circumstances, I ended up pursuing scientific research in college instead of continuing with the development aspects. Therefore, my formal training is as an applied mathematician, however with what I feel is an undergraduate level of mastery.

Interestingly enough, it was a combination of this same scientific research, and study of differential equations, which re-ignited my love of programming and finding software solutions to complex problems.

That being said, to make myself into an even better programmer, I've decided to chronicle the lessons I learn. More or less as I learn them. There will be drama, intrigue, soul-crushing despair, and utter euphoric bliss, as we take this journey. There is no specific instructor, no designated teacher, for this material…as much as it'd be more convenient for me if there were. There are, however, senior developers, books, friends, tutorials, guides, online classes/videos, and various projects, which teach me everything that I know. Yes, teach; either directly or indirectly on a case by case basis. To these resources, I have immense gratitude; I would only hope to be as much of a resource to those for whom I would appear as a pioneer.

While this may seem a frivolous post, it is primarily to show my gratitude to those who believe in and support me on my journey. So, if any of you happen to see this:

THANK YOU!!

Okay, so onto the more interesting materials. The first topic I'd like to discuss is clean code. This is something which will be revisited on various occasions and, as such, my view is likely to change as I develop. That is to say, by no means should you or anyone else take these words as gospel; I strongly encourage you to explore various methods and develop your code-sense as it's very clearly something I'm developing as well.

The book I'm using to develop this is Clean Code by Robert C. Martin. Although I may post excerpts, I suggest that you to read through this book on your own at your leisure. I am thus far loving this book because the first thing he does is state:

"You are reading this book for two reasons. First, you are a programmer. Second, you want to be a better programmer."

I would argue that you are undertaking this journey with me for the same reason.

Nevertheless, I find this part to be highly important since it's all too easy to doubt ourselves and our own capabilities. As much as I am equivalently guilty, I'd encourage you not to fall into this trap of self-doubt; it is disrespectful to those who back you as much as it is disrespectful to the hours of hard work you invest.

Now, I'm sure we've all had the fantasy of having our own edition of Jarvis; or some other such intelligent machine which understood our intents to their core and would accomplish what we wanted instead of being a veritable monkey's paw. However, we will never escape the need to be absolutely specific in our details; our requests. This is true with our interactions with other humans, and even more so does it remain true with our interactions with machines. While I'm not going to pretend to know much about humans, or their desired escape sequences, perception is not reality. Reality is reality. Since at some level there are details which cannot be ignored or abstracted, then they have to be specified. In this case, specifying those requirements in such detail that they can be executed by a machine is, in and of itself, programming.

Now, even though I say that, just like there are horrible specifications, terrible instructions, and numerous other things which may fall under that umbrella, you can always assume that there will inevitably be bad code. At some point in our lives. Most likely, if you're staring at a program you wrote and are at least as inexperienced as I am, then you are also guilty. However, you shouldn't let bad code bog you down. Don't become significantly impeded by it. That is, don't try to go fast, don't try to rush. There's a saying I once encountered regarding coding:

Code can fall into a few categories; good, fast, or cheap. Choose two.

"Programmers face a conundrum of basic values. All developers with more than a few years' experience know that previous messes slow them down...In short, they don't take the time to go fast! ...The only way to make the deadline -- the only way to go fast -- is to keep the code as clean as possible at all times."

Writing such code takes discipline. Discipline and a developed code-sense. I'm still working on my code-sense, so I have no advice to offer there. But hey, we'll learn together. That said, clean code does one thing well. That is, a one line function is better than a multi-line function.

Let's think about mathematics for a moment. Yes, we'll be using math and chemistry a lot as reference points, but don't worry, I'll keep the examples superficial. Take the slope-intercept form of a line; y = mx + b. While we all know that the slope, m, and the intercept, b, have various interpretations and methods of evaluation, imagine how messy it would be if all of them were to be included in that equation? Instead, we tend to find the "individual pieces" elsewhere and then inject them accordingly into this function to find the solution. Similarly, your code should be elegant. Each function, or method, should do one thing and do it well.

"Clean code is simple and direct."

Your code should be readable such that the original intent is never obscured. The author makes the allusion to reading a well written novel such that the words themselves seem to fall away and you are able to immerse yourself in the world the story creates; seeing images instead of walls of text. Similarly, your code should have a high level of readability and remain crisp.

Similarly, again relating to mathematics, I'm sure any viewers of the classic slope-intercept form of a line, y = mx + b, would never feel that the original author was indecisive about how that line should be fit. I would argue, in this instance, that you should not at all be afraid to delete, edit, and re-edit, your code as your write to accomplish this aspect of cleanliness.

"Clean code can be read and enhanced by a developer other than its original author."

While this seems fairly straightforward, its implications are of a very high degree. Think about it for a moment. Your code should be readily able to be edited by someone other than yourself. That is to say, well written code, clean code, is literate. To that end,

Clean code has "reduced duplication, high expressiveness, and early building of simple abstractions."

Now, something about that which is implicitly critical. Implying that clean code is also minimal. Almost as if to say that it only speaks when necessary; not verbose. One of the talks I was watching (on YouTube of course) made reference along the lines of, "a function that takes no arguments is better than a function that takes one argument". Although his language was Ruby, so take it for what it's worth. If your language doesn't allow you to do that then don't force the issue. Conversely, if you're able to pass values inside your function itself by means of a reference, a pointer, or result stored in a variable, then that may be a better option. Most of the time.

Uncle Bob (the author of Clean Code gave this self reference) goes on to state, "It's not the language that makes programs appear simple. It is the programmer that makes the language appear simple!"

Then, "...if you want your code to be easy to write, make it easy to read."

Thus far, there have been a few aspects of clean code which have been explicitly stated. Clean code is:

Elegant

Crisp

Literate

Now, it's important to remember that this is simply one school of thought. There are others, and mine may change as I continue to grow. "There will be lots of code. You'll see good code, you'll see bad code. You'll see bad code transformed into good code... You'll see example after example. After that, it's up to you."

There was once a concert violinist who got lost on his way to a performance. He stopped an old man on the corner and asked him how to get to Carnegie Hall. The old man looked at the violinist and the violin tucked under his arm, and said: "Practice, son. Practice!"