DEV Community: John

Rails Portfolio Project Flatiron School (mod3 project)!

John — Sat, 19 Sep 2020 14:53:27 +0000

Rails contains alot of magic, it provides all kinds of generators that can make live easier. As part of our project we were told to stay away from the scaffold generator. The command ‘rails new ordersystem’ already creates a big part of our project by creating the entire filestructure.
What I learned is to be aware what version it will use during creating a new app. Two days in my project I discovered my project was using Rails 6.0.1. Our cohort lead has advised to switch to postgresql in stead of sqlite. During a live lecture he lead us through the setup and it was working without trouble on my side. Needless to say when re-initiating my project with Rails 5.2.3 I forgot to complete the ‘rails new’ command with the database option to use postgress. when I noticed that and with the break-week available I decided to start again with the correct Rails version and a Postgres database.

Different than in Sinatra, and maybe lacking in the magic, is that although having the option to configure the database automatically we have to run ‘rails db:create’ to create it.

The reason for the ‘stay away from the scaffold generator’ is that for a single model project it will generate literally everything for you to get things up and running very quick. Only the big question will be if the stuff generated is to your liking. Most likely you want things to be different and change things. Also the question will be, if you have to use generators like scaffold do you really learn how rails works.

That still leaves us with a lot more but we will use the following generators:

model
resource
controller
migration

The model generator is the simplest one. When used it will generate a database migration file and a ruby model.
rails generate User name username email
Will create:
a migration in db/migrate/20200710934260_create_users.rb

class CreateUsers < ActiveRecord::Migration[5.2]
  def change
    create_table :users do |t|
      t.string :name
      t.string :username
      t.string :email
      t.timestamps
    end
  end
end

the User model in app/models/user.rb

class User < ApplicationRecord
end

In addition of this the resource generator will generate an entry in the /config/routes.db file to all the restful routes in the UserController that will also be generated.

The controller generator will generate the controller, the views folder, the helper for the controller and two files in the assets folder for javascript and stylesheet. If you add an action to the end of the generator it will also create the view file for it.

The migration generator can be used to create a migration file either for the initial setup (create) of the table or to edit the content of the table (eg. rename column, add column)

All this creates the magic of Rails, more or less after setting up your application with this you only have to add some logic to your controller methods and you are up and running. It is a bit simplistic to state it like this as it still took me about a week to complete my project, but in big lines it is that simple.

Sinatra Portfolio Project Flatiron School (mod2 project)!

John — Sat, 19 Sep 2020 14:48:07 +0000

Sinatra Final Portfolio Project

Time went quick for me in mod 2. Started as a part timer and ended up being a full timer. This meant that the week following on my change I had to start thinking about a project as project week was starting right away.

I chose to make a HamLogbook, as Ham Radio is one of my hobbies and I thought would give plenty of room to apply the things we learned about Sinatra.

After learning the basics in mod 1 we took a dive into Sinatra. Although Sinatra takes away a lot of programming of what we have learned in mod 1, it brings some interesting features with the additional methods. We learned about ActiveRecord, the Routes in Sinatra, the MVC structure of Modules, Views and Controllers, Sinatra and ActiveRecord and User Authentication. Almost too much in such a short time, but we managed to get a grasp of it, which we now can display in our Portfolio Project.

One of the interesting introduced gems is bcrypt, in combination with the ActiveRecord macro has_secure_password. The macro has_secure_password writes methods for us to set and authenticate against a Bcrypt password. The mechanism requires that we have a (attribute)_digest as attribute name for the desired password. Additionally it adds a requirement that a password is present the moment our User Object is created. It also checks to see that the password length is 72 bytes or less, and offers an optional password_confirmation attribute for password confirmation.

Bcrypt itself is a hashing mechanism that takes a digital fingerprint of a password. There is no way back, from the hash you cannot regenerate the password. Hackers however can create a database of hashed passwords using the bcrypt algorithm. If a database was compromised, hackers could do a reverse lookup of the hash to find the password. To make it more difficult for hackers a random chunk of data (salt) can be added to the password before it is hashed. This would make the database hackers would need to use to generate a possible salt and password hash extremely big in storage space and is most likely not done. So what is left for them in a plain dictionary attack including a possible salt. Herein lies the strength of bcrypt, as the algorithm is relatively slow compared with other algorithms like md5. It takes much longer to hash a password and thus you are slowing down the hacker guessing the password.

Bcrypt works for us in the background providing us with a password= method that generates the encrypted password based on the given plaintext password and stores it in the password_digest.

Creating a new User instance without supplying a password:

 pry(main)> user = User.new(username: "john", password: "")
=> #<User:0x00007f80525718c0
 id: nil,
 username: "john",
 password_digest: nil,
 name: nil,
 email: nil>
 pry(main)> user.save
=> false

Although in the command line we are able to create a new instance of a User Object, we are not able to save it. As User.create has the .save method build in, this will fail too.

pry(main)> user=User.create(username: "john", password: "")
=> #<User:0x00007f80525db248
 id: nil,
 username: "john",
 password_digest: nil,
 name: nil,
 email: nil>
 pry(main)> User.last
=> #<User:0x00007f8052618f80
 id: 4,
 username: "af5se",
 password_digest:
  "$2a$12$Qbg0kWJQSzdBcdM1wCc4Tefe4U4wAFEL/ADRdFOL1MUleIYXt/Mna",
 name: "John",
 email: "">
 pry(main)>

As shown in the above section, the newly created User Object is not persisted to the database. In other words the .save method included in the .create method failed because a password was not present If a password is supplied the create method will show the created object with an id value that is not nil. From this we see that the object has been persisted succesful into the database.

pry(main)> user=User.create(username: "alex", password: "1234")
=> #<User:0x00007f8052318f88
 id: 8,
 username: "alex",
 password_digest:
  "$2a$12$5GqNWDrtxQMnsmnmixU0N.WiflNh0BX/DMarA3JUlz0ya9bF5GUuS",
 name: nil,
 email: nil>
[13] pry(main)>

When we add a password the User Object will save succesfully and will be persisted to the database.

pry(main)> user.password = '1234'
=> "1234"
 pry(main)> user.save
=> true
 pry(main)> User.last
=> #<User:0x00007f80554f54c0
 id: 7,
 username: "john",
 password_digest:
  "$2a$12$O4u602uMzjxy529e9VNX4.Oe73GbzNvjGRXVsXLagw6NySPldHVXa",
 name: nil,
 email: nil>
 pry(main)>

When we try to authenticate with the user with the correct password the User object will be returned.

pry(main)> user.authenticate('1234')
=> #<User:0x00007f80525db248
 id: 7,
 username: "john",
 password_digest:
  "$2a$12$O4u602uMzjxy529e9VNX4.Oe73GbzNvjGRXVsXLagw6NySPldHVXa",
 name: nil,
 email: nil>
 pry(main)>
With an wrong password we will see this.

pry(main)> user.authenticate('wrong password')
=> false
 pry(main)>

About my project, as mentioned above I decided to make HamLogbook. A place where you can store you contacts made with HAM radio. I created three tables in my database, users, callsigns and contacts. With the logic that a user can have many callsigns and can have many contacts through callsigns. A callsign belongs to a user and a callsign can have many contacts. And at last the contacts belong to a callsign.

Normally a user would have enough with 1 call sign. A callsign is aplhabetically generated by the FCC and stays with the user as long as his license is valid (plus a 2 year grace period), independent of its class (Technician, General or Extra). At this moment new callsigns given out consist of a combination of 2 letters, a regional number and 3 letters (mine in december 2017 was KG5WJQ). However it is possible to apply for a vanity callsign. Depending on the license class this could mean a much shorter call sign. That is something I did and obtained in 2018 K5GT which is also displayed in the webpages of HamLogbook. So in my case although my old callsign is no longer active I still have contacts that would be valid for a certificate logged with that callsign or combination of both.

This led in my HamLogbook to the decission that a user can have more than one callsign. This also created a challenge because the result of User.contacts would include all the contacts a user had made disregarding what callsign was used. So I had to find a way to pass the choice of callsign I wanted to see the contacts for. I solved this by storing the chosen callsign in the session so I could obtain it with session[:callsign]. This sesion[:callsign] is set in the CallsignController and set to nill either when the user logs out or when the user visits his homepage.

Because of my three tables, users, callsigns and contacts I also have three models with the same names, to complete the MVC (Model Views Controller) strcuture I also created views and controllers belonging to each. The Model is where the object are defined, the views where the pages displaying the objects are rendered in erb files and controllers where the action takes places for the routing to the views and necessary data to be rendered. I created an extra SessionsController as that takes care of creating and logging in and out of the user and thus the session.

We are using CRUD, Create, Read, Update and Delete to manipulate our data we faced another challenge with our html form methods. Create and Read are not the problem because html forms have GET and POST. But for the UD part of our CRUD we need two more methods that are not implemented in the html forms. Lucky for use Rack came around the corner with the MethodOverride module which we can call for in our config.ru file with the line Rack::MethodOverride. This module gives us the option to add an extra input line in our html form where we use name="_method" and put the required post method in the value field value="patch" or value="delete".

For example:

<form method="post" action="/contacts/<%=@contact.id%>">
            <input id="hidden" type="hidden" name="_method" value="patch">

One thing that I did find out is that the place in the config.ru file does matter. Make sure to place it above your controllers, I found it out the hard way and it did cost me some time. All the other routes of my controllers did work except for the patch and delete. I only saw the POST in my traces and never a PATCH.

All together it was fun and I learned a lot. Project came sooner than expected, but it totally put me on a different track now being full time. Next project I’m probably better prepared and I’m looking forward to it.

Data CLI Gem Project Flatiron School (mod1 project)!

John — Sat, 19 Sep 2020 14:42:28 +0000

CLI Data Gem Portfolio Project

Looking at today’s date I realize that it is exactly a month ago that I started my Part-Time Software Engineering track at Flatiron School. Before that date all my programming was trial and error and googling for some hobby projects. But so far enjoying the adventure.

Now in the program for a month we came to the point that it is time for our first portfolio project. Where to start??

Luckily the project requirements were supplied and are:

Provide a CLI
Your CLI application must provide access to data from a web page.
The data provided must go at least one level deep.
Use good OO design patterns. You should be creating a collection of objects, not hashes, to store your data.

First Step

Where to begin and what to choose as a project. I decided pretty quick to use the hitlist of the Dutch Top40 listed on http://top40.nl Guess being dutch pushed me in that direction, it also helped that it contains a listing that could be scraped.

Second Step

Planning the gem was the next step, what Classes to use and how they would interact. I used http://draw.io to create an overview how I thougt my program would look like

http://github.com/raspimeteo/dutch_top40/blob/master/dutch_top40_flowchart.jpg

Third Step

In the project’s page posted YouTube clip from Avi he pointed out pretty well how to get started to build a gem and create a skeleton for your program. Doing this through Visual Studio Code was pretty simple with the given instructions. After running the ‘bundle gem dutch_top40’ command bundler creates a complete directory tree for the program. Left only to put the necessary requirements in place like ‘open-uri’ and nokogiri’ that are needed for the scraping. For debugging I also added pry to be able to see how things are working within the program.

Time to create a repository on github and push the project to github. Unfortunately just this week github was dealing with some outages that made github acces unreliable. But I managed to get things done.

Fourth Step

Time to put things in their place. As from the flowchart I needed to create three classes CLI, Songs, Scraper with their methods. I started out with the CLI class and used fake data and some simple puts commands to see if things interconnected were still outputting the expected output. After the CLI was working and gave me the options I neede it was time to start with the Songs class. Looking at the website of the top40 I saw the info I could obtain from one page. As the Songs class is actually the workhorse of the app I was a bit too focused on obtaining the data and by mistake placed also the scraper in the Songs class. The website has two options to obtain the list. The mainpage or one level deeper on http://top40.nl/top40. As the second page displayed the info on the site a bit clearer than the main page I tried to scrape the data from that one. All was working and found the tags to use, only to find out there was another column row that had the same tags and was causing an empty array element. This caused my post-scraping to fail. Needless to say this has cost me some time. After all I went back to the main page, meanwhile also used the repl.it Scraper Check Tool and found a better result on the main page.

After my scheduled meeting with our cohort-lead she also pointed me that I should put the scraper in its own Class.

For a nicer appearance I converted the top40 logo to ASCII to be able to use it as a logo in my CLI class.

class DutchTop40::CLI

    def call
        logo

        puts "One moment, acquiring data.",""
        list_songs
        # binding.pry
        menu
    end

    def list_songs
        @songs = DutchTop40::Songs.list
        print_songs
        puts
    end

    def print_songs
        puts "Dutch Top40 - week  #{Time.now.strftime("%U")}", ""
        @songs.each.with_index(1) do |song, index| 
            puts "#{index}.  #{song.title}"
        end
    end

    def menu       
            input = nil
            while input != 'exit'
                puts "Which song do you want more info on? Type list to see list again, type exit to quit.",""
                input = gets.strip.downcase
                case input.to_i
                    when 1..@songs.size
                        puts "------------------------------------------------------------------------------------"
                        puts "Current rank #{input}."
                        puts "------------------------------------------------------------------------------------"
                        puts "#{@songs[input.to_i-1].title} - performing artist(s): #{@songs[input.to_i-1].name}"
                        puts "weeks in Top40: #{@songs[input.to_i-1].listed} - last weeks rank: #{@songs[input.to_i-1].last_weeks_rank}"
                        puts "------------------------------------------------------------------------------------"
                    else 
                        puts "Invalid input!" unless input == 'exit' || input == 'list'
                    if input == 'list' 
                        print_songs
                    end
                    if input == 'exit' 
                        goodbye
                    end
                end
            end
    end

    def goodbye
        logo
        puts "See you next time..."
    end

    def logo
        puts "                                                                                                    
        ** logo  removed because the blog markup messed up the way it was displayed **"
    end
end

As you can see the CLI is pretty straightforward. The program is started and the bin file calls the call method with DutchTop40::CLI.new.call. In the call method the first action is a call to the logo method to display an ASCII representation of the top40-logo next a call to the list_songs method is made. This list_songs method in its turn calls the list method in the Songs class. After returning from there the returned array of objects is stored in @songs. Next is a call to the print_songs method. In the print_songs method the program iterates over the array and prints out a list of the song titles. Next line of code will call the menu method. The menu is dynamic although my expected output is always 40. But it simplified things not to have to program all 40 options for the song titles. At the end there is a check on invalid input which also gives room for the two options of the list and exit choice.

class DutchTop40::Songs

    attr_accessor :title, :name, :listed, :last_weeks_rank

    @@songs = []

    def initialize(title, name, listed, last_weeks_rank)
        @title = title
        @name = name
        @listed = listed
        @last_weeks_rank = last_weeks_rank
        @@songs << self
    end


    def self.list
        DutchTop40::Scraper.scrape_songs
        @@songs
    end

end

After removing the scraper from the Songs class it is slimmed down pretty much. Its basic duty now is when the call for the list method comes in to make a call to the Scraper class to do the actual scraping with the scrape_songs method.

The Scraper::scrape_songs method will call the initialize method to create new song objects with the supplied variables. The initialize method also stores the create song objects in @@song . After the scraping is finished the list method will return the @@songs array to the CLI class. In this class I needed the attribute accessors to create the methods to store the variables.

class DutchTop40::Scraper

    attr_accessor :title, :name, :listed, :last_weeks_rank

    def self.scrape_songs
        doc = Nokogiri::HTML(open("http://top40.nl"))
        doc.search('.listScroller .list-right').each do |song|
            # binding.pry
                title = song.css('.songtitle').text.strip
                name = song.css('.artist').text
                details = song.css('.details').text
                listed = details.split(' | ')[1].gsub(/Aantal weken: /,'').strip
                if details.split(' | ')[0].gsub(/Vorige week: #/,'').strip  == '-'
                    last_weeks_rank = 'new entry'
                else
                    last_weeks_rank = details.split(' | ')[0].gsub(/Vorige week: #/,'').strip
                end
                DutchTop40::Songs.new(title, name, listed, last_weeks_rank)
        end
    end
end

Scraping as described above was the most challenging part, but with the above listed code I was able to obtain all the info I needed. I could go one level deeper in scraping as the song info also contains a url. That url is leading to a page with some general information about the performing artist. But that would need a second scraping routine to obtain that information. On the bottom of the scrape_songs method you can see the call to DutchTop40::Songs.new with passing of the title, name, listed and last_weeks_rank. This will results that the Songs class creates an array of objects with all the objects that are created with this Songs.new command.

Final step

The final step was beautifying the CLI class so it would give a nice look to the interface and the way the provided info is presented. If you would like to see how it works you can either download the dutch_top40.gem on rubygems.org or download the repo at https://github.com/raspimeteo/dutch_top40

Road to Software Engineering Part-Time/Full-Time

John — Sat, 19 Sep 2020 14:34:13 +0000

Have to admit I’m not the youngest one on the block with my 56 years, but feel young at heart and mind.

In my spare time I always have been messing with either motorcycles or computers. Staying up late at night to fix something that wasn’t working. At home I was already playing around with Linux and SIP before the telecom world changed to VoIP. In that way I had a head start when finally both were introduced at work. During the last seven years, after my immigration to the US, I picked up interest in the SBCs like the BeagleBone Black and the Raspberry PI. At first mainly to run a small PaBX at home, later followed more to hobby around. Because of my HAM radio interest a lot of my projects had something to do with radio waves.

One of my first hobby projects was to monitor two remote temperature sensors, one outside in the shade and another one in the attic. Mainly to see if active ventilation of the attic might reduce our electricity bill. At first it was a daily chore to copy the data from the Raspberry Pi to my computer and create graphs. Later by copying ant pasting scripts found online I was able to automate that task.

Another project involved receiving satellite images from the NOAA weather satellites, it was a complete walk-through I had found on instructables.com. Also this was working locally, and images had to be retrieved daily from the Raspberry Pi to prevent it from running out of memory space.

Part of these projects was trying to understand the scripting and adapting it to my situation. But being faced with javascript and python that sometimes was difficult without knowing the languages.

Needless to say programming always has had my interest, was it the Pascal, Fortran and C+ labs in college or PLC programming at work and later programming telephone switches.

Since I moved to the US I have done some freelance telecom work, done some simple jobs just to be out of the house. Recently installed Windows 10 computers at a state facility. Have been looking around for other jobs and found some software engineering jobs in my area. Did apply but was failing the experience. So when I came across CareerKarma my interest of doing a guided study into Software Engineering was boosted. Looking at the options and to be able to support my family a bit when a short-time contract is offered I decided to go for the Flatiron School Part-Time Software Engineering track.

Update 1.0
Meanwhile things have changed, already in module 1 I was pretty far ahead of schedule. After finishing my project and scheduling my review we (four of us) received early access to the curriculum of module 2. Also module 2 went pretty smooth, which resulted that in the official second week I already came to the project. I requested a transfer to full time and was granted the switch. This meant that the week after the switch I was in project week for module 2. This switch will also mean that I will be finished, if all goes well, in September in stead of end of January. In the next few months I will post more blogs about my trip into becoming a Software Engineer. Hopefully after completion it will open doors that have been closed for me until now. But for sure it will help me to get a better understanding of things that have been a mystery in the past

Update 2.0
Live happens and I had to take a leave of absence. This resulted that I was rescheduled in a cohort that won't catch up with me until first week of January 2021. So thing are delayed with me finishing up. But in the back of my mind I still will be finished at about the same time as my original part time cohort, in that way it is not so much lost.