DEV Community: AdrnlnJnky

Daily Blog

AdrnlnJnky — Thu, 17 Dec 2020 02:46:17 +0000

Adrnlnjnky's Daily(ish) Blog Post

Today I spent several hours working on my weather app. Now I should explain
that the weather app is my punching bag. When I work on new concepts or
projects I like to then go try them out on that project. Today I migrated the
project to the Gin framework and rebuild the recording fucntions. I record
weather from three locations. When I was done I deleted it all.

And I gotta say that wasn't easy because I deleted more lines than I wrote.
Which is exactly why I did it. Let me see if I can explain. The code I wrote
today wasn't hard in fact refactoring and shrinking code is really fun and I
felt great watching the code become easier to read and better organized.

When I was done I jumped around the program and looked at what I had done, I
though about pushing it all up; then I though, is this really what I want to
push up? How long did that take me? Could I do it better? I also had this
though creep into my head, something I heard CodingGarden (a twitch streamer)
say, code roulette. And I did a pull instead of a push. Maybe tomorrow I'll
keep what I write.

Thanks for reading
and remember:
Smile it might hurt!

Tom Peltier

This has been another SmokingEars Production!!

RestFull API Detailing

AdrnlnJnky — Wed, 09 Dec 2020 22:47:27 +0000

Shaping UP

In the last few posts I build a CRUD interface, set up some testing and connected to a database. Great now I can store three strings of data. I know things are labelled with the word contacts but really we are just storing three strings. You could store anything you want in here right now. That doesn't seem right. I need to give this thing some constraints.

A quick review of my main.go file:


package main

import (
    "log"
    "net/http"

    "github.com/gorilla/mux"

    "gitlab.com/adrnlnjnky/RestAPI/contacts"
)

func main() {
    port := "23.73.19.2:5000"
    r := mux.NewRouter()

    r.HandleFunc("/contacts", contacts.GetContacts).Methods("GET")
    r.HandleFunc("/contacts/{search}", contacts.GetContact).Methods("GET")
    r.HandleFunc("/contacts", contacts.CreateContact).Methods("POST")
    r.HandleFunc("/contacts/{search}", contacts.UpdateContact).Methods("PUT")
    r.HandleFunc("/contacts/{search}", contacts.UpdateContact).Methods("PATCH")
    r.HandleFunc("/contacts/{search}", contacts.DeleteContact).Methods("DELETE")

    if err := http.ListenAndServe(port, r); err != nil {
        log.Fatalf("could not listen on port %v. Error: %v", port, err)
    }
}

This is a pretty simple server set up using gorilla/mux and HandleFunc to direct the json string where I want it to go. So far so good, now lets look at the top of my contacts.go file:


package contacts

import (
    "encoding/json"
    "fmt"
    "net/http"

    "github.com/gorilla/mux"
    "gitlab.com/adrnlnjnky/RestAPI/checker"
    "gitlab.com/adrnlnjnky/RestAPI/dbase"
    "gorm.io/gorm"
)

As you can see from my imports I'm using GORM to control the database which is opened by a package located in /dbase. I'm passing the information around with json and the /checker module is were we are going to qualify our entries before
we insert them; you know, make sure a phone number looks like a phone number. Here is my Contact and Contacts types and a and a few helper functions:


// Contact is the struct for the contacts list.
type Contact struct {
    gorm.Model
    Name string `json:"name"  gorm:"index"`
    Phone string `json:"phone" gorm:"uniqueIndex"`
    Email string `json:"email" gorm:"uniqueIndex"`
}

// Contacts is a slice of type Contact.
type Contacts []Contact

// sendJSON takes in and sends data back in JSON form with the Header set.
func sendJSON(w http.ResponseWriter, data interface{}) {
    w.Header().Set("Content-Type", "application/json")
    json.NewEncoder(w).Encode(data)
}

// This calls out to open the database then makes the link with the table.
func setupDB(r *http.Request) *gorm.DB {
    query := r.URL.Query()
    database := query.Get("database")
    db := dbase.GormOpenDB(database)
    db.AutoMigrate(&Contact{})
    return db
}

func dbError(w http.ResponseWriter, err error) {
    errNote := fmt.Sprintf("Something went wrong, got this error: %+v\n", err)
    w.WriteHeader(http.StatusBadGateway)
    sendJSON(w, errNote)
}

func noRowsError(w http.ResponseWriter, results string) {
    w.WriteHeader(http.StatusNotFound)
    note := fmt.Sprintf("Was not able to find your request: %v", results)
    sendJSON(w, note)
}

I've labels my Contact fields so that json and gorm both know exactly what I am talking about. The helpers should be self explanatory, one send json and the other one sets up a database to use. My first function is GetContacts which simple returns all the contacts.


func GetContacts(w http.ResponseWriter, r *http.Request) {
    contact := Contacts{}
    db := setupDB(r)

    err := db.Model(&Contact{}).Find(&contact)

    if err.Error != nil {
        dbError(w, err.Error)
    return
    }

    if err.RowsAffected < 1 {
        w.WriteHeader(http.StatusNoContent)
        note := fmt.Sprint(":Nobody found: Have you created anybody in your contact list?")
        sendJSON(w, note)
        return
    }

    sendJSON(w, contact)
}

Not much new here other than constructing a more elaborate message to return if there are no contacts and handling any errors that might come up. Ask for all the contacts you get all the contacts. This seems OK, lets move on and look at
the GetContact function.


func GetContact(w http.ResponseWriter, r *http.Request) {

    par := mux.Vars(r)
    name := par["name"]       // extract the name wanted

    contact := Contacts       // create a variable to put the info in.

    db := openDB(w, r)        // open the database

    results := db.Model(&Contact{}).Where("name = ?", name).First(&contact)

    if err.Error != nil {
        dbError(w, err.Error)
        return
    }
    if results.RowsAffected != 1 {             // Check we found someone
      noRowsError(w, "")
      return
    }

    w.WriteHeader(http.StatusFound)        //respond with desired info
    sendJSON(w, contact)
  }

Well that does get one contact but it only searches by name, and what if I don't remember exactly how I named my contact? Lets see if we can beef this function
up a bit...


func GetContact(w http.ResponseWriter, r *http.Request) {
    par := mux.Vars(r)
    search := par["search"]

    column, regsearch := checker.SortSearch(search)    // This Sorts name | phone | email

target := fmt.Sprintf("%v LIKE ?", column)  

    db := setupDB(r)       // grab the database
    contact := Contacts{}  // create a slice to send back

             // make the call
err := db.Debug().Model(&Contact{}).Where(target, regsearch).Find(&contact)

if err.Error != nil {         // deal with errors
    dbError(w, err.Error)
    return
}
if results.RowsAffected != 1 {    // deal with no results
    noRowsError(w, "")
    return
    }

   // Send back all contacts matching the request.
   sendJSON(w, contact)
}

Again I handled any errors that might come up and added some text to the returning messages. I also added some regex to the postgres call so that I can return on partial names. In addition I build a helper function that will look at the incoming request and tell postgres which column to look in. Wanna see my sorter?


package checker

import (
    "net"
    "regexp"
    "strings"
)

var (
    emailRegex = regexp.MustCompile("^[a-zA-Z0-9.!#$%&'*+/=?^_`{|}~-]+@[a-zA-Z0-9](?:[a-zA-Z0-9-]{0,61}[a-zA-Z0-9])?(?:.[a-zA-Z0-9](?:[a-zA-Z0-9-]{0,61}[a-zA-Z0-9])?)*$")

    phoneRegex = regexp.MustCompile(`^(?:(?:\(?(?:00|\+)([1-4]\d\d|[1-9]\d?)\)?)?[\-\.\ \\\/]?)?((?:\(?\d{1,}\)?[\-\.\ \\\/]?){0,})(?:[\-\.\ \\\/]?(?:#|ext\.?|extension|x)[\-\.\ \\\/]?(\d+))?$`)
)

OK so I don't speak regex as well as I should but I found these variables here:

https://www.golangprograms.com/regular-expressions.html


func SortSearch(data string) string {

    isEmail := IsEmail(data)    
                             // if it's an email return that
    if isEmail == true {
        parts := strings.Split(data, "@")        
                             // split on the @
        reg = "%" + parts[0] + "%" + parts[1]    
                            // send only the first half back
        return "email", reg                      
                            // with a little regex
    }

    isPhone := isPhone(data)     
                           // if it's a phone return that
    if isPhone == true {
        return "phone", reg
    }
    return "name", reg                                                 
                        // for anything else send back name
}

func isPhone(data string) bool {            
                              // searching the phone column

    if len(data) < 3 || len(data) > 25 {      
                              // at least 3 numbers
        return false                            
                        // otherwise might as well search all
    }
    if !phoneRegex.MatchString(data) {        
                                   // do the regex check
        return false
    }
    return true                               
                      // if it passes all my tests send true
}

All that seems easy enough and now we can search on all three of the columns in the database, and phone numbers and email addresses will be accepted. I also split out the email and I'm only searching on the handle note the domain portion. I think this will allow for ar least some searching potential. At this point I felt kind of dirty because my function names were drifings. I decided to rename my get functions and add one more. I renamed:

GetContacts -> GetAllContacts
GetContact to GetContacts

Actually I copied GetContatcts and renamed it then I made a one change to GetContact. I changed the GORM call from .Find to .Last. Now GetContact will give back exactly one contact while GetContacts will give back all contacts meeting the search. In my main function I made the adjustments and then I pointed /contact/{search} to GetContact.


    go r.HandleFunc("/contacts/{search}", contacts.GetContact).Methods("GET")
    go r.HandleFunc("/contacts/{search}", contacts.GetContacts).Methods("GET")
    go r.HandleFunc("/contacts", contacts.GetAllContacts).Methods("GET")

While I was working this out I was also adding tests to my test file. Test files can get long and messy, basically I added test to cover the phone searching, the email searching and the partial information searches. Like I said things get long and repetitive so lets move on to the CreateContacts.


func CreateContact(w http.ResponseWriter, r *http.Request){
db := setupDB(r)

var contact Contacts                                     
                                   // where to store data
_ = json.NewDecoder(r.Body).Decode(&contact)            
                                   // store the data sent

checkPhone := checker.IsPhoneValid(contact.Phone)     
                                    // qualify phone
checkEmail := checker.IsEmailValid(contact.Email)     
                                    // qualify email

switch {    This switch kicks back if the 
            email and phone do  not evaluate true

case checkPhone == false:
  w.WriteHeader(http.StatusNotAcceptable)
  response := fmt.Sprintf("Something seems wrong with the provided phone number. | %v\n", contact.Phone)
   sendJSON(w, response)

case checkEmail == false:
  w.WriteHeader(http.StatusNotAcceptable)
  response := fmt.Sprintf("Something seems wrong with the provided email address. | %v\n", contact.Email)
  sendJSON(w, response)
    }

results := db.Model(&Contact{}).Create(&contact)   
                      // record the new contact

switch {              // handle the errors or not recorded

case err.Error != nil:
   dbError(w, err.Error)  
case results.RowsAffected != 1:     
             // I'm not sure this code will ever run but hey

default:                // send back the all good
  newcontact := fmt.Sprintf("New contact created %v\n", &r.Body)
  sendJSON(w, newcontact)
    }
}

We already looked at my sorting helper, when it comes to recording a new number or email I have a bit stricter requirements when it comes to recording. So I build these function in my checker module.


func IsPhoneValid(data string) bool {

    if len(data) < 10 || len(data) > 20 {  // here I want at least 7 numbers
        return false 
    }
    if !phoneRegex.MatchString(data) {    
                                       // do the regex check
        return false
    }
    return true                // if you pass my tests send true
}




func IsEmailValid(search string) bool {

if len(search) < 5 && len(search) > 54 {     
                                          // Length limiter
  return false
  }

if !emailRegex.MatchString(search) {        
                                // match that string I found
  return false
  }

parts := strings.Split(search, "@")         
                                // split on the @

mx, err := net.LookupMX(parts[1])           
                             // is this from an email server?

if err != nil || len(mx) == 0 {             
                             // if an error return false
  return false            // if mx has no length return false
  }
  return true             // if you pass my tests send true
}

I don't ever make international calls but some folks do I expect, so I left some wiggle room for the numbers to take a few shapes. I'm also checking to make sure that the email is valid instead of just stripping it down for a search.

IsEmailValid checks if the email provided passes the required structure and length test. It also checks the domain has a valid MX record.

From the functions descritpon:

// LookupMX returns the DNS MX records for the given domain name sorted by preference.

So more or less I'm getting email address and phone numbers, I'm not doing anything on names because this is a contacts page and you can name things whatever you want.

DeleteContact like GetContacts doesn't need so much work but there are a few errors to handle and some flowery language to be added.


func DeleteContact(w http.ResponseWriter, r *http.Request) {
    db := setupDB(r)
    par := mux.Vars(r)
    search := par["search"]
    column, _ := checker.SortSearch(search)
    target := fmt.Sprintf("%v = ?", column)
    err := db.Where(target, search).Delete(&Contact{})
  if err.Error != nil {
    dbError(w, err.Error)
  }
      if err.RowsAffected != 1 {
    noRowsError(w, search)
    }
    w.WriteHeader(http.StatusGone)
    note := fmt.Sprintf("Deleted contact with %v: %v", column, search)
    sendJSON(w, note)
}

Update like Create contact is going to use the email and phone validators to qualify any entry changes. I also changed the way that I read into the database. Lets look at our new and longer update.


func UpdateContact(w http.ResponseWriter, r *http.Request) {
    db := setupDB(r)

    par := mux.Vars(r)
    search := par["search"]

    column, _ := checker.SortSearch(search)  // throwing away regex modification

    var contact Contact                    // read info into contact
    _ = json.NewDecoder(r.Body).Decode(&contact)

    checkPhone := checker.IsPhoneValid(contact.Phone)
    if checkPhone == false {
        w.WriteHeader(http.StatusNotAcceptable)
        response := fmt.Sprintf("Not a valid phone number. | %v\n", contact.Phone)
        sendJSON(w, response)
    }

    checkEmail := checker.IsEmailValid(contact.Email)
    if checkEmail == false {
        w.WriteHeader(http.StatusNotAcceptable)
        response := fmt.Sprintf("Not a valid email address. | %v\n", contact.Email)
        sendJSON(w, response)
    }

    target := fmt.Sprintf("%v = ?", column)

    results := db.Model(&Contact{}).Where(target, search).Updates(&contact)

    switch {
    case err.Error != nil:
    dbError(w, err.Error)
    case results.RowsAffected != 1:
    noRowsError(w, "search")
    default:
        fixedcontact := fmt.Sprintf("contact %v Updated to:\n name : %v\n  phone: %v\n email : %v\n",
            search,
            contact.Name,
            contact.Phone,
            contact.Email,
        )

        w.WriteHeader(http.StatusAccepted)
        sendJSON(w, fixedcontact)             // send back the update
    }
}

I feel a little bit better about all of this now. Of coarse I might want some other fields for my contacts. That should be pretty easy to do now that we have these three build up. So what else do you want to know about your contacts? Lets make a list and sort the list into searchable and non-searchable columns.

Searchable | NonSearchable

birthday | notes
alt. phone | spouce
work address | children
work phone
work
last name
first name
nick name
alt email 1
alt email 2
alt email 3
twitter
instagram
facebook?
linkdin
github
gitlab
twitch
youtube
zoom -> it that a thing?

I'm sure there are a other things that could be configured too. The other thing that I can do before I start making the contacts all specialized and featured is copy it off and reuse it. It can easily be used for users, employees, members, and with not to much work products, services and whatever else I might want. I mean for users I can just delete the phone number and presto. Then I can build
out a few features and some copy and paste magic presto employee it born. I think I will build up a few of these things and just stick them on a shelf, but not today. Well it's getting kind of late and my brain kinda hurts so until
next time, be well and remember to smile cause yup it might hurt.

Rest API Database Blog Post

AdrnlnJnky — Wed, 09 Dec 2020 16:15:26 +0000

In my last two articles I built a Restful api with a simple CRUD interface operating on contacts and set up some basic tests for it. Now that I can interact with my contact list it would be nice if I could keep that data someplace other than in the computers memory. For this task I'm going to reach for Postgres. I chose Postgres because it's the database I've been learning the last couple of months.
In addition to Postgres I've been trying to learn the GORM interface. GORM is an ORM for Go that works with Postgres and is supposed to make working with SQL easier. I have to share with you that I have spent more time smoking from the
ears trying to get GORM to do something that I had already implemented with raw SQL and the standard Go library than I spent writing the SQL statements in the first place. Persistence is an important part of learning so I'm going to keep beating my head against the GORM project a bit longer.

GORM does make some things very simple, like setting up the database. Automigrate(&SomeStruct) will set up/ align a table in your database with a struct in your code. GORM also works well with embedded tables. It comes with a nice embedded table gorm.Model which gives back ID, CreatedAT, UpdatedAt,
DeletedAt. (Yes DeletedAt means that there is a soft delete function, nice for those over zealous button pushers.) I'm going to use both of these features right away. Now when it comes to the Automigrate function I'm not really sure what the best practice is but I'm going to use it pretty prolificly here to align my Contact struct with my Postgres Table. I should not here that you can name your tables anything you want but GORM has naming convention that I followed. Here is the top bits of my contacts.go file.


package contacts

import (
    "encoding/json"
    "fmt"
    "net/http"

    "github.com/gorilla/mux"
    "gorm.io/gorm"
)

// Contact is the strict for the contacts list.
type Contact struct {
    gorm.Model
    Name  string `json:"name"`
    Phone string `json:"phone"`
    Email string `json:"email"`
}

// Contacts is a slice of Contact.
var Contacts []Contact

GORM likes the use of tags just like JSON so one of the things I'm gong to explore today is how I want to build those tags. One option is a generic tag something like "name", the other is more explicit json:"name" gorm:"name". In this case with both using name the generic probably works
fine, in fact most of the time it's probably just fine.

now the question: Do I alter the existing functions to hit the database or do I make a call to a new function? The delete function will get shorter if we just reach out to the database but do we need to worry about time? Do I want to pull the database into memory? Probably not and I don't really like the delete function I have, on the other hand I want to thing about making functions as generic as possible to help with a growing program.

The other question that I have to solve is how to handle the database. In the past I have just passed it around. I'm not sure how best to handle this problem here. I can set the params to be "database"="projectDB". Now I know the name of the database I'm working in but I'm not opening the database. What is the best way to pass around the database? For now I'm going to build a helper function to open the database for me. Before a helper function will be helpful I need to open the database. In a folder called dbase I wrote opener.go.


package dbase

import (
    "log"

    "gorm.io/driver/postgres"
    "gorm.io/gorm"
)

func GormOpenDB(dbase string) *gorm.DB {
    dsn := "user=postgres dbname=devdatabase port=5432 sslmode=disable"
    db, err := gorm.Open(postgres.Open(dsn), &gorm.Config{})
    if err != nil {
        log.Fatalf("Failed to open database: %v", db)
    }
    return db
}

Easy enough just tell it what database you want to open and it will send you back a pointer you can use. Now the helper function.


func openDB(w http.ResponseWriter, r *http.Request) *gorm.DB {
    query := r.URL.Query()
    database := query.Get("database")
    db := dbase.GormOpenDB(database)
    db.AutoMigrate(&Contact{})
    return db
}

I just pass the hole thing over grab the Params get the desired database, open it link to my table and send the database back. (I should note here that I added a query to my tests so that they would set the params properly.) This should help keep my code clean and easy to read. Oh while I was at it I decided to pull the json encoding bit out:


func sendJSON(w http.ResponseWriter, data interface{}) {
    w.Header().Set("Content-Type", "application/json")
    json.NewEncoder(w).Encode(data)
}

I know it's only two lines but every little bit helps. Ok now I can query my database. I'm going to start with create because there isn't anything in the contacts table yet.


func CreateContact(w http.ResponseWriter, r *http.Request) {
    var contact Contact
    _ = json.NewDecoder(r.Body).Decode(&contact)
    db := openDB(w, r)
    db.Model(&Contact{}).Create(&contact)
    sendJSON(w, contact)
}

Pretty simple read the info create the new contact and away we go. I ran that a few times and made a couple instances of my text contact. In the future I may restrict multiple instances and I'll most likely do that with a database restriction. For now I'm going to let it be and move on to getting all contacts.


func GetContacts(w http.ResponseWriter, r *http.Request) {
    contact := Contacts
    db := openDB(w, r)
    db.Model(&Contact{}).Find(&contact)
    sendJSON(w, contact)
}

Again here like create I call out to the dbOpen to create my database instance, then I query the database for all the users and send it back with my json helper. Getting one contact is similar and GORM has a few commands to do this depending on your needs. I'm just going to ask for the first instance.


func GetContact(w http.ResponseWriter, r *http.Request) {

    par := mux.Vars(r)
    name := par["name"]       // extract the name wanted

    contact := Contacts       // create a variable to put the info in.

    db := openDB(w, r)        // open the database

    results := db.Model(&Contact{}).Where("name = ?", name).First(&contact)

    if results.RowsAffected != 1 {             // Check we found someone
        w.WriteHeader(http.StatusNotFound)      // respond if we did not
        sendJSON(w, par)
    }

    w.WriteHeader(http.StatusFound)        //respond with desired info
    sendJSON(w, contact)
}

I can make them and I can find them now lets delete them.


func DeleteContact(w http.ResponseWriter, r *http.Request) {
    par := mux.Vars(r)
    name := par["name"]
    db := openDB(w, r)
    results := db.Where("name = ?", name).Delete(&Contact{})
    if results.RowsAffected != 1 {
        w.WriteHeader(http.StatusNotFound)
        sendJSON(w, par)
    }
    w.WriteHeader(http.StatusGone)
    sendJSON(w, "Deleted")
}

Everything here should look pretty familiar the only new bit here is the Delete
move. Now to update update!


`func UpdateContact(w http.ResponseWriter, r *http.Request) {
    par := mux.Vars(r)
    name := par["name"]                   // still finding our name the same way

    var contact map[string]interface{}    // a place to store the new information

    blob, _ := ioutil.ReadAll(r.Body)
    err := json.Unmarshal(blob, &contact)  // had to change our reader here.
    if err != nil {
        panic(err)                          // I promise to handle the err
    }
    for k, v := range contact {           // range the info sent in
        db := openDB(w, r)                  // open database and save the stuff
        db.Model(&Contact{}).Where("name = ?", name).Updates(map[string]interface{}{k: v})

        w.WriteHeader(http.StatusAccepted)  // write Status to header
        sendJSON(w, contact)                // send it back
    }
}

I had to change things up a bit here. The reason I did things this way was to handle cases where I get only part of the new data. This way if I get all new information or only the updated information the function can handle it. This
will update everyone with the same name but that's a problem for my next post.

This program is up and running with a simple interface a database to keep our contacts in and a bit of testing. I'd like to note that the only change I made to the test was to add the database information into the params. Other than that I just leaned on my tests to make sure my code was still working. So, With the code covered by some testing I'm
going to build out my contact info and tighten up some restrictions. I mean it would be nice if phone numbers looked like phone numbers and I didn't duplicate contacts a bunch. That said Thank you for reading and have a great day!

Tom Peltier
Smile it might hurt!

RestAPI Blog Post

AdrnlnJnky — Wed, 09 Dec 2020 00:00:00 +0000

In this Post I am going to talk about building a Rest API with a CRUD
interface. Over the next few blog posts I will build this out with several
features including an inventory of contacts and services, a contact list for
users, a secure log in and an interface to work with all of it. I am not going
to talk about installing Go or how packages work. I may do some of these types
of blog posts in the future but for now if you don't know the very basics with
go I recommend you go get Go installed and write a Hello World, then come
back. That said, I'm going dive straight into building the program.

Over the next few posts I am going to build this project that will have.

@. CRUD interface
@. Contacts
@. Users w/secure log in
@. contact Inventory
@. Services Inventory
@. Postgresql database for persistence
@. Some other stuff I have not thought of or decided on

In this post we are going to create the CRUD interface. In other words we are
going to write some code to add, change, look at, and delete things in our
database.

C R U D
create review update delete

I started with an outline. My outline is going to take several forms. This
first outline might seem rough but it's where I started. I figured there are
several ways to go about this. I could just build a main function and start
building out features and extract as I go. Another option is to create an
outline of the project and then build out features based on the outline. Here is
my first outline.

@. contacts/services/contacts/users -> all need a CRUD
-> I think I'll us gorilla/mux for this section.
@. review
@. find all contacts
@. find one contact
@. create
@. check that it does not exists
@. create the contact
@. update
@. check that it does exist
@. change one field
@. change several fields
@. delete
@. Check that it does exist
@. delete it
@. server
@. go http package is pretty cool
@. routers
@. gotta find the stuff
@. database
@. I think I'm going to use GORM for this

With at least an idea of structure set up I created a main.go and three folders:
users, dbase, contacts. I'm going to start with contacts, so I created a
contacts.go file in the contacts folder. I started out with some basic code; this is
in package contacts. To get this thing going I'm going to make my contact
struct very simple. We will come back and flush this out in a bit.

package contacts

import (
    "encoding/json"
    "fmt"
    "net/http"

    "github.com/gorilla/mux"
)

type Contact struct {
    Name   string
  E-mail string
  Phone  string
}

Next I'm going to build my transactions for the Contacts. Those are find,
update, create and delete. First is the find:

(Oh by the way I'm using NVIM and at this point I recorded a register. My regerster will give me back (w http.ResponseWriter, r
*http.Request) {, because it's going to save me a bunch of typing.

I'm going to use j for this register. So in vim from the normal mode I set my
curser on the last letter of a function and I typed:

qja(w http.ResponseWriter, r *http.Request) {}kq
I now have that whole thing in my j register which I reach in normal mode by
typeing @j.

func Getcontacts(w http.ResponseWriter, r *http.Request) {
    w.Header().Set("Content-Type", "application/json")
    json.NewEncoder(w).Encode(contacts)
}

I want to use json to pass the information around so the first thing I'm going
to do is set the header type to json. Then I just encode everything stored in my
contacts variable and send it back. Later we will connect this to a database
but for now I want to keep it simple and get it working.

Before I get to far ahead of myself I'm going to build the rest of my functions
to just return a simple message, then I'll set up our main function and see that
we are making our connections.

This is a pretty simple main function that will allow me to test my endpoints.
You could use something like Postman to test this or launch a server locally and then curl localhost:5445. This should return 'Home Page', but I get ahead of myself. Let me show you my main.go:

package main

import (
    "log"
    "net/http"

    "github.com/gorilla/mux"

    "gitlab.com/adrnlnjnky/RestAPI/contacts"
    "gitlab.com/adrnlnjnky/RestAPI/users"
)

That stuffs important but hopefully we don't need to talk about it now. If we
do then head to (golang)[http://www.golang.org].

func main() {
    contacts.contacts = append(contacts.contacts, contacts.contact{ID: 1, Name: "CoCo Puffer Pants", Phone: 23.00, Email: "good@stuff"})
    contacts.contacts = append(contacts.contacts, contacts.contact{ID: 2, Name: "Lucky Dog", Phone: 3.00, Desc: "the@stuff"})
    contacts.contacts = append(contacts.contacts, contacts.contact{ID: 3, Name: "Dorin Dog", Phone: 300.00, Desc: "wool@stuff"})
    contacts.contacts = append(contacts.contacts, contacts.contact{ID: 4, Name: "Thorin Dog", Phone: 233.00, Desc: "dw@stuff"})

Right up top I set up my in memory inventory so I have something to work and
test with.

    port := ":5445"
    r := mux.NewRouter()

    r.HandleFunc("/contacts", contacts.Getcontacts).Methods("GET")
    r.HandleFunc("/contacts/{name}", contacts.Getcontact).Methods("GET")
    r.HandleFunc("/contacts", contacts.Createcontact).Methods("POST")
    r.HandleFunc("/contacts/{name}", contacts.Updatecontact).Methods("PUT")
    r.HandleFunc("/contacts/{name}", contacts.Deletecontact).Methods("DELETE")

    if err := http.ListenAndServe(port, r); err != nil {
        log.Fatalf("could not listen on port %v. Error: %v", port, err)
    }
}

Setting up some variables first to make things easier to read, then I use
Handlefunc to look for the contacts key word and then also look for the method
using the Methods option from gorilla/mux to determine the
GET/POST/PUT/DELETE options. Finally at the bottom I use ListenAndServe from
the net/http package to listen on the port variable and serve up the goods.

Now I can build the package and test the server. Right now all of my end points
simply return the list of contacts. Cool, things are working, Now to make them
work correctly.

func Getcontact(w http.ResponseWriter, r *http.Request) {
    w.Header().Set("Content-Type", "application/json")
    par := mux.Vars(r) // this sweet little bit gets the parameters from the call
    for _, target := range contacts {
        if target.Name == par["name"] {
            json.NewEncoder(w).Encode(target)
            return
        }
    }
    json.NewEncoder(w).Encode(&contact{})
}

Getcontact needs to return only one contact not all of them so we need to do
some filtering. Gorilla/mux gives an easy way to extract the name. I pull that name and save it to par (parameters) and then I range over the contacts and find a match if it exists. I think the code is
pretty straight forward here.

My next feature is going to be CREATE.

func Createcontact(w http.ResponseWriter, r *http.Request) {
    w.Header().Set("Content-Type", "application/json")
    var contact contact
    _ = json.NewDecoder(r.Body).Decode(&contact)
    contacts = append(contacts, contact)
    json.NewEncoder(w).Encode(contact)
}

Once the header business is handled I create a variable to hold the new contact,
append the Contacts list and then encode the information and send it back.

Now for Delete:

func Deletecontact(w http.ResponseWriter, r *http.Request) {
    w.Header().Set("Content-Type", "application/json")
    par := mux.Vars(r)
    for dex, target := range contacts {
        if target.Name == par["name"] {
            contacts = append(contacts[:dex], contacts[dex+1:]...)
            break
        }
    }
    json.NewEncoder(w).Encode(contacts)
}

I'm using the par variable again to extract the desired contact to delete. I'm
ranging over the inventory again to find a match. Once the contact is located I
rebuild my contact variable with everything before and after the target contact,
deleting the contact from the inventory.

Update is going to use the same tricks as delete to remove the old data and then
add the contact back in with the new data.


func Updatecontact(w http.ResponseWriter, r *http.Request) {
    w.Header().Set("Content-Type", "application/json")
    par := mux.Vars(r)
    for dex, target := range contacts {
        if target.Name == par["name"] {
            contacts = append(contacts[:dex], contacts[dex+1:]...)
            var contact contact
            _ = json.NewDecoder(r.Body).Decode(&contact)
            contact.Name = par["name"]
            contacts = append(contacts, contact)
            json.NewEncoder(w).Encode(contact)
            return
        }
    }
}

Like I said I used the functionality from delete to remove the contact to be
appended. I then the create funtionality to appended the inventory with the
corrected information.

To put this all together I have my project folder with:

go.mod
go.sum
main.go
contacts/contacts.go
README.md
dbase/
users/

I have created a few in memory contacts, set a variable for the port that the program will work on and a router variable.

Then I use HandleFunc to route the requests to the correct functions, and then set the program to listen and serve back data on the given port.

For the CRUD functions we take advantage of the gorilla/mux module and qualify contacts by name. In my next article I will set up some testing to cover the logic I've built so far. Once my tests are in place I will turn my attention
to making the data persist. Thank you for taking the time to read this post, come on back and see where it leads.

Tom Peltier
Smile it might hurt!

PART II First tests

AdrnlnJnky — Tue, 08 Dec 2020 18:58:42 +0000

My last post I build up a basic REST API. I set up the server and routing in
the home page and then build out a CRUD interface for the server to interact
with. Before we go any further I want to get some basic testing set up. There
is more than one way to set up testing and today I want to explore
(apitest)[https://apitest.dev/].

"A simple and extensible testing Library for
Go. You can use apitest to simplify testing of REST services, HTTP handlers
and HTTP clients."

Well simple sounds good to me so lets get started. One of the things I found
right way is that I can test headers and body content pretty easily and that
there is a debug option. OK, there are three parts Configuration, Request and
Assertions.

Configuration sets up the call:
Request is where you define the test input.
Assertions: thats what you expect back.

Before we start testing here is the top bits of my code:


package users

import (
    "encoding/json"
    "fmt"
    "net/http"

    "github.com/gorilla/mux"
    "gorm.io/gorm"
)

// Contact is the strict for the contacts list.
type Contact struct {
    gorm.Model
    Name  string `json:"name"`
    Phone string `json:"phone"`
    Email string `json:"email"`
}

// Contacts is a slice of Contact.
var Contacts []Contact


//GetContacts retrieves all contacts
func GetContacts(w http.ResponseWriter, r *http.Request) {

    w.Header().Set("Content-Type", "application/json")

    json.NewEncoder(w).Encode(Contacts)
}

OK so with a bit of context lets look at the first piece of code I'm going to test.
in my contacts_test.go file I set it up like this:


package users

import (
    "net/http"
    "net/http/httptest"
    "testing"

    "github.com/gorilla/mux"
    "github.com/steinfletcher/apitest"
)

func TestGetcontacts(t *testing.T) {
    r := mux.NewRouter()
    r.HandleFunc("/contacts", GetContact).Methods("GET")
    ts := httptest.NewServer(r)
    defer ts.Close()
    res, err := http.Get(ts.URL + "/contacts")
    if err != nil {
        t.Errorf("Got an error: %s", err.Error())
    }
    if res.StatusCode != http.StatusOK {
        t.Errorf("Expected %d, received %d", http.StatusOK, res.StatusCode)
    }
}

Here is the code I'm testing again:


func GetContacts(w http.ResponseWriter, r *http.Request) {

    w.Header().Set("Content-Type", "application/json")

    json.NewEncoder(w).Encode(Contacts)
}

I mean pretty simple it sends back everything in the Contacts struct. In order
to get this up and running I populated a contacts variable with four names,
numbers and emails.


    Contacts = append(Contacts, Contact{Name: "Dorin", Phone: "555-4385", Email: "Dorin@coolcat.com"})
    Contacts = append(Contacts, Contact{Name: "CoCo", Phone: "444-4385", Email: "CoCo@coolcat.com"})
    Contacts = append(Contacts, Contact{Name: "Lucky", Phone: "888-4385", Email: "Lucky@coolcat.com"})
    Contacts = append(Contacts, Contact{Name: "Sidney", Phone: "999-4385", Email: "Sidney@coolcat.com"})

Wait a minute hold on. I thought we were using apitest? Oh we are going to use
apitest, but I've never used it before so I wanted to have something to compare
it with. Plus it was right after writing this bit that I went out and found
apitest. Here is what my Apitest test looks like.


    t.Run("Get All Contacts", func(t *testing.T) {

        apitest.New().                // this line starts a new test in the apitest framework
            Handler(r).                 // remember we defined r to our mux router at the top of the function.
            Get("/contacts").           // what I want to get
            Expect(t).                  // THIS IS THE PIVOT POINT
            Status(http.StatusOK).      // this is what I want to get back
            End()  // this ends the statement
    })

Yes I used a t.Run and just tacked this onto my first test. Mostly because I
wanted to have the two to compare. I wanted to see this thing fail so I
changed the StatusOK to StatusNotFound and sure enough the test failed. I've
seen it fail, and I've seen it work, life is good. Next seems logical to test
getting a single contact.

I can reuse a lot of the code from my last test I just need to get more
specific. The GetContact function returns one contact so I need to test not
only that it returns a contact, but that it doesn't return a contact if non
exists. Lets look at the test.


func TestGetcontact(t *testing.T) {

// The setup!
    r := mux.NewRouter()
    r.HandleFunc("/contacts/{name}", GetContact).Methods("GET")
    ts := httptest.NewServer(r)
    defer ts.Close()

    t.Run("Not Found", func(t *testing.T) {  // yeah testing name not found!
        apitest.New().
            Handler(r).

            Get("/contacts/DoesNotExist").    // asking for a name that's not there
            Expect(t).

            Status(http.StatusNotFound).
            End()
    })

This test fails -> expected: 404 || actual 200
I'm looking for StatusNotFound but I never told anybody to deal with the
headers. Had there been an error we would know about it but an empty json
package is just empty so I need to set the header when we don't find
anything. In my GetContact function I use an if statement to single out the
desired contact, encode it to json and send it back. All I have to do is add
a w.WriteHeader(http.StatusNotFound) right before the program exits and after
the if statement has returned. Check it out.


func GetContact(w http.ResponseWriter, r *http.Request) {
    w.Header().Set("Content-Type", "application/json")
    par := mux.Vars(r)
    for _, target := range Contacts {
        if target.Name == par["name"] {        // where we single out the name
            json.NewEncoder(w).Encode(target)
            return                               // exit function if we found the name
        }
    }
    w.WriteHeader(http.StatusNotFound)       // if we got we the name wasn't found
    json.NewEncoder(w).Encode(&Contact{}     // so we write the status we want and
                                           // send back the json.
}

that one line of code and now the test passes. What about when we find a
person?


    t.Run("Found", func(t *testing.T) {
        apitest.New().
            Handler(r).
            Debug().
            Get("/contacts/Dorin").
            Expect(t).
            Status(http.StatusOK).
            End()
    })

}

Once again the test is going to fail because we don't have anybody in our
Contacts, so I pull down a name from the last test and use it.


    Contacts = append(Contacts, Contact{Name: "Dorin", Phone: "555-4385", Email: "Dorin@coolcat.com"})

When I run this test again it passes! Now I'm getting back StatusOK and
presumably Dorin and his information. Lets just make sure that we did get back
Dorin and his phone and email information. Or even better I'm going to copy
over one more contact and I'm going to search for them, except this time I'm
going to check the body. So I make another contact named CoCo (Dorin and CoCo
are my dogs by the way). Then I set the test up like this:


    t.Run("Found Check the Body", func(t *testing.T) {
        apitest.New().
            Handler(r).
            Get("/contacts/CoCo").
            Expect(t).
            Assert(jsonpath.Contains(`$.phone`, "299-232-2385")).    //here's the new bit
            End()
    })

Assert allows for just that and the jsonpath is a helper to make it easier to
check your json. The Contains() option allows you to pick any item in the json
string and check it. I checked the phone number because, why not.

Now the GET functions are tested lets move on to CreateContact.

A quick review of the code bit:


func CreateContact(w http.ResponseWriter, r *http.Request) {
    w.Header().Set("Content-Type", "application/json")
    var contact Contact
    _ = json.NewDecoder(r.Body).Decode(&contact)
    Contacts = append(Contacts, contact)
    json.NewEncoder(w).Encode(contact)
}

Here is the test code, most of this should be familiar. Since we are creating
no need for an in memory contact, we can just create one:


func TestCreatecontact(t *testing.T) {
    r := mux.NewRouter()
    r.HandleFunc("/contacts", CreateContact).Methods("POST")
    ts := httptest.NewServer(r)
    defer ts.Close()

        apitest.New().
            Handler(r).
            Post("/contacts").
            Header("Content-Type", "application/json").
            JSON(`{"name": "X Tester", "phone": "555-5555", "email": "junik@gilskd"}`).
            Expect(t).
            Assert(jsonpath.Contains(`$.phone`, "555-5555")).
            Status(http.StatusOK).
            End()
}

In this test I added some json above the Expect(t), This will get sent with the
POST method. Then I Assert that it will give me back a StatusOK and the new
contact. That is exactly what we got back.

Now lets delete someone.


func DeleteContact(w http.ResponseWriter, r *http.Request) {
    w.Header().Set("Content-Type", "application/json")
    par := mux.Vars(r)
    for idx, target := range Contacts {
        if target.Name == par["name"] {
            Contacts = append(Contacts[:idx], Contacts[idx+1:]...)
        w.WriteHeader(http.StatusGone)
        json.NewEncoder(w).Encode(Contacts)
            break
        }
    }
    w.WriteHeader(http.StatusNotFound)
    json.NewEncoder(w).Encode(Contacts)
}

I remeber this, we found the name, deleted it and tacked it back on the end
with the changes. OK, lets test:


func TestDeletecontact(t *testing.T) {
    Contacts = append(Contacts, Contact{Name: "Dorin", Phone: "555-4385", Email: "Dorin@coolcat.com"})

    r := mux.NewRouter()
    r.HandleFunc("/contacts/{name}", DeleteContact).Methods("DELETE")
    ts := httptest.NewServer(r)
    defer ts.Close()

    t.Run("Delete Existing contact Mocking", func(t *testing.T) {
        apitest.New().
            Handler(r).
            Debug().
            Delete("/contacts/Dorin").
            Expect(t).
            Status(http.StatusGone).
            End()
    })

This is pretty straight forward. I put Dorin into memory and then asked
DeleteContact to delete him. Now what happens if somebody tries to delete
someone that doesn't exits?

    t.Run("No contact To Delete Mocking", func(t *testing.T) {
        apitest.New().
            Mocks().
            Handler(r).
            Debug().
            Delete("/contacts/xxxNotacontact").
            Expect(t).
            Status(http.StatusNotFound).
            End()
    })

}

Sure enough when I run this test I get back StatusNotFound and the test passes.
Now there is only one function to test. Lets look at update.


func UpdateContact(w http.ResponseWriter, r *http.Request) {
    w.Header().Set("Content-Type", "application/json")

    par := mux.Vars(r)
    fmt.Println(par)
    json.NewEncoder(w).Encode(r)

    for idx, target := range Contacts {
        if target.Name != par["name"] {
            fmt.Println("Trouble in Paradise")
            w.WriteHeader(http.StatusNotFound)
            json.NewEncoder(w).Encode(par)
            return
        }

        if target.Name == par["name"] {
            Contacts = append(Contacts[:idx], Contacts[idx+1:]...)
            var contact Contact
            _ = json.NewDecoder(r.Body).Decode(&contact)
            contact.Name = par["name"]
            fmt.Println("Can you believe we are in the if statement ", par["name"])
            Contacts = append(Contacts, contact)
            w.WriteHeader(http.StatusMovedPermanently)
            json.NewEncoder(w).Encode(contact)
            return
        }
    }
}

So update finds the name, deletes the contact and creates a new contact with the
updated information. We can test this like we tested create except here we will
need somebody in the contact list to make changes too. Here is my test code.


func TestUpdatecontact(t *testing.T) {
    Contacts = append(Contacts, Contact{Name: "CoCo", Phone: "999-888-9999", Email: "Coco@onyxrocks.com"})

    r := mux.NewRouter()
    r.HandleFunc("/contacts/{name}", UpdateContact).Methods("PUT")
    ts := httptest.NewServer(r)
    defer ts.Close()

    t.Run("contact Updated", func(t *testing.T) {
        apitest.New().
            Handler(r).
            Observe().
            Debug().
            Put("contacts/CoCo").                         // PUT instead of POST
      JSON(`{"name": "CoCo", "phone": "555-5555", "email": "CoCo@Best.com"}`).
            Expect(t).
            Assert(jsonpath.Contains(`$.phone`, "555-5555")).
            Status(http.StatusMovedPermanently).
            End()
    })

}

That rounds out my test suite for the time being. If there is any refactoring
to do it's always nice to do it under the protection of my tests. Once I've
looked over my code I'm going to jump back into my outline and I'll be back soon
with my next step. Thanks for reading and smile, it might hurt!

Tom Peltier

Adrnlnjnky's Daily(ish) Blog Post

AdrnlnJnky — Mon, 07 Dec 2020 17:19:16 +0000

I'm going to start writing a daily-ish blog about what I'm working on and I'm
going to place it on my homepage. The first thing I'm going to do is pull in the script
from my Daily Journal builder and alter it to make my daily Blog Post.
Then I will write a script to build the markdown file that I will actually
write the daily(ish) Blog posts in. Finally I'm going to build a template for my Blog home page
so I can build the actual home page with the daily post embedded into it.

Here is a bit of my build script relevant to the Daily Journal

// create some date variables
month=$(date +'%B')
day=$(date +%d)
yesterday=$(date '+%d' --date='1 day ago')
year=$(date +Y)

// Build my Daily Journal
page='Daily'
rm /PATH/TO/Journals/Daily/Daily.md
touch /PATH/TO/Journals/Daily/Daily.md
i=1
while [[ $i < 9 ]]; do
  next_journal=/Journal/Daily/$(date '+%B/%d%B.md' --date="${i} day ago")
  cat $next_journal >> /PATH/TO/Journals/Daily/${page}.md
  // echo "$next_journal"
  let i++
done

pandoc -s /PATH/TO/Journals/Daily/${page}.md \
-c /PATH/TO/Helpers/Style/Header.css \
-c /PATH/TO/Helpers/Style/index.css \
-o /PATH/TO/Journals/Daily/index.html \
-A /PATH/TO/Helpers/HTML/Footer.html

Ok this bit goes out and pulls the articles from my personal folder and
compiles them into an html document. For my journal I want several days posted
but for my blog post things are a bit easier.

( I've repeated the relevant bits for clarity here. )

// Build my Daily Blog Post

// create some date variables
month=$(date +'%B')
day=$(date +%d)
year=$(date +Y)

// create a variables to be used for the folder and the document names.
page='Daily'
title=${day}${month}${year}.md

// This will go to my personal articles folder find todays blog post and generate an HTML
// page in my Bloging folder.

pandoc -s  PATH/TO/${page}/${title}
-c /PATH/TO/Helpers/Style/Header.css \
-c /PATH/TO/Helpers/Style/index.css \
-o /PATH/TO/Articles/Daily/index.html \
-A /PATH/TO/Helpers/HTML/Footer.html

Now for all of that to work my Blog posts have to actually exist. For that I
wrote a little template file with the title, date, a header and a signature.
Then I wrote this little script to generate the file. This will help me avoid
the inevitable typeos if I should write my file name hand every day.

// create some date variables
month=$(date +'%B')
day=$(date +%d)
year=$(date +%Y)

// create a variables to be used for the folder and the document names.
dir='Daily'
title=${day}${month}${year}.md

// check to see if the file exists
if [ -f PATH/TO/$month/$NewJournal ];
then
  echo "${title" Blog Exists"
else
  cp PATH/TO/Daily/template.md PATH/TO/${dir}/${title}
fi

With those two bits set up I now have a daily Blog post that I can write in
Markdown, and convert into HTML when I move it into my Blog. That if loop is
there to make sure I don't accidentally write over an existing blog post.

As I put this together I realize that it's probably time for me to look at the
html/template package in Go to start building my entire blog site, so I decided
to through a quick hack together so that this will work for now. In the next
few days I'll post what I came up with after I have studied the HTML packages for Go.

The last part of my hack I copied my template into two files, Top and Bottom. Then I used pandoc to build the index page, with the Blog post sandwiched in the middle.

// Build the Home Page

// variable that matches the name of the daily blog
title=${day}${month}${year}.md

// pandoc the two halves of the Home page with the
// article sadwiched in the middle
pandoc -s  PATH/TO/${page}/${title}
--include-before-body="PATH/TO/Helpers/Templates/HomeHeader.html" \
--include-after-body="PATH/TO/Helpers/Templates/HomeFooter.html"

Thanks for taking the time to read my first Daily Blog and check back soon to
see what I'm up to next.

Thanks for reading
and remember:
Smile it might hurt!

Tom Peltier

This has been another SmokingEars Production!!