Building Out A Dev Blog
Welcome back to what might be the final trip into our static site generator. For now at least! This week we are going to add a very basic command line function. This will allow us to target a source directory and a destination directory. We'll also look at a couple of things that came up in the comments. Let's get right into it!
Nested If/Else
@ladydascalie
commented on the makePost() function, point out the wordy if/else statements. By moving the else up and out of the if makes this much easier to read. This is one instance were being verbose to a fault makes the code harder to follow.
Original Snippet
descIntf, ok := fm["description"]
if ok {
description, ok := descIntf.(string)
if ok {
post.Description = description
} else {
post.Description = ""
}
} else {
post.Description = ""
}
Refactored Snippet
descIntf, ok := fm["description"]
post.Description = ""
if ok {
if desc, ok := descIntf.(string); ok {
post.Description = desc
}
}
I moved the post.Description-esc portions over the assignment of the interface. Either way works in the end. I also began pulling out many of the panic()'s that littered the code. Many were there because I wasn't quite sure how I wanted to handle these cases and decided that failing fast was a good first step. I'll comment a bit more on them as we go through the code.
Constants
@dirkolbrich
also left a comment on the use of my constant delimiter. When I first started I was thinking that we would accept different delimiters. We won't be adding that we'll stay with the three dashes, like Jekyll. But, by passing it into splitData() the function will be easier to test and comprehend. I should also pass it into makePost() but the signature is very long already so, for now, I didn't. It was also pointed out that I had not used all upper case on the constant name, const delimiter = "---". For many languages, you would use DELIMITER to show in the code that it is a constant. I hadn't noticed it at the time and neither had go lint. Interesting. Go has some excellent resources so I checked Effective Go and the Go Wiki. And as it turns out constants are not all caps, Go prescribes the use of mixed case. This makes a certain amount of sense. A constant/variable (or function, or struct, etc.) starting with a capital letter gets exported. We don't want the delimiter exported so lower case it is. There are exceptions to this rule in the Go code base but, I'm not going to argue one way or the other. Maybe when I've got a better grasp of the language though...
Code Walkthrough
Once again we'll start from the top and I'll try to call out what's changed. Near the top not much, we've added flag and log. flag so we can use command line options and log for formatting errors.
package main
import (
"bufio"
"bytes"
"flag"
"fmt"
"html/template"
"io"
"io/ioutil"
"log"
"os"
"regexp"
"strings"
"github.com/microcosm-cc/bluemonday"
"github.com/russross/blackfriday"
yaml "gopkg.in/yaml.v2"
)
const delimiter = "---"
type post struct {
Title string
Published bool
Description string
Tags []string
CoverImage string
Series string
PostBody template.HTML
}
type index struct {
Pages []Page
}
type Page struct {
FileName string
Title string
}
var indexTempl = `<!DOCTYPE html>
<html lang="en">
<head>
<title>shindakun's dev site</title>
<meta charset="utf-8">
<meta http-equiv="X-UA-Compatible" content="IE=edge">
<meta name="HandheldFriendly" content="True">
<meta name="viewport" content="width=device-width, initial-scale=1.0">
<meta name="referrer" content="no-referrer-when-downgrade" />
<meta name="description" content="shindakun's dev site" />
</head>
<body>
<div class="index">
{{ range $key, $value := .Pages }}
<a href="/{{ $value.FileName }}">{{ $value.Title }}</a>
{{ end }}
</div>
</body>
</html>
`
var postTempl = `<!DOCTYPE html>
<html lang="en">
<head>
<title>{{.Title}}</title>
<meta charset="utf-8">
<meta http-equiv="X-UA-Compatible" content="IE=edge">
<meta name="HandheldFriendly" content="True">
<meta name="viewport" content="width=device-width, initial-scale=1.0">
<meta name="referrer" content="no-referrer-when-downgrade" />
<meta name="description" content="{{.Description}}" />
</head>
<body>
<div class="post">
<h1>{{.Title}}</h1>
{{.PostBody}}
</div>
</body>
</html>
`
I could get rid of getContentsOf() and use ioutil.ReadAll() in main(). I've left it though for now since I keep telling myself I'm going to write tests and it'd be an easy function to test. I had it returning depending on the error state in an if. There was no need for that as ioutil.ReadAll() matches the return already so it can return.
func getContentsOf(r io.Reader) ([]byte, error) {
return ioutil.ReadAll(r)
}
func parseFrontMatter(b []byte) (map[string]interface{}, error) {
fm := make(map[string]interface{})
err := yaml.Unmarshal(b, &fm)
if err != nil {
msg := fmt.Sprintf("error: %v\ninput:\n%s", err, b)
return nil, fmt.Errorf(msg)
}
return fm, nil
}
func splitData(fm []byte, delimiter string) ([][]byte, error) {
b := bytes.Split(fm, []byte(delimiter))
if len(b) < 3 || len(b[0]) != 0 {
return nil, fmt.Errorf("Front matter is damaged")
}
return b, nil
}
makePost() is where the largest set of changes occurred. As mentioned in the opening those changes are a direct response to a very good comment on the last article. I was thinking about splitting this function up since as it is we're returning the template and the post{}. It may be clearer, in the long run, to populate the post struct first. Then feed that into a different function to populate the template. That's some easy refactoring I can tackle if we do have one more post in this part of the "ATLG" series.
// makePost creates the post struct, returns that and the template HTML
func makePost(fm map[string]interface{}, contents []byte, s [][]byte) (*template.Template, *post, bool) {
post := &post{}
post.Published = false
pubIntf, ok := fm["published"]
if ok {
if published, ok := pubIntf.(bool); ok {
post.Published = published
}
}
if !post.Published {
return nil, nil, true
}
post.Title = ""
titleIntf, ok := fm["title"]
if ok {
if title, ok := titleIntf.(string); ok {
post.Title = title
}
}
if post.Title == "" {
return nil, nil, true
}
post.Description = ""
descIntf, ok := fm["description"]
if ok {
if description, ok := descIntf.(string); ok {
post.Description = description
}
}
post.Tags = []string{}
tagsIntf, ok := fm["tags"]
if ok {
if tags, ok := tagsIntf.(string); ok {
post.Tags = strings.Split(tags, ", ")
}
}
post.CoverImage = ""
covIntf, ok := fm["cover_image"]
if ok {
if coverImage, ok := covIntf.(string); ok {
post.CoverImage = coverImage
}
}
post.Series = ""
seriesIntf, ok := fm["series"]
if ok {
if series, ok := seriesIntf.(string); ok {
post.Series = series
}
}
pBody := contents[len(s[1])+(len(delimiter)*2):]
bf := blackfriday.Run(pBody)
bm := bluemonday.UGCPolicy()
bm.AllowAttrs("class").Matching(regexp.MustCompile("^language-[a-zA-Z0-9]+$")).OnElements("code")
post.PostBody = template.HTML(bm.SanitizeBytes(bf))
tm := template.Must(template.New("post").Parse(postTempl))
return tm, post, false
}
writeIndex() has changed a bit. Notice that we now take in a destination directory string and return an error. This is much nicer than just panicking on error. I have a bit more digging to do in working with my own error structs and better ways to surface and handle errors.
func writeIndex(idx index, destination string) error {
indexFile, err := os.Create(destination + "/" + "index.html")
if err != nil {
return err
}
defer indexFile.Close()
buffer := bufio.NewWriter(indexFile)
tm := template.Must(template.New("index").Parse(indexTempl))
err = tm.Execute(buffer, idx)
if err != nil {
return err
}
buffer.Flush()
return nil
}
func main() {
var idx index
Here we have the beginning part of our command line option code. Note that I'm using the standard library flag package and not something like Cobra. This keeps it very simple for now. Requiring only an option for a source directory and a destination directory. The source directory would be the location of all our markdown files. The destination is where we are going to save our generated HTML. Note, that I am still using panic() if we can't create the directory or if it already exists. I have decided it's better to fail out if the directory is already there for now. I have also been trying out log.Panicf() though in this iteration it only appears once. Later we could add switches for overwriting existing files, or deleting everything first, etc.
destination := flag.String("destination", "", "destination folder")
source := flag.String("source", "", "source directory")
flag.Parse()
if _, err := os.Stat(*destination); os.IsNotExist(err) {
err := os.Mkdir(*destination, 0777)
if err != nil {
panic(err)
}
} else {
log.Panicf("error: destination '%s' already exists", *destination)
}
_, err := ioutil.ReadDir(*destination)
if err != nil {
panic(err)
}
srcDir, err := ioutil.ReadDir(*source)
if err != nil {
panic(err)
}
We've also swapped out some of the panic()'s and continue's in our main loop. We don't want the process to fail out so logging any issues and continuing on is the way to go. continue exits the loop and "continues" on from the top.
for _, file := range srcDir {
if fileName := file.Name(); strings.HasSuffix(fileName, ".md") {
openedFile, err := os.Open(fileName)
if err != nil {
log.Println(fileName, err)
continue
}
contents, err := getContentsOf(openedFile)
if err != nil {
openedFile.Close()
log.Println(fileName, err)
continue
}
openedFile.Close()
s, err := splitData(contents, delimiter)
if err != nil {
log.Println(fileName, err)
continue
}
fm, err := parseFrontMatter(s[1])
if err != nil {
msg := fmt.Sprintf("%v\n", err)
log.Println(fileName, msg)
continue
}
Yes, I did leave a couple of panic()'s in place. If we can't create the output file or execute the template we come to a halt. I am assuming if one has a problem then it's better to not try and continue.
template, post, skip := makePost(fm, contents, s)
if !skip {
trimmedName := strings.TrimSuffix(fileName, ".md")
outputFile, err := os.Create(*destination + "/" + trimmedName + ".html")
if err != nil {
panic(err)
}
buffer := bufio.NewWriter(outputFile)
err = template.Execute(buffer, post)
if err != nil {
panic(err)
}
buffer.Flush()
outputFile.Close()
indexLinks := Page{
FileName: trimmedName + ".html",
Title: post.Title,
}
idx.Pages = append(idx.Pages, indexLinks)
}
}
}
I couldn't decide whether I wanted to error out if something went wrong with the index. For now, I've left it as a logline.
if len(idx.Pages) > 0 {
err := writeIndex(idx, *destination)
if err != nil {
log.Println(err)
}
}
}
Next Time
I haven't quite decided on what we are going to look at next time. I can continue building this out or we can try something new. I was thinking about writing a Go backend for a very old project I put together way back when. It was a PHP/HTML5 music player. I had ported part of it over to Glitch, which you can see part of at https://carnation-motion.glitch.me/. It may take a second to start up if the VM has shut down. I don't think any songs will play on that version but the basic UI is in place. We'll see where the mood takes me over the next week.
You can find the code for this and most of the other Attempting to Learn Go posts in the repo on GitHub.
shindakun
/
atlg
Source repo for the "Attempting to Learn Go" posts I've been putting up over on dev.to
Attempting to Learn Go
Here you can find the code I've been writing for my Attempting to Learn Go posts that I've been writing and posting over on Dev.to.
Post Index
| Enjoy this post? |
|---|
| How about buying me a coffee? |
Top comments (2)
Well this has been a very interesting series thus far.
I could recommend a few things which were good learning exercises for me early on:
Write a program to sort files within a folder by their extension
.txtin Documents,.jpgin Images etc...Write a batch picture downloader. Easily done against an api such as Reddit, for example, to collect all images in a page and save them somewhere.
Thanks! I certainly hope so!
I really like both of your ideas, I think I may have to see about implementing the first one at some point in the next couple of days. Would be good practice.