Advanced CLI Tool Development with JavaScript and OClif

In my last article, I used OClif and Node.js to build a CLI tool that allows users to auto-generate a brand new blog, with integration to ButterCMS. In this new installment, I’m going to show you how to extend that tool in order to add some new commands and improve the existing one.

What are we doing?

So, the idea now is to extend this buttercms-cli into a tool you can continue to use even after you’ve generated your blog’s code. You see, ButterCMS isn’t just a blog system, it also allows you to add extra pages to your site; such as “About Us”, “Meet our Team” and so on, and Butter allows you to completely customize these fields to your requirements.

And to capitalize on that, I’m going to show you how to modify the code from the last article, into something that also has the ability to add an interaction with these pages into your existing blogs.

Essentially what I’m adding is the following:

• Ability to add a single page into your blog, which will generate the right path, the right controller and yes, the right view, with properly formatted fields, depending on the type of content your page has.
• Ability to create a listing of pages of the same type (more on page types in a bit).

The additional feature I’m going to be adding that I want to highlight is support for an authentication token in all new and old commands. This will enable the CLI tool to communicate with ButterCMS to gather information about the page you’re trying to create, or even, auto-adding this token into the configuration file of the blog, allowing you to simply start the app right after you generate it without having to change a single line of code.

Even though the internal architecture of the tool is quite simple, let me help you visualize the extent of the changes we’ll introduce in this article. The following image exemplifies the existing relationship between the main components:

As I mentioned, it’s all quite simple, we have our Blog class, which extends OClif’s Command class, which in turn, uses our Generator class (which ironically extends Command).

 Now, we’re moving into something like the following:

You’ll notice that not a lot has changed in the sense that we still have, what are essentially commands using generators. That being said, we have added two new generators, and a base class for them to implement.

This new BaseGenerator class will simply define the mandatory methods each generator needs to implement and will make sure they’re implemented. That last bit needs to be done through a simple trick (you’ll see in a bit) since JavaScript does not provide the concept of Interface or a similar construct that forces mandatory methods to be implemented.

The end result

Let’s take a look now at how you’ll be able to use the command once we’re done with it.

The first one to change will be the Blog generator since it’s the easiest one to modify. Here’s how it will look:

**`$ buttercms-cli generate:blog --for=express --auth_token=`**

The change there is minimal, but once implemented, the generated code will already have your token in the configuration file, which is something you’re not getting with the current version.

Now, for the big changes, here is what the two new commands will look like:

If you want to add a new page to your blog, such as the About Us section many websites have, you can create it in the CMS and then, go to your favorite terminal, make sure you’re inside your blog’s folder (remember, that blog should’ve been created with the generate:blog command, or at least, follow the same folder structure, otherwise this command will not work) and enter:

**`$ buttercms-cli generate:page --for=express --page=about-us --auth-token=`**

That’s it! Once entered, it’ll ask for confirmation, just like before, and it’ll create the following:

-The route in your system: /about-us

-The controller associated with that route, under the routes folder

-The Jade template for that view. And this is where it gets interesting since the CMS allows you to create different types of fields, the template will not be generic (more on this in a bit).

Note that for the above command, the --page parameter takes the API slug for the page, this is something you can easily get from the UI of ButterCMS though, so don’t worry too much about it right now.

Finally, the third addition to the previous version of the tool is the ability to create a list of pages. To give you an example of the purpose of this, let’s assume you’re creating the “Meet our team” page, where you briefly introduce each of your team members and link each of them to a specific profile page. By creating these individual pages with a custom page type in Butter (say, Team member page type), you can simply use the following command to create a listing page:

**`$ buttercms-cli generate:pages --for=express --page_type=team_member --auth_token=`**

The above line would create:

-The route for your listing page

-The controller that would handle that request

-And finally, the template view for it as well. In this particular case, since it’s a set of links, this template will not be customized (there is nothing to customize here, though).

Custom field types

Now that we’ve covered what the new features will look like, let me give you a quick overview of the custom field types you can create in a new page, by using the current version of the UI.

The list is basically the following:

Short Text: Pretty self-explanatory, you’ll be able to enter a quick sentence or two into this field and it should be rendered as such in your page.

Long Text: Nothing much to explain here, it’ll be rendered as a p block.

WYSIWYG: This is a funny name if you haven’t encountered it before, but it basically means “What You See Is What You Get” and it’s a reference to the fact that you’ll be able to visually format the content inside this field (think about it as a mini Google Docs or Word inside a text field). The content you enter into it will be rendered formatted in your page.

Reference: These are simply links to other pages inside your site. You can link to a single page or link to a set of them. Depending on which type of field you choose, you’ll either get a single link rendered or a list (an actual ul list) of them.

Date: Date fields are pretty straightforward, and they’ll be rendered as an input field of type datetime, with the default value set to whatever you’ve entered. This will create a nice browser-specific interaction once you click on that field, showing a calendar to change the value.

Dropdown: This field allows you to enter multiple options, and once you create the page, you can select one. The selected option will be rendered a simple text in your page.

Image field: This type of field allows you to upload an image file. Sadly, since the API does not provide field types when requesting page content, on the page you’ll have the actual URL rendered as text.

True/False: This option represents a simple boolean check, and it’ll be rendered as such in your page (a checkbox field that will either be selected or not).

Number: This field allows you to enter a single numeric value. Once rendered, it’ll appear as simple text.

HTML: Much like the WYSIWYG field, this one allows you to format the content you enter, but you can do so by adding HTML tags. The content you enter here will be rendered in your page by interpreting the HTML and showing the end result.

Repeater: The repeater field allows you to add lists of the previously mentioned fields (i.e adding a list of short text fields). Whatever you configure your lists to be, they’ll be rendered as a ul element and each set of fields as an individual li item.

That’s an overview of all the types of fields you can add, and a rundown on how they’ll be treated by our CLI tool.

Let’s now move on to the actual implementation to understand how to achieve this.

Implementation

Since I’ve added several files for each new command added into the CLI tool and several hundred new lines of code, I don’t really want to bore you with the entire set of changes. Instead, I want to go over the highlights of what I had to do, in order to get the above-mentioned features into our ButterCMS-cli tool.

Remember that you can browse the entire source code directly from GitHub.

Adding support for the --auth_token flag

This is the easiest one since we already had the generate:blog command ready. The changed code for our command looks like this:

'use strict'

const {Command, flags} = require('@oclif/command')
const requiredir = require("require-dir")
const BlogGenerators = requiredir("../../generators/blog")

class BlogCommand extends Command {

    async run() {

     const {flags} = this.parse(BlogCommand)

     const targetGeneration = flags.for.toLowerCase().trim();

     //error handling
     if(BlogCommand.flags.for.options.indexOf(targetGeneration) == -1) {
         return this.error (`Target not found '${targetGeneration}', please try one of the valid ones - ${BlogCommand.flags.for.options.join(",")} - `)
     }

     const gen = new BlogGenerators[targetGeneration](flags.auth_token)

     gen.run();

    }
}

BlogCommand.flags = {
    for: flags.string({
     description: 'Target destination for the generator command',
     options: ['express'] //valid options
    }),
    auth_token: flags.string({
     description: "Your AUTH token used to communicate with ButterCMS API",
     required: true
    })
}

module.exports = BlogCommand

Yes, that’s it, by adding our flag into the static _flags _object, we now can have OClif check its existence for us. All we gotta do is pass it along to the generator, so it can use it as follows:

/*
    Create the destination folder using the application name given,
    and copy the blog files into it
    */
    copyFiles(appname) {
     const folderName = this.cleanAppName(appname)
     fs.mkdir(folderName, (err) => { //create the new folder
         if(err) {
             return this.log("There was a problem creating your blog's folder: " + chalk.red(err.toString()))
         }
         this.log("Folder - " + chalk.bold(folderName) + " -  " + chalk.green("successfully created!"))
         ncp(SOURCEEXPRESSBLOG, folderName, (err) => { //copy all files
             if(err) {
                 return this.log("There was a problem while copying your files: " + chalk.red(err))
             }
//  This is the comment text
             let configFilePath = folderName + "/config/default.json"
             fs.readFile(configFilePath, (err, configContent) => { //overwrite the configuration file, with the provided AUTH KEY
                 let newConfig = configContent.toString().replace("", this.auth_token)
                 fs.writeFile(configFilePath, newConfig, (err) => {
                     this.printSuccessMessage(folderName)
                 })
             })
         })
     })
    }

There under the comment text is the trick, after copying the files for our brand new blog, by default we have our configuration file created, but it contains a placeholder “****” string where your Authentication token should be. Now with the addition of the code under the comment text, we read it, replace that string with whatever auth token you gave the tool, and save it again. Simple, quick change. The rest of the file remains pretty much the same, so there is nothing noteworthy to mention.

Adding the new BaseGenerator class

The code for this one is really simple, the only interesting bit about it and why I’m showing it here, is the way you can “force” your developers to implement methods. Remember that even though we now have the _class _construct in JavaScript, we’re still really far away from being a strong OOP language. As such, we don’t have constructs like Interfaces, which would allow you to fix a basic set of methods which every class would have to implement in order to comply.

Instead, we can achieve that, by doing the following dirty trick:

'use strict'

const {Command} = require('@oclif/command')

module.exports = class BaseGenerator extends Command{

    prompts() {
     throw new Error("::Base Generator - prompts:: Needs to be implemented")
    }    

    execute() {
     throw new Error("::Base Generator - execute:: Needs to be implemented")    
    }

    async run() {
     this
         .prompts() //ask the questions
         .then(this.execute.bind(this)) //execute the command
    }
}

The _run _method for all generators is the same, you prompt the user for a set of questions and a final confirmation, then you execute the actual code which takes care of creating whatever it is you need. So the _prompt _and the _execute _methods are the ones you need to implement, or in this case, overwrite. If you don’t, you’ll get a new exception thrown your way.

It might be a quick and dirty fix, but it works and you need to make sure that whenever creating new and future generators, you don’t forget about the important methods.

The new commands

For this, I’ll only show the code for the generate:pagecommand, since it’s the most complex of the two, especially due to the custom field types I mentioned above. Again, you can check out rest of the code in the repository.

As I previously mentioned, the command files all look alike, so instead, I’ll focus on the generator file, since there is where all the interesting logic resides.

For this one, the execute method looks like this:

execute(answer) {
     if(!answer.continue){
         return this.log("OK then, see you later!")
     }

     this.getPageData( (err, pageData) => {
         if(err) {
             return this.log("There was a problem getting the data for your page: " + chalk.red(err.data.detail))
         }
         this.copyFiles(pageData.fields)
     })

    }

We’re basically getting the page’s data, which essentially is the list of fields inside it, and then we’re copying the files. The files to be copied are inside the src/generators/[command]/express-template folder like we did before. Only for this one, we have 2 files, the controller inside the routes folder, that looks like this:

var express = require('express');
var router = express.Router();
var config = require("config")
var butter = require('buttercms')(config.get("buttercms.auth_token"));

router.get('', function(req, res, next) {
    butter.page.retrieve("*", "[[SLUG]]").then( (resp) => {
     res.render('[[SLUG]]', resp.data.data);
    })
    .catch(console.err)
});

module.exports = router

Note that the route is not set in this file because if you take a look at the _app.js _file generated by the _generate:blog _command, you’ll find the following lines of code:

//...
const routes = requiredir("./routes")
//....
Object.keys(routes).forEach( route => {
    console.log("Setting up route", route)
    app.use("/" + route, routes[route])
})

The requiredir command will automatically require everything inside the routes folder and return an object, with keys equal to the file names. Since those files are named after the slug, you don’t need to worry about setting up the specific route for the second part of the code (that forEach loop, will correctly create the routes and associate the right controller to them)

And the view file, inside the _views _folder, that looks like this:

extend layout

block content
[[FIELDS]]

Most of this one will be dynamically generated as you can see.

Notice the [[SLUG]] and [[FIELDS]] tags in both of them, these are placeholder tags I made up that will be replaced by our code in a bit.

Now, I wanted to show the _getPageData _method, since it’s interacting with ButterCMS’s API through the ButterCMS npm module, like this:

getPageData(cb) {

     let butterClient = new ButterCMS(this.auth_token)
     butterClient.page.retrieve("*", this.page).then( resp => {
         cb(null, resp.data.data);
     }).catch(cb)
    }

Not a lot of code there, but by using the _page.retrieve _method, we can get what we want. That method takes the page’s type and the page’s slug as parameters, but we don’t really need the type, so we can simply pass an “*” instead.

Now, for the _copyFiles _method:

copyFiles(fieldsToRender) {

    let finalViewName = './views/' + this.page + '.jade'
    let finalRouteName = './routes/' + this.page + '.js'

    this.generateTemplate(fieldsToRender, (err, content) => {
     fs.writeFile(finalViewName, content, (err) => { //save the view template to its destination
         if(err) {
             return this.log("There was a problem saving the view file at '" + finalViewName + "': " + chalk.red(err.toString()))
         }

         this.generateRouteFile( (err, routeContent) => {
             fs.writeFile(finalRouteName, routeContent, (err) => {
                 if(err) {
                     return this.log("There was a problem copying the route template: " + chalk.red(err.toString()))
                 }
                 this.printSuccessMessage();
             })
         })
     })

    })    
}

This one is:

1. Generating template content (in other words, filling in the fields inside the jade file)
2. Saving it into the correct path
3. Generating the routes file (by replacing the [[SLUG]] tag inside the code)
4. Saving it into the right path
5. Finishing by printing the success message.

From those 5 steps, the most important one I want to cover is the first one, generating template content. Here is what that code looks like:

generateTemplate(fields, cb) {
    fs.readFile(SOURCE_VIEW_PATH, (err, viewTemplate) => {
     if(err) return cb(err);
     let newContent = []

     newContent = Object.keys(fields).map( (field) => {

         if(Array.isArray(fields[field])) { //if it's an array, then we need to add a loop in the template
             let firstRow = fields[field][0]
             let content = 'ul'
             content += OS.EOL + '\teach f in fields.' + field    
             if(firstRow.slug && firstRow.fields) { //it's a reference
                 let item = ['\t\t', 'li','\ta(href="/" + f.slug)=f.slug'].join(OS.EOL + "\t\t")
                 content += item
             } else {
                 content += [OS.EOL + "\t\tli",
                             OS.EOL + "\t\t",
                             Object.keys(firstRow).map( k => {
                                 return this.guessRepresentation(firstRow, k, 'f')
                             }).join(OS.EOL + "\t\t")].join('')
             }
             return "\t" + content
         }
         return this.guessRepresentation(fields, field)

     })

     viewTemplate = viewTemplate.toString().replace("[[FIELDS]]", newContent.join(OS.EOL))
     cb(null, viewTemplate)
    })
}

It might look like a lot, but it just iterates over the list of fields passed as parameters, and for each one, it tries to guess its representation (remember, the API does not return the actual type, so we need to interpret its content in order to guess correctly). There is a special section inside the loop for both reference fields (when they are referring to a list of links) and repeater fields.

But in essence, the code goes over each field, trying to get its representation and adding it into an array of lines, called _newContent _, which in the end, gets joined together and that final string gets to replace the [[FIELDS]] tag we saw earlier.

A little detail to note here as well is the use of the _OS.EOL _variable, which comes from the _os _module. This represents the current Operating System’s End Of Line character. Remember that Windows, Linux, and Mac have somewhat similar but not exactly the same ideas of what that means, so if you’re going to be sharing code that makes use of EOL characters, make sure you use that variable so it works as expected everywhere.

The last bit of code I wanna show you here is the _guessRepresentation _method, which tries to understand how to render each field, based on its content:

guessRepresentation(fields, field, prefixValue) {
    if(!prefixValue) prefixValue = "fields"
    if(typeof fields[field] === 'boolean') { //true/false
     return '\tinput(type="checkbox"  checked=' + prefixValue + '.' + field +' name="' + field + '")'
    }
    if(typeof fields[field] === 'string') {
     if(fields[field].match(/[0-9]{4}-[0-9]{2}-[0-9]{2}T/g)) { //dates
         return '\tinput(type="datetime-local" value=' + prefixValue + '.' + field + ')'
     }
     if(fields[field].match(/<[a-z][\s\S]*>/i)) { //HTML 
         return '\tp!=' + prefixValue + '.' + field
     }
    }

    if(fields[field].slug) { //reference
     return '\ta(href="/" + ' + prefixValue + '.' + field + '.slug)=' + prefixValue + '.' + field + '.slug'
    }

    return '\tp=' + prefixValue + '.' + field //default representation (plain text)
}

In the end, you need to remember: the generated template view will be there, inside the _views _folder for you to edit, so this is just a guideline to help you get started.

Extending the tool

If you wanted, you could easily extend the CLI tool into supporting new target tech stacks (like adding support for Vue.JS) or even adding new commands altogether. In order to do that, here are the steps:

• If you want to add new commands, you need to create them inside the "src/commands" folder. Since this is how OClif works we can’t really do anything about that. If you want further sub-commands inside the "generate", they need to be inside the "generate" folder.  Again, OClif forces this standard into the project.
• As for adding further targets, the way I created the folder structure means that you'll need to add a generator file inside the "src/generators/" folder, named after your target (i.e if you wanted to add support for VueJS, you would create a vuejs.js file inside the “src/generators/generate” folder). That is because every command does a requiredir of its generators. Basically, generate:blog _does a _requiredir of src/generators/blog _, _generate:pages _of _src/generators/pages _and so on. The _requiredir call requires EVERY file inside the folder, creating an object with keys equal to the file names (minus the .js). So if you have 3 files inside the _src/generators/blog _called "express", "vue" and "next", you can perfectly do _generate:blog --for=vue _and it'll automatically pick it up for you. The beauty of this structure is that in order to add new generators, you don't need to touch the existing commands.

Finally, the folder structure for the templates though, that is entirely up to each generator, so that can easily be changed and customized into whatever you feel required.

Summary

That’s it for this article, I hope you managed to get something useful out of it. OClif is certainly a very interesting module and we’re still not using 100% of its features, so there is probably a lot more we can do with it in order to make this CLI tool grow and help you even more. Let me know in the comments below what you would like to see added to it!

And remember that you can look at the full version of the source code, directly from GitHub!

Sign up to make sure you miss out on our other tutorials.