DEV Community: Eugene Dorfling

How to publish a programming tutorial

Eugene Dorfling — Tue, 08 Dec 2020 09:41:46 +0000

Introduction

Programming tutorials consist of three things: text, images, and code. Text should be readable, images should be pretty, and code should have syntax highlighting. It is way more time-consuming than it sounds: converting Markdown to HTML, you often find yourself spending more time trying to perfect the look and feel than writing the content.

Usually, If you want to write a programming tutorial and share it with the world your options are:

Upload it to managed publishing platforms like Medium, give up control and let them bloat your content with trackers, paywalls, and dark patterns. Have a look at this article for a comparison of the top publishing platforms.
Spend a few hours or days yourself setting up something like Hugo with GitHub pages and trying to get the theme to work just right.
Use something super minimalistic but not to everyone’s aesthetic taste like bearblog.dev.

As a fourth option, you can use this Opinionated Tutorial Publisher in three steps.

Write your tutorial in Markdown.
Run a curl command or upload your Markdown on this page.
Upload the resulting single HTML file anywhere you want.

Your tutorial will look like this example and get good lighthouse scores as it's lightweight and simplistic. With syntax highlighting it will add about 25KB of "bloat" on top of your Markdown.

In this tutorial, we will build the Tutorial Publisher application that will convert a Markdown file into a beautiful but simplistic HTML page that you can host anywhere. We will build a Flask front end for your application and we will show how you can add PrismJS syntax highlighting as well as change the CSS template.

The final Python application will consist of the following:

Flask for serving a simple web interface.
Water.css for basic CSS styling (default CSS).
PrismJS for syntax highlighting.
Pandoc for the Markdown to HTML conversion with a Pandoc filter to make the HTML output play nicely with PrismJS.

This project repository is also available here on GitHub.

Setting up the environment

Installing Flask

First, we will build a Flask web application, which will serve a basic HTML page that will allow us to upload our Markdown file.

Flask is a micro web framework for Python. Flask is one of the quickest ways you can get a web application up and running, hence its popularity among web developers. It is not only great for small POC (proof of concept) projects but also capable of running large-scale production solutions. We will use Flask in this tutorial to serve a simple HTML page where we can interact with our tutorial publisher.

Install Flask by typing pyhton3 -m pip install flask into the command line.

$ python3 -m pip install flask
Collecting flask
Using cached https://files.pythonhosted.org/packages/f2/28/2a03252dfb9ebf377f40fba6a7841b47083260bf8bd8e737b0c6952df83f/Flask-1.1.2-py2.py3-none-any.whl
<<<...some output omitted...>>>
Installing collected packages: itsdangerous, Werkzeug, click, MarkupSafe, Jinja2, flask
Successfully installed Jinja2-2.11.2 MarkupSafe-1.1.1 Werkzeug-1.0.1 click-7.1.2 flask-1.1.2 itsdangerous-1.1.0

From the output above we can see that Flask and its dependencies were installed successfully.

Installing Pandoc

Pandoc is a command line tool for document format conversion and also needs to be installed before we can use it.

Follow the instructions for your operating system.

Creating your project directory

Create a file called app.py within your project directory.
Download assets.zip and extract the contents to your project directory. This includes: fix-pre-code.lua, template_pre.html and template_post.html.

On Linux you can copy the following command and run it from within your project directory. This will download and extract the assets folder to the current directory.

wget https://i.ritzastatic.com/tutorial-publisher/assets.zip && unzip -o assets.zip

You should now have the following within your project directory:
```
- project directory/ - assets/ - fix-pre-code.lua
                               - template_pre.html
                               - tempalte_post.html
                     - app.py
```
Depending on your extraction method assets.zip could also be in your project directory

Building the Flask web application

Now that we have our environment set up, let's build the web server. Open the app.py file and add the following code:

from flask import Flask, request

app = Flask(__name__)

@app.route('/', methods=['GET', 'POST'])
def publish_tutorial():
    return '''
        <!doctype html>
        <title>Upload new file</title>
        <h1>Upload new file</h1>
        <p>Upload a Markdown file and get a beautiful HTML file with code highlighting in return</p>
        <form method=post enctype=multipart/form-data>
        <input type=file name=file>
        <input type=submit value=Upload>
        </form>
        '''

if __name__ == "__main__":
    app.run("0.0.0.0")

In the above code, we import the Flask libraries, create the Flask app and create the app route. Within the app route (triggered from the root dir '/') we define a function that will return a simple HTML page with two inputs: one for uploading the file and one for submitting it to the tutorial publisher. In the end, we have an if statement that will start the web server when you run the app.py script.

Now run the app by typing python3 app.py in the terminal from within your project directory.

$ python3 app.py
 * Serving Flask app "app" (lazy loading)
 * Environment: production
   WARNING: This is a development server. Do not use it in a production deployment.
   Use a production WSGI server instead.
 * Debug mode: off
 * Running on http://0.0.0.0:5000/ (Press CTRL+C to quit)

From the above output, we can see that our web app is now running and is listening for requests on any (0.0.0.0) IP on port 5000.

Open your browser and navigate to 127.0.0.1:5000. This should render the HTML page like in the image below.

We have now built our web server with a basic HTML page. The buttons don't do anything at the moment so let's add some functionality to them.

Uploading files

Now we will write the code that allows us to upload a Markdown file. In order to do this we will need to import some more libraries.

Add the following code under from flask import Flask, request to import the rest of the libraries needed.

from flask import send_from_directory
from flask import redirect
from werkzeug.utils import secure_filename

import os
import random
import string
import subprocess

Before we write the code that will upload a file to the server we need to add the following code that will ensure the uploaded file is a Markdown file. Add the following code under your imports.

ALLOWED_EXTENSIONS = {'md', 'markdown'}

def allowed_file(filename):
    return '.' in filename and filename.rsplit('.', 1)[1].lower() in ALLOWED_EXTENSIONS

This code will check that the uploaded file is a Markdown file and correctly format it for later use.

Add the following code to your app.py file right after def publish_tutorial():.

    if request.method == 'POST':
        if 'file' not in request.files:
            return redirect(request.url)
        fle = request.files['file']
        if fle.filename == '':
            return redirect(request.url)
        if fle and allowed_file(fle.filename):
            filename = secure_filename(fle.filename)
            save_name = ''.join(random.sample(string.ascii_letters, 6))
            out_noex = os.path.join("/tmp/uploads", save_name)
            out_md = out_noex + ".md"
            out_html = out_noex + ".html"
            out_html_final = out_noex + "f" + ".html"

            fle.save(out_md)

In the code above we check for a POST request. When we hit the submit button from our front end it sends a POST request to our back end (main.py). The first line of code catches that request and executes the code within the if statement. The next two if statements check that there is a file in the request and that the filename is not empty. If any of these two checks fail we redirect back to the home page. (Note: NameError: name 'redirect' is not defined.)

In the last if statement, we check that the file is a Markdown file, use secure_filename to return a secure version of the filename that we can safely store on the file system and pass it to os.path.join, create a temp name for the files with 6 random ASCII letters, add the file extensions to each file (".md", ".html", "f.html"), and save the Markdown file.

We can now use our app to upload a Markdown file to the server. In this case, the server saves the uploaded file in tmp/uploads. Note: you have to create the uploads directory on your server manually. We use the tmp/ directory as we temporarily save a few files in order to build the final file. The tmp/ directory gets cleaned after each reboot ensuring we don't pile up these temporary files. This also means however that the uploads directory will be removed after a reboot and you will have to recreate it on system startup.

Getting a title and metadata from the uploaded file

From here let's use the uploaded file and get a title and meta description from it. To accomplish this, add the following code to your app.py file under the line fle.save(out_md).

            title = ""
            metadescription = ""
            try:
                with open(out_md) as f:
                    head = [next(f) for x in range(10)]
                    head = [x for x in head if len(x) > 2]

                    title = head[0].replace("#", "").strip()
                    metadescription = ' '.join(head[1:3])[:150]
            except Exception as e:
                print(e)

For SEO and general readability, it’s good to have a title and meta description saved in your HTML file. We don’t want to overcomplicate our program by making the user define these, but we can make a "guess" at them by taking the first line that contains content as the title and the first 150 characters as the meta description. The code above does exactly this.

We specify two empty variables: one for the title and one for the meta description. We then open out_md (which is the saved Markdown file) and we loop through the first 10 lines to create a list called head and then remove all the blank lines from it. We then take the first item [0] from the head list, remove the #, remove leading and trailing white spaces, and add it to the title variable. We then take the rest of the head from the second [1] item until the third item ([1:3], excluding the fourth item [3]), limit it to 150 characters and add it to the metadescription variable.

Converting Markdown to HTML with Pandoc

We currently have a program that can get a Markdown file through a web page, save it to the system and get a title and meta description from it.

Now comes the conversion of the content from Markdown to HTML. We can accomplish this with Pandoc, a universal document converter. There are a lot of Markdown converter libraries out there but none of them works quite as well as Pandoc and it can convert other formats too, a lot of them.

By default Pandoc will mark up code blocks like this: <pre class="*"><code>. PrismJS follows the recommended way to mark up a code block by having a <pre> element with a <code> element inside it like this: <pre><code class="language-*". So in order to get PrismJS working with Pandoc we will add a Pandoc filter. Pandoc reads the input file and writes an output file. Between these steps Pandoc allows us to add a filter that the input file will pass through before it writes the output file.

INPUT --reader--> AST --filter--> AST --writer--> OUTPUT

This filter can be written in Lua or JSON. We are using a Lua filter as it has some advantages over JSON and is usually slightly faster.

The fix-pre-code.lua file in your assets directory is the Lua filter that finds and converts the Pandoc code block markup to the PrismJS supported markup.

Now we can add the code that pulls the Pandoc filter into our code for later use. Add the following code under print(e). Make sure that your indentation is correct so that this code falls within the if fle and allowed_file(fle.filename): block.

            assets_path = os.path.join(os.getcwd(), "assets")
            pandoc_filter = os.path.join(assets_path, "fix-pre-code.lua")

In the code above, we specify the assets_path which points to the assets directory. Then we specify the pandoc_filter by adding the fix-pre-code.lua file.

Now we are ready to convert the Markdown content with Pandoc. Add the following code under pandoc_filter = os.path.join(assets_path, "fix-pre-code.lua"), also checking that it falls within the if fle and allowed_file(fle.filename): block.

            try:
                output = subprocess.check_output([
                    'pandoc',
                    out_md,
                    '--no-highlight',
                    '-f', 'markdown-auto_identifiers-citations',
                    '-t', 'html',
                    '--lua-filter', pandoc_filter,
                    '-o', out_html
                    ])
            except subprocess.CalledProcessError as e:
                print(e.output)

In this code, we use the subprocess library to run a command line command pandoc and pass a list of parameters: out_md is the input file, Pandoc syntax highlighting is disabled, the "from" format is markdown-auto_identifiers-citations, the "to" format is html, pandoc_filter is the Lua filter, and the output file is out_html.

Building the final HTML file

Now that we have the Markdown content converted to HTML we can add it to our HTML template which includes a basic CSS theme.

We have two HTML templates: one to prepend the top part to the content, called template_pre.html, and one to append the last part, called template_post.html. Together these will form the complete HTML page.

The template_pre.html file contains the HTML head with metadata and the start of the body element. The template_post.html file contains the CSS styling and the end of the body element. The post template is also where we will add PrismJS syntax highlighting.

Let's add the code that will combine all three HTML files into the final HTML page. Add the following code under print(e.output), checking that your indentation falls within the if fle and allowed_file(fle.filename): block.

            try:
                # open the templates to prepend and append to the file
                template_pre = os.path.join(assets_path, "template_pre.html")
                template_post = os.path.join(assets_path, "template_post.html")
                with open(template_pre, encoding="utf-8") as f:
                    pre = f.read()
                    pre = pre.replace("{title}", title)
                    pre = pre.replace("{metadescription}", metadescription)
                with open(template_post, encoding="utf-8") as f:
                    post = f.read()

                with open("/tmp/uploads/{}.html".format(save_name), encoding="utf-8") as f:
                    content = f.read()
                complete = pre + content + post
                with open(out_html_final, "w", encoding="utf-8") as f:
                    f.write(complete)

                return send_from_directory("/tmp/uploads/", save_name + "f" + ".html" , as_attachment=True)

            except Exception as e:
                print(e)
                return "Sorry :( something went wrong."

In the code above we specify the locations of the templates with variables, open and read the pre template to add the title and meta description. We then load the post template into our program as well as the content (Pandoc converted HTML). We then combine the three and save the final (complete) HTML doc to the server. Lastly, we return the final document for download through our Flask front end.

That's it for app.py; the whole file should now look like this.

from flask import Flask, request
from flask import send_from_directory
from flask import redirect
from werkzeug.utils import secure_filename

import os
import random
import string
import subprocess

ALLOWED_EXTENSIONS = {'md', 'markdown'}

def allowed_file(filename):
    return '.' in filename and filename.rsplit('.', 1)[1].lower() in ALLOWED_EXTENSIONS

app = Flask(__name__)

@app.route('/', methods=['GET', 'POST'])
def publish_tutorial():
    if request.method == 'POST':
        if 'file' not in request.files:
            return redirect(request.url)
        fle = request.files['file']
        if fle.filename == '':
            return redirect(request.url)
        if fle and allowed_file(fle.filename):
            filename = secure_filename(fle.filename)
            save_name = ''.join(random.sample(string.ascii_letters, 6))
            out_noex = os.path.join("/tmp/uploads", save_name)
            out_md = out_noex + ".md"
            out_html = out_noex + ".html"
            out_html_final = out_noex + "f" + ".html"

            fle.save(out_md)

            title = ""
            metadescription = ""
            try:
                with open(out_md) as f:
                    head = [next(f) for x in range(10)]
                    head = [x for x in head if len(x) > 2]

                    title = head[0].replace("#", "").strip()
                    metadescription = ' '.join(head[1:3])[:150]
            except Exception as e:
                print(e)

            assets_path = os.path.join(os.getcwd(), "assets")
            pandoc_filter = os.path.join(assets_path, "fix-pre-code.lua")

            try:
                output = subprocess.check_output([
                    'pandoc',
                    out_md,
                    '--no-highlight',
                    '-f', 'markdown-auto_identifiers-citations',
                    '-t', 'html',
                    '--lua-filter', pandoc_filter,
                    '-o', out_html
                    ])
            except subprocess.CalledProcessError as e:
                print(e.output)

            try:
                # open the templates to prepend and append to the file
                template_pre = os.path.join(assets_path, "template_pre.html")
                template_post = os.path.join(assets_path, "template_post.html")
                with open(template_pre, encoding="utf-8") as f:
                    pre = f.read()
                    pre = pre.replace("{title}", title)
                    pre = pre.replace("{metadescription}", metadescription)
                with open(template_post, encoding="utf-8") as f:
                    post = f.read()

                with open("/tmp/uploads/{}.html".format(save_name), encoding="utf-8") as f:
                    content = f.read()
                complete = pre + content + post
                with open(out_html_final, "w", encoding="utf-8") as f:
                    f.write(complete)

                return send_from_directory("/tmp/uploads/", save_name + "f" + ".html" , as_attachment=True)

            except Exception as e:
                print(e)
                return "Sorry :( something went wrong."

    return '''
        <!doctype html>
        <title>Upload new file</title>
        <h1>Upload new file</h1>
        <p>Upload a markdown file and get a beautiful HTML file with code highlighting in return</p>
        <form method=post enctype=multipart/form-data>
        <input type=file name=file>
        <input type=submit value=Upload>
        </form>
        '''

if __name__ == "__main__":
    app.run("0.0.0.0"

Adding PrismJS to the post template

For syntax highlighting we will use PrismJS, a lightweight and extensible syntax highlighter.

TIP: Don't include syntax highlighting into your HTML template if your tutorial doesn't have code snippets as you don't want to include unnecessary overheads to your final file.

Download PrismJS from here.

A customization page will open where you can customize the look of the syntax highlighting as well as choose languages and plugins that you want to include. Note: the more languages and plugins you choose the larger your final HTML page will be.

Choose compression level: Minified version.
Choose your theme, in our case Okaidia.
Choose the languages you want it to support.
Choose plugins. In this tutorial publisher we don't use any plugins.

After you have selected all that you want to include you'll find the JS and CSS files at the bottom of the page.

First copy all of the CSS content, open minifier.org, paste and minify the CSS.

Now open the template_post.html file in the assets directory.

Add the following under the last </style> and paste the minified CSS between <style type="text/css"> and </style>.

<style type="text/css">

</style>

Now copy the PrismJS content (already minified) from the PrismJS download page. Create the following under the last </style> (created above) and paste the minified PrismJS between <script type="text/javascript"> and </style>.

<script type="text/javascript">

</script>

Great! Now we have included PrismJS syntax highlighting to our HTML template.

Run your app again with python3 app.py and navigate to 127.0.0.1:5000 in your browser.

Upload your Markdown formatted document and hit the submit button. A download screen will pop up where you can save your HTML file.

Open your HTML file in your browser. You should now be looking at a pretty HTML version of your tutorial. You can host this page anywhere you want. Hosting is pretty complicated, but it’s easier if you just want to host one HTML file. You can push it to GitHub pages, scp it to a VPS running Apache or nginx, ask your Internet Service Provider for a static hosting package, or use one of the many free hosting services online. You can put it in AWS S3, use Netlify, or Repl.it.

Changing the CSS template

If you don't like the default CSS theme you can swap it out for another or your own custom one.

To do this simply open up the template_post.html file and replace the CSS between the second <style type="text/css"> and </style> tags where it says :root.... Remember to minify your CSS before adding it.

Conclusion

Congratulations! You have built your own tutorial publisher that'll save you a lot of time. You can now write your tutorials in Markdown and use this publisher to create a beautiful HTML page that you can host anywhere. Writing the tutorials is the hard part though, so we'll leave that up to you. Happy publishing!

DEV.to vs Medium vs Hashnode vs Hackernoon

Eugene Dorfling — Fri, 20 Nov 2020 14:14:24 +0000

Introduction

So, you want to publish tutorials or technical articles, whether it’s for yourself or for a brand you manage. Creating and hosting your own blog from scratch is no easy task; luckily there are pre-built and managed platforms where you can publish your work without having to run your own site.

These platforms mostly let you create an account and start writing. From there they handle distribution and take care of all the other platform management tasks.

If you are running your own blog already, it is still a good idea to take advantage of the large communities of these platforms to redistribute your articles and gain more exposure.

There are a plethora of blogging platforms to choose from, each with their own advantages and disadvantages. In this article, we aim to make the decision a little easier by comparing the top blogging platforms for developers.

Just tell me which to use

If you just want to know which to use and get on with it you should probably use:

Dev.to if you want to exchange knowledge and experience with the largest active developer community whether you write professionally or not.
Hashnode if you want to completely customize your blog page to represent your brand and link your own domain.
Hackernoon if you want to work with a professional team of editors and publish to a platform that only accepts high-quality content.
Medium if you want to write about general topics and monetize your work.

Quick overview

Before we dive into detailed comparisons, here’s a quick overview of each platform.

DEv.to

Dev.to is one of the largest online communities of software developers. It is a place where developers and aspiring developers meet to share their knowledge and stories. They don’t have paywalls or adverts, but instead make their revenue from sponsors, listings and the DEV shop.

Their text editor uses Markdown formatting with built-in syntax highlighting which makes it easy to embed code snippets, tables and other media. They also have a public API that developers can use to automate their publishing workflow.

Medium

Medium is an online publishing platform for all kinds of writers and topics. They have a clean look and feel with an easy-to-use text editor. It is a great place for writers to share their content and monetize their articles. They have a very large reader base so with consistent writing and submission to publications your target audience will find you.

As Medium caters to all types of writers their text editor is plain and simple; however, because they don’t support Markdown and syntax highlighting it is not the best place for developers to write technical articles where code snippets or tables are needed. They don’t allow API integration, so for redistribution your only route is using their import tool which is much like copy-pasting as you’ll always have to manually tweak the article to work in the Medium editor.

They opted for a revenue model where readers have to pay a monthly subscription fee to read articles. This is great if you are writing for an income but it’s not so great if you merely want to freely share knowledge.

Hashnode

Hashnode is a free blogging platform and community of developers that enables you to publish articles on your domain with a custom blog page. This is a great place to start your personal blog as a developer because you get traffic to your own domain, growing your brand, while your articles get distributed to the Hashnode developer community.

Hashnode allows you to completely customize your blog page with built-in features, widgets and integrations. They have also released a custom CSS feature that will allow you even more flexibility as to the look and feel of your blog page.

It’s easy to sign up and get started with a custom blog and they promise to be free forever. They don’t support adverts or have a paywall of any kind. They have an easy to use text editor that supports Markdown so code embeds and syntax highlighting is not a problem. They are working on a public API that will enable developers to automate their publishing workflow and they have a GitHub integration where a Markdown version of your article will be pushed to your repo when hitting the publish button.

Hackernoon

Hackernoon is a technology publishing service that focuses on topics such as software development, startups, artificial intelligence and cryptocurrencies. They originally started as a publication on Medium but decided to move away when Medium adjusted their business model. They have a similar text editor to Medium that does not support Markdown, making it difficult to embed things like tables, and it doesn’t support syntax highlighting.

Hackernoon has a lengthy signup process where you choose to write either as an individual or as a brand. The individual option allows you to publish articles for free whereas publishing as a brand will cost you $199 per published article.

You will have to jump through a few hoops and work with editors but because of this Hackernoon has higher quality content that will help you gain traction as a professional writer when you get published. All articles are subject to approval by their editors before publishing and the process takes up to 4 days.

Dev.to vs Medium

Dev.to is an online community of developers sharing their developer journey from complete beginners to experts through articles, blog posts and discussions while Medium is a publishing platform for all kinds of writing where short, opinionated posts seem to be prioritized over more lengthy technical articles.

Consider Dev.to if you are a developer of any caliber who wishes to connect with other developers whether through sharing knowledge, learning or taking part in discussions.
Consider Medium if you want to write more creative or opinionated articles on various topics and want to monetize your work.

Dev.to vs Hashnode

Dev.to and Hashnode are both blogging platforms that have large developer communities. However, Dev.to is an open-source blogging platform that you can use to build your own (although most people just sign up for an account and publish on the dev.to domain). Hashnode is a proprietary blogging platform that allows you to easily build your own blog page with custom CSS and link it to your own domain name.

Consider Dev.to if you want to be part of an open community of developers and publish content on the fast-growing dev.to domain.
Consider Hashnode if you want to publish on your own domain while still being able to distribute your content to a developer-focussed community.

Dev.to vs Hackernoon

Dev.to is an open community where developers can write about anything they wish: they can write technical articles, how-to guides and even start discussions. Hackernoon is a developers’ publication that migrated from Medium to their own platform. They don’t allow you to just post what you like: they are a publication so you submit your articles for review by their editors; once approved your article will be published to their reader base.

Consider Dev.to if you want an easy place to write and publish without constraints.
Consider Hackernoon if you are willing to submit your articles for approval and work on them with editors. Overall quality is higher on Hackernoon because of their approval process so you will likely get more traction if you get published.

Hashnode vs Medium

Hashnode is a free blogging platform with a large community specifically of developers while Medium is a publication service with the largest existing general audience but some dark monetizing patterns that can be off-putting for readers. The other main difference especially for technical writing is that the Hashnode text editor uses Markdown formatting with built-in syntax highlighting where with Medium’s editor you’ll have to do some hacking to get similar results.

Consider Hashnode if you want to specifically write to developers on a free blogging platform.
Consider Medium if you want the widest existing general audience, and don't mind that they will be spammed with paywalls and tracking.

Hackernoon vs Medium

Hackernoon is similar to Medium in that they are both publication platforms where you can submit articles to be published. In fact, Hackernoon started as a publication on Medium but moved to its own (similar) platform. The difference between the two is that Hackernoon is purely a developer’s publication service and their content is free to read and write as a developer whereas Medium caters to all types of content but they charge their readers a fee to read.

Consider Hackernoon if you want to write technical articles and don’t mind working on them with editors.
Consider Medium if you want to write about general topics and make an income off your content.

Final remarks

While each platform has its own advantages and disadvantages, it is up to you to find the one that aligns with your specific needs. You can always choose more than one in order to reach more readers but remember to specify the canonical URL when redistributing your article. All of the above platforms allow you to configure a canonical URL. This will help your domain to rank better on Google: if you don’t set the canonical URL Google sees it as duplicate content.

My first Python project: Auto-Publish to DEV API

Eugene Dorfling — Tue, 10 Nov 2020 15:44:42 +0000

My first Python project

As I am learning programming skills I have thought it a good idea to have an ongoing project that I can work on while learning.

My weapon of choice of course is Python.

The starting line is still in my sights and this is only the beginning but I found that I learn, remember and understand better when I actually do something with what I have learned. It challenges me to test my knowledge and clearly highlights the areas where I am still lacking.

The project I chose is to build is a publishing automation app. As a new technical writer, publishing to all the different platforms we use is becoming a job on its own. Logging into each site, editing the formatting of the piece and setting all the different configurations like tags and the canonical URL(which I only recently discovered) simply takes too much of my time that could be better utilized learning how to code.

One of my biggest reasons for learning how to code, other than my job, is to start automating some of my time consuming repetitive tasks. Looking into how I spend my time, I realized that a lot of things can actually be automated if I only knew how to code.

So I started with this project as my first. I know it might take a while before I can really use it but I know that when it is done(it might become a never-ending project with all the features I am so optimistically adding every day) I will have more time and skills to start building more automation apps. Who knows what else I will be building next, the sky really is the limit when you know how to use the tools.

Where am I with my project?

Well, I have jumped into the deep end as I usually do and started figuring out how can I publish to these platforms and I found APIs. APWhat? Yeah, I am still learning about lists, tuples and dictionaries and then I start my project off by connecting to APIs. I know I tend to always err on the over-ambitious side but hey it wouldn't be a challenge if it was a random number generating app.

Just so we are on the same level, what is an API?

API stands for Application Programming Interface, it is a software intermediary that allows two applications to talk to each other(thanks Google). I see it as a translator that allows you to interface with a server through a pre-defined set of rules. So you can send instructions to a URL that is configured to accept certain commands and do a specific job based on the commands and parameters given. Regardless of the language you use, the API can understand what you are asking and it translates your request to the server. The server then responds through the API so that the API can translate back to the originating party.

Anyways, I obviously still have a lot to learn about APIs so please don't take my view on APIs as the truth. I will probably write a post on APIs when I better understand them.

Publishing through the DEV.to API

So I started out with DEV.to as it is by far my favorite place to publish my articles. I found the documentation on the DEV.to API and started hacking endlessly until I managed to get it working. Well, a few hours but it felt endless.

Now my auto-publisher app can publish articles to DEV.to, I only have to write it and run the program. I can now happily write in vim or vscode or any text editor for that matter and tell my app which Markdown file I want to publish and boom it does it all for me.

It took me a while to figure it out as I am only working off the documentation and other friendly dev writers who shared their process. But here is what I did to make this happen.

First and most important write an article and save it in Markdown format.
Include the front matter at the top of the body of your article. This is where I set the tags, canonical URL and so on. You can also set these with the params in your request but I found it easier to leave them blank in the request and simply configure them within each article. Note: If the Markdown contains a front matter, it will take precedence on the equivalent params given in the payload.

---
title: Hello, World!
published: true
tags: beginners, replit
date:
series:
canonical_url:
cover_image: 
---

# Article Heading 

Rest of your article

Create a Python file, import the requests library and write the code that gets the content from your file and copies it into the variable that we will use as the body.

import requests

filename = input('File Name:')

f = open(filename, 'r')

body_markdown = f.read()

Set up the connection variables => url, api_key, body_markdown(defined above). The API key is unique to each user. You can generate yours on your DEV.to Settings page under Account -> DEV API Keys

url = "https://dev.to/api/articles"
api_key='eXUXQXpX5XVXXyXaXwX4XgXA'

Then I have a function defined that does the magic. As you will see the create_article() function takes all the parameters that are already embedded within your article so they all default to None. If you choose to rather set these here you can, however, just remember that if your file contains front matter these will be ignored.

def create_article(
        title,
        body_markdown="",  # must default to empty string instead of None otherwise dev.to raises error on edit
        published=None,
        series=None,
        main_image=None,
        canonical_url=None,
        description=None,
        tags=None,
        organization_id=None,
    ):
        """Create an article
        :param title: Title
        :param body_markdown: Article Markdown content
        :param published: True to create published article, false otherwise
        :param series: Article series name
        :param main_image: Main image (or cover image)
        :param canonical_url: Canonical Url
        :param description: Article Description
        :param tags: List of article tags
        :param organization_id: Organization id
        :return: newly created article
        """

        data = {
            "title": title,
            "body_markdown": body_markdown,
            "series": series,
            "published": published,
            "main_image": main_image,
            "canonical_url": canonical_url,
            "description": description,
            "tags": tags,
            "organization_id": organization_id,
        }
        # remove None keys from dict
        data = {k: v for k, v in data.items() if v is not None}
        response = requests.post(url, json=data, headers={"api-key":api_key}).json()
        if response.get("id") == None:
            print('Error Article not published')
        else:
            print('Article Posted. ID:{} /n Created at:{}'.format(response.get("id"),response.get("created_at")))

Then I call the create_article() function, pass an empty title as it's already defined within the front matter of the file and then pass the body_markdown which contains the content.

create_article('',body_markdown)

The function will put everything together and post your article with this line of code response = requests.post(url, json=data, headers={"api-key":api_key}).json(). If successful the response will give you an article id and the date & time it was created. If it fails it only prints an error message but you can also print the actual response if you want to see what went wrong.

That is it for the DEV.to integration. Now I am off to study some more so I can add more platforms and features. I will definitely pay special attention to APIs when they come across my learning path, looks like I will be working with them more often than I thought.

Learning Python: Where to start?

Eugene Dorfling — Mon, 09 Nov 2020 14:39:15 +0000

Why am I learning python?

I have recently made a career change from a Cisco collaborations engineer to technical writing. I became bored with the tedious repetitive tasks I was doing and knew that those tasks could be automated with code. But I have always told myself that I am not a "programmer" and that those jobs are for smart people. Then I started playing with IoT devices for fun, specifically Raspberry Pi with Python and realized that the code was actually quite understandable. Then I decided to give it a go, but never fully committed to the journey though.

Anyways now that I am a technical writer, my job is to write... about technical stuff, mostly programming.

This is great because I now have the opportunity to study programming and then share my findings through writing. It is certainly best of both worlds because both are crafts that I would love to master one day.

Python being the most popular language is one thing but there is just something about the language that I instantly fell in love with. When I first really looked at Python code I thought, "Hey, I can actually understand what is going on here", and that's when I realized that programming isn't just for the smart people. We are all smart people and we can choose to learn and become experts at whatever it is we are curious about.

Python is easy to read and understand, however, there is a lot to learn and understand before you get to a point of building your own cool programs.

My process of learning is a bit all over the place at the moment but if you are here looking for a way to start I would recommend something like this:

How am I learning Python?

Firstly I started off with some video tutorials that introduce programming and Python as a programming language. This is a good starting point if you have no idea what Python is as it is a quick way to familiarize yourself with Python and programming at a high-level. This also helps you to have a basic understanding of concepts when you dive deeper. One of the videos I would recommend here is "Learn Python - Full Course for Beginners [Tutorial]" by FreeCodeCamp
After watching a few video tutorials you should have a better idea of what Python is at a high-level and also what it can be used for. So I started working through an online course that focuses on the theory behind the Python language, its usage and a few of the most commonly used Python libraries. This is good for really learning the language, syntax, libraries and how to use them. I have chosen the "Python Tutorial" at w3schools for this as I enjoy the exercises after each section. FreeCodeCamp also has nice tutorials which I will probably move to after finishing with w3schools for an even deeper understanding. I think it's good to learn from various sources as they often teach different ways of achieving the same results which help you develop your own way.
After having a good understanding of the theory I will move to "Automate the boring stuff with Python". The book is well written and easy to follow but what makes it so good to me is that it is full of examples, exercises and project ideas that you can use to expand your practical knowledge. This makes is a great next step as you can refresh your theoretical knowledge while coding along to practice the practical side. The second half of the book is where things start getting fun. It introduces you to more complex programs and actual examples of automation apps including project ideas which you can build to automate some of your own boring repetitive tasks.
Then I have already chosen a larger more complex project for myself to work on while I am learning. I learn through doing and I found it best to set a project for myself and break it into bite-sized pieces that I build as I learn. For most of it, I don't have any idea how it will be accomplished but it really is fun when I learn something and add it to my project (when it works). When it doesn't work it is also great because I then have a good idea of what I still have to learn.

Knowing how to code is one thing, knowing how to do it correctly and efficiently is where practice comes in. From here you would be able to build projects without following a tutorial so let your imagination run wild and start coding daily. You can write small programs or work on more complex ones as long as you consistently code every day. It is also a good idea that these projects are aligned with what you intend to accomplish as there are many different areas where Python is used and they each have their own learning curve. This can be anything like Machine Learning, Web-development, data-science and automation to name a few.

I am still a long way to mastering the Python programming language but I am enjoying every bit of the journey. I hope that you too find a way that works for you and most of all that you enjoy every moment!

Happy Coding!

A list of Great Python Projects for Beginners

Eugene Dorfling — Thu, 05 Nov 2020 12:49:40 +0000

Beginner Python Projects

So you have learned the basics of Python programming but you are looking for projects that will test and expand upon what you have learned.

Here is a list of project tutorials that will get you on the road to building your own Python projects in no time.

Find the links to all the tutorials here

Your first Python Game: Mad Libs
A Guessing Game with Python
Rock, Paper, Scissors
Your first two-player Game: Tic, Tac, Toe
Juggling with PyGame
Crawl Wikipedia Pages with Python
Web scraping with BeautifulSoup
Web scraping with Selenium
An Alarm Clock with Python GUI
Data Science with Python
Analyze Survey Data
Your first web-app with Django
Hangman Game
Basic Calculator GUI
Memory Puzzle Game
Password Generator
Text Adventure Game
Snake Game in Python
URL Shortening Service with Django
Introduction to Machine Learning
Predict Wine Quality with Machine Learning
Bitcoin Price Notification App
Dice Rolling Simulator
Encoding & Decoding Messages
Discord Bot with Python
Secret Communication App: Hiding text in images

More fun projects

You can find the links to all these tutorials at beginnerpythonprojects.com.

Real-world projects

If you are looking for some real-world projects to work on, check out DevProjects. It's a free community where you can learn programming through building projects specifically designed to give you a real-world challenge. But not to worry, they have projects for all levels of programmers and you can discuss projects, review code, and learn from peers and mentors.

We are adding more projects daily so please let me know in the comments if you know of any good Python projects for beginners that we can add to the list.

Happy Coding!

How to manage multiple remote repositories with git (terminal)

Eugene Dorfling — Thu, 29 Oct 2020 06:47:08 +0000

It is not uncommon to have more than one repo to keep track of; sometimes we have a local copy of the main repo that we work on and fiddle with until we are happy to submit a well-prepared pull request to the main repo.

In this guide, we will go through the steps that I use when setting up and working with multiple remote repositories. In this case, we will be working with two namely origin which is our copy and upstream which is the main repo.

Fork the repository

To get started we will first fork and clone the main repo so that we have our own version to work on and test things out before we submit changes to the main repo.

Open the GitHub page of the repository that you want to work on and click on "Fork" in the top right corner. Select the account where you would like to fork to.

Clone the repository locally

$ git clone https://github.com/eugenedorfling/leanpub-guide.git
Cloning into 'leanpub-guide'...
remote: Enumerating objects: 30, done.
remote: Counting objects: 100% (30/30), done.
remote: Compressing objects: 100% (18/18), done.
remote: Total 30 (delta 11), reused 24 (delta 7), pack-reused 0
Unpacking objects: 100% (30/30), done.
Automatically creates origin remote

Now that you have a local copy of your fork, "change directory" to the cloned directory.

cd leanpub-guide/

By default, cloning a repository will set the URL of that repository as the 'origin' remote. Let's confirm that by listing all configured remotes:

$ git remote -v
origin  https://github.com/eugenedorfling/leanpub-guide.git (fetch)
origin  https://github.com/eugenedorfling/leanpub-guide.git (push)

$ git remote get-url origin
https://github.com/eugenedorfling/leanpub-guide.git

git remote -v lists all configured remotes where git remote get-url origin shows the URL of the remote named 'origin'

Configure upstream remote

Now we want to configure the 'main' repo, from which we forked, as the 'upstream' remote.

First, let's check if the name 'upstream' is available by getting the URL for 'upstream'

$ git remote get-url upstream
fatal: No such remote 'upstream'

Above you can see that we have no such remote so let's configure it next.

To configure the upstream remote we will use the git remote add upstream https://main-repo-url.git command as seen below:

$ git remote add upstream https://github.com/ritza-co/leanpub-guide.git

The above adds the remote URL with the name 'upstream'. Note you can give these remotes any name that makes sense to you as it is not uncommon to have more than two remotes configured.

To update the name of an existing remote use the command git remote set-url upstream https://new-url.git where 'upstream' is the name of the remote that you want to edit.

To remove a remote completely use the command git remote remove upstream where 'upstream' is the name of the remote that you want to remove.

Now, let's confirm that we have two remotes configured one for our fork called 'origin' and one for the main repo called 'upstream'. We can do this by listing all remotes with git remote -v or you can check them individually with git remote get-url upstream as seen below.

$ git remote -v
origin  https://github.com/eugenedorfling/leanpub-guide.git (fetch)
origin  https://github.com/eugenedorfling/leanpub-guide.git (push)
upstream        https://github.com/ritza-co/leanpub-guide.git (fetch)
upstream        https://github.com/ritza-co/leanpub-guide.git (push)

$ git remote get-url upstream
https://github.com/ritza-co/leanpub-guide.git

If you want to see more information about a particular remote, use the git remote show command as seen below:

$ git remote show origin
* remote origin
  Fetch URL: https://github.com/eugenedorfling/leanpub-guide.git
  Push  URL: https://github.com/eugenedorfling/leanpub-guide.git
  HEAD branch: master
  Remote branch:
    master tracked
  Local branch configured for 'git pull':
    master merges with remote master
  Local ref configured for 'git push':
    master pushes to master (up to date)

$ git remote show upstream
* remote upstream
  Fetch URL: https://github.com/ritza-co/leanpub-guide.git
  Push  URL: https://github.com/ritza-co/leanpub-guide.git
  HEAD branch: master
  Remote branch:
    master new (next fetch will store in remotes/upstream)
  Local ref configured for 'git push':
    master pushes to master (up to date)

Make some changes to your local repo and push to origin

Now, let's add a file and push it to our origin repo.

First, we do a sanity check to make sure we are working on the correct branch with the command git status.

$ git status
On branch master
Your branch is up to date with 'origin/master'.

nothing to commit, working tree clean

Above we can see that we are working on branch 'origin/master' which is exactly where we want to be.

Now we'll add a simple text file and then add and commit it to be pushed.

$ vim firstfile.txt
$ git add .
$ git commit -m 'add firstfile to origin'
[master 43e5983] add firstfile to origin
 1 file changed, 1 insertion(+)
 create mode 100644 firstfile.txt

With vim firstfile.txt we add a text file. With git add . we add all(.) files to be committed, then we commit them with git commit -m 'add firstfile to origin' where -m 'add firstfile to origin' is the commit message.

Now, let's push our changes to our origin repo:

$ git push origin 
Counting objects: 3, done.
Delta compression using up to 4 threads.
Compressing objects: 100% (3/3), done.
Writing objects: 100% (3/3), 342 bytes | 342.00 KiB/s, done.
Total 3 (delta 1), reused 0 (delta 0)
remote: Resolving deltas: 100% (1/1), completed with 1 local object.
To https://github.com/eugenedorfling/leanpub-guide.git
   d3600bc..43e5983  master -> master

Make some changes and push to upstream to simulate changes from "their end"

From here we will fetch our upstream repo with git fetch upstream. The command fetch only makes a local copy of the upstream repo which you can explore and work on, it does not merge the upstream with what you have locally like git pull will try to do.

$ git fetch upstream 
From https://github.com/ritza-co/leanpub-guide
 * [new branch]      master     -> upstream/master

From here we can checkout the upstream master branch and work on it without overwriting or merging any of our origin files.

$ git checkout upstream/master 
Note: checking out 'upstream/master'.

You are in a 'detached HEAD' state. You can look around, make experimental
changes and commit them, and you can discard any commits you make in this
state without impacting any branches by performing another checkout.

If you want to create a new branch to retain commits you create, you may
do so (now or later) by using -b with the checkout command again. Example:

  git checkout -b <new-branch-name>

HEAD is now at d3600bc add latest published versions

From here let's list the files in the directory to verify that 'firstfile.txt', which we created earlier on the 'origin/master', is not there.

$ ls
01-getting-started.md  02-first-chapter.md  03-book-cover-publish.md

Now let's create a branch and add a file to that branch then push to 'upstream/branch'.

First we 'checkout' a new branch with git checkout -b upstream-change where 'upstream-change' is the name of the new branch and -b allows you to create and checkout to a new branch. This lets you use one command instead of creating the branch first git branch branch-name and then switching to it git checkout branch-name.

$ git checkout -b upstream-change
Switched to a new branch 'upstream-change'

Then, let's add a file to this branch and push it to upstream.

$ vim secondfile.txt
$ ls
01-getting-started.md  02-first-chapter.md  03-book-cover-publish.md  secondfile.txt

$ git add .
$ git commit -m 'add-secondfile-upstream'
[upstream-change a6d2e71] add-secondfile-upstream
 1 file changed, 1 insertion(+)
 create mode 100644 secondfile.txt

$ git status
On branch upstream-change
nothing to commit, working tree clean 

$ git push upstream 
Counting objects: 3, done.
Delta compression using up to 4 threads.
Compressing objects: 100% (3/3), done.
Writing objects: 100% (3/3), 330 bytes | 330.00 KiB/s, done.
Total 3 (delta 1), reused 0 (delta 0)
remote: Resolving deltas: 100% (1/1), completed with 1 local object.
remote: 
remote: Create a pull request for 'upstream-change' on GitHub by visiting:
remote:      https://github.com/ritza-co/leanpub-guide/pull/new/upstream-change
remote: 
To https://github.com/ritza-co/leanpub-guide.git
 * [new branch]      upstream-change -> upstream-change

Pushing the branch to upstream returns a URL where you can go and merge the branch into master through a Pull Request on the GitHub web interface. For this tutorial, I will not be showing this part but you are welcome to check out this guide on on GitHub for more.

Now you can checkout back to master as we are going to look at the differences between origin and upstream next. You can do this with git checkout master

Check the different commits between origin and upstream

Now we want to compare the two repositories and see if there are any differences, ie. commits that were made to the main repo that we don't have on ours.

First, we need to do a fetch again of the latest upstream repo.

$ git fetch upstream
remote: Enumerating objects: 1, done.
remote: Counting objects: 100% (1/1), done.
remote: Total 1 (delta 0), reused 0 (delta 0), pack-reused 0
Unpacking objects: 100% (1/1), done.
From https://github.com/ritza-co/leanpub-guide
   d3600bc..3ef9c08  master     -> upstream/master

Next, we want to see what commits we have in the upstream that we don't have on our origin.

$ git log upstream/master ^master
commit dfedb0bf3725619b04720fa44d332e1ed57280d5 (upstream/master)
Author: eugenedorfling <70516548+eugenedorfling@users.noreply.github.com>
Date:   Wed Oct 28 16:28:46 2020 +0200

    Update secondfile.txt

commit 3ef9c0885e6a9a9967f87d660e21e82eee60d5e3
Merge: d3600bc a6d2e71
Author: eugenedorfling <70516548+eugenedorfling@users.noreply.github.com>
Date:   Wed Oct 28 09:13:56 2020 +0200

    Merge pull request #1 from ritza-co/upstream-change

    add-secondfile-upstream

commit a6d2e718676cc1ba63228e118381f91c4dd527ef (HEAD -> upstream-change, upstream/upstream-change)
Author: eugenedorfling <eldorfling@gmail.com>
Date:   Wed Oct 28 09:01:50 2020 +0200

    add-secondfile-upstream

The above command pulls the logs and shows the commits that are in 'upstream/master' but not(^) in master(checked out origin/master). We can see that there are three commits in upstream that we don't have in our local master. One where a file was added to a branch and one where that branch was merged into the upstream master and one where the file 'secondfile.txt' was updated.

Then we want to check which commits we have in our origin master that the 'upstream/master' does not have by reversing the above command as follows:

$ git log master ^upstream/master
commit 0630646154be76ac6de3e603488dfcf79d363c38 (HEAD -> master, origin/master, origin/HEAD)
Author: eugenedorfling <eldorfling@gmail.com>
Date:   Wed Oct 28 16:37:50 2020 +0200

    add some text

commit 43e598384f2802ca486aeef62c1652e08a3f5f59
Author: eugenedorfling <eldorfling@gmail.com>
Date:   Wed Oct 28 08:28:26 2020 +0200

    add firstfile to origin

Above we can see that we have two commits that are not in the upstream/master. One where 'firstfile.txt' was added and one where the file was edited.

Cherry-picking commits to push to upstream / origin

Now we can look a little deeper at what exactly was changed by doing a git diff between the two commits we have in our origin git log.

$ git diff 43e598384f2802ca486aeef62c1652e08a3f5f59 0630646154be76ac6de3e603488dfcf79d363c38
diff --git a/firstfile.txt b/firstfile.txt
index 31b052a..951c992 100644
--- a/firstfile.txt
+++ b/firstfile.txt
@@ -1 +1,3 @@
 Here is some text that we want to push to the origin/master branch.
+
+Another line added
\ No newline at end of file

Above we do a git diff between the two commits that we have by copying and pasting the commit id's found in the git log command from the previous step.

So we can see some text was added: 'Another line added'. We don't want to move this change over to the upstream, we only want to add 'firstfile.txt' before it was edited. We can accomplish this by cherry-picking the commit we want into a new branch and push that to upstream. Be sure to checkout to the master branch if you haven't already.

First, we want to reset our origin branch to match the upstream branch.

$ git reset --hard upstream/master 
HEAD is now at dfedb0b Update secondfile.txt

$ git status
On branch master
Your branch and 'origin/master' have diverged,
and have 3 and 2 different commits each, respectively.
  (use "git pull" to merge the remote branch into yours)

Above we first did a reset to match our local master with the upstream master. From here we can do a git pull to merge origin and upstream but we don't want that so we will work on a new branch that we can push to upstream without changing our origin master branch.

Now that our master matches that of the upstream we can create a new branch and cherry-pick the commits we want to keep.

$ ls
01-getting-started.md  02-first-chapter.md  03-book-cover-publish.md  secondfile.txt

$ git checkout -b add-origin-commit-to-upstream
Switched to a new branch 'add-origin-commit-to-upstream'

$ git cherry-pick 43e598384f2802ca486aeef62c1652e08a3f5f59
[add-origin-commit-to-upstream b3fa649] add firstfile to origin
 Date: Wed Oct 28 08:28:26 2020 +0200
 1 file changed, 1 insertion(+)
 create mode 100644 firstfile.txt

$ ls
01-getting-started.md  02-first-chapter.md  03-book-cover-publish.md  firstfile.txt  secondfile.txt

$ cat firstfile.txt 
Here is some text that we want to push to the origin/master branch.
$

Above we verify with ls which files exist in the repo. Then we cherry-pick the commit by pasting the commit-id form before (where the file was added). Doing ls again shows that 'firstfile.txt' was added. Lastly, we confirm that the other commit was not added (the second line of text).

If you do a git status now you will see that the working tree is clean because the cherry-pick command takes the commit and adds it to this branch directly, there is no need to commit again.

$ git status
On branch add-origin-commit-to-upstream
nothing to commit, working tree clean

Now we can push this branch to upstream, create a pull request and merge it into the master branch of upstream.

$ git push upstream add-origin-commit-to-upstream 
Counting objects: 3, done.
Delta compression using up to 4 threads.
Compressing objects: 100% (3/3), done.
Writing objects: 100% (3/3), 344 bytes | 344.00 KiB/s, done.
Total 3 (delta 1), reused 0 (delta 0)
remote: Resolving deltas: 100% (1/1), completed with 1 local object.
remote: 
remote: Create a pull request for 'add-origin-commit-to-upstream' on GitHub by visiting:
remote:      https://github.com/ritza-co/leanpub-guide/pull/new/add-origin-commit-to-upstream
remote: 
To https://github.com/ritza-co/leanpub-guide.git
 * [new branch]      add-origin-commit-to-upstream -> add-origin-commit-to-upstream

Once you have created the PR and merged it into master, do a fetch from upstream and verify that the firstfile.txt is there.

$ git fetch upstream 
remote: Enumerating objects: 12, done.
remote: Counting objects: 100% (12/12), done.
remote: Compressing objects: 100% (9/9), done.
remote: Total 11 (delta 4), reused 5 (delta 2), pack-reused 0
Unpacking objects: 100% (11/11), done.
From https://github.com/ritza-co/leanpub-guide
 * [new branch]      add-origin-commit-to-upstream -> upstream/add-origin-commit-to-upstream
 + 0630646...9a290d7 master                        -> upstream/master  (forced update)
 * [new branch]      upstream-change               -> upstream/upstream-change

From the above output, we can already see that the add-origin-commit-to-upstream branch was added but let's verify that the commit and 'firstfile.txt' are there.

$ git log upstream/master 
commit 9a290d7d655d952259d19cd8f9dd5a5ab38612be (upstream/master)
Merge: dfedb0b b3fa649
Author: eugenedorfling <70516548+eugenedorfling@users.noreply.github.com>
Date:   Wed Oct 28 17:20:11 2020 +0200

    Merge pull request #2 from ritza-co/add-origin-commit-to-upstream

    add firstfile to upstream

commit b3fa649afdfc906700fcf36bd31ee7c6090446b6 (upstream/add-origin-commit-to-upstream)
Author: eugenedorfling <eldorfling@gmail.com>
Date:   Wed Oct 28 08:28:26 2020 +0200

    add firstfile to origin

$ git checkout upstream/master 
Note: checking out 'upstream/master'.

You are in 'detached HEAD' state. You can look around, make experimental
changes and commit them, and you can discard any commits you make in this
state without impacting any branches by performing another checkout.

If you want to create a new branch to retain commits you create, you may
do so (now or later) by using -b with the checkout command again. Example:

  git checkout -b <new-branch-name>

HEAD is now at 9a290d7 Merge pull request #2 from ritza-co/add-origin-commit-to-upstream

$ ls
01-getting-started.md  02-first-chapter.md  03-book-cover-publish.md  firstfile.txt  secondfile.txt

Above we check the log of upstream/master to confirm the new file commit is listed there. Then we checkout to the upstream/master branch to verify the actual file it there with the ls command.

Syncing upstream with origin

Let's say you want to sync your origin with upstream, you can do this with the merge command.

First, check that you are on the local master branch.

$ git checkout master 
Previous HEAD position was 9a290d7 Merge pull request #2 from ritza-co/add-origin-commit-to-upstream
Switched to branch 'master'
Your branch is up to date with 'origin/master'.

Now make sure you have the latest copy of the upstream by doing a fetch

$ git fetch upstream

Then we will merge upstream with our local master branch.

$ git merge upstream/master 
[master 77566b0] Merge remote-tracking branch 'upstream/master'.

Once the merge is complete, you can push to origin and upstream to make sure both remote repositories are in sync.

$ git push origin
Counting objects: 3, done.
Delta compression using up to 4 threads.
Compressing objects: 100% (3/3), done.
Writing objects: 100% (3/3), 336 bytes | 168.00 KiB/s, done.
Total 3 (delta 2), reused 0 (delta 0)
remote: Resolving deltas: 100% (2/2), completed with 2 local objects.
To https://github.com/eugenedorfling/leanpub-guide.git
   0630646..77566b0  master -> master

$ git push upstream 
Counting objects: 8, done.
Delta compression using up to 4 threads.
Compressing objects: 100% (8/8), done.
Writing objects: 100% (8/8), 808 bytes | 808.00 KiB/s, done.
Total 8 (delta 4), reused 0 (delta 0)
remote: Resolving deltas: 100% (4/4), completed with 1 local object.
To https://github.com/ritza-co/leanpub-guide.git
   9a290d7..77566b0  master -> master

Lastly, check the git log to verify that origin/master and upstream/master are the same.

$ git log
commit 77566b03978eacdefaaa114b777e10f601909d64 (HEAD -> master, upstream/master, origin/master, origin/HEAD)
Merge: 0630646 9a290d7
Author: eugenedorfling <eldorfling@gmail.com>
Date:   Wed Oct 28 17:57:38 2020 +0200

    Merge remote-tracking branch 'upstream/master'

Above in the second line, you can see that the following are all in sync (HEAD -> master, upstream/master, origin/master, origin/HEAD).

Hopefully, you now have a better idea of how to work with multiple remote repositories, configuring remotes, cherry-picking commits and syncing different repositories.

Wishing you well on your git adventures!

Installing the Firefox web driver on Linux for selenium

Eugene Dorfling — Wed, 21 Oct 2020 19:24:02 +0000

Installing the Firefox web driver on Linux for selenium

I found this cool Python tutorial for beginners on web scraping using the selenium module. However, in the book, there is a part where the author says "If you encounter the error message: "'geckodriver' executable needs to be in PATH" you will have to manually install the webdriver to get selenium working".

I was one of the lucky ones that got to learn how to install the driver manually, hence this guide was born.

In this guide, I will explain the steps I took to manually install the Firefox webdriver on Linux(Debian) and configure the PATH to get selenium working correctly.

Specifically, I will explain the following steps:

Downloading the Firefox web driver
Understanding the PATH environmental variable
Configuring the webdriver to work with selenium
Testing selenium with Python.

Assumptions:

You have the Firefox web browser installed. (If you don’t you can download it for free from https://getfirefox.com/.)
You have Python installed.
You have selenium installed. (You can install selenium by running pip install --user selenium from the command line terminal.)

Downloading the Firefox webdriver

Go to https://github.com/mozilla/geckodriver/releases and scroll down to assets. There you will find the gecko driver for the different operating systems.
Click on "geckodriver-v0.27.0-linux64.tar.gz" to download the Linux 64bit driver. Choose the directory where you want to save the zipped file and start the download.

Understanding the PATH environmental variable

In order for selenium to execute the webdriver successfully, it needs to know where the executable file “geckodriver” is located. To accomplish this there is an environmental variable called PATH in which your program looks for the address of executable files.

PATH is an environmental variable in Linux that tells the shell in which directories to search for executable files in response to commands given through the command line or shell scripts. This is also the reason you can type a command like ls (list) without having to specify its directory /bin/ls.

PATH (upper case letters) is different from path (lower case letters) where the latter refers to the address of a file or directory. ie./home/user/file.txt

Next, I will explain two ways that you can set up the webdriver to work with the PATH variable. The first is a permanent solution where you place the executable within a directory already in the PATH variable. The second is a temporary solution where you add the directory of the webdriver executable to the PATH variable. With the latter, the PATH resets when a new session is started.

Unzip to the directory in PATH

The easiest method is to unzip the geckodriver.tar into the /usr/local/bin directory, which is already in the PATH by default. To achieve this enter the following command into your command line from within the directory where the geckodriver.tar file is located. Note the name of your file should match that of which you downloaded.

tar -C /usr/local/bin/ -xvf geckodriver-v0.27.0-linux64.tar

This places an unzipped copy within the /usr/local/bin folder that is already listed in the PATH variable. You will now be able to run the geckodriver command to test it out.

This change will remain in place even after the session is restarted.

Add the chosen geckodriver directory to PATH

To temporarily add the geckodriver, add the directory where the geckodriver executable is located to the PATH variable. Enter the following command where YourDirectory is the directory of the geckodriver executable:

export PATH=$PATH:/YourDirectory

This adds the directory where the executable is located to the PATH variable. You can check this by looking at the PATH variable values. Enter the following command:

echo $PATH

This will show all the directories that are currently listed within the PATH variable separated by semicolons. You will now also see your geckodriver directory listed there and will be able to run the geckodriver command to test it out.

Open a web page with Python

Now you will be able to use selenium from within Python. You can test it out by opening a web page from the interactive Python shell. Type python3 to open the python interactive shell then enter the following commands:

>>> from selenium import webdriver
>>> browser = webdriver.Firefox()
>>> browser.get('https://beginnerpythonprojects.com/')

Note webdriver.Firefox() opens up the Firefox web browser and the next line opens up the webpage https://beginnerpythonprojects.com/

Technical Writing: Publishing your own book

Eugene Dorfling — Mon, 19 Oct 2020 16:42:56 +0000

Software engineers are in high demand and people are jumping at the opportunity to join the party and become involved themselves. There are a lot of great resources online to get started on learning the basics and also the more advanced stuff.

The thing is the online courses and materials are mostly generic, teaching you the basics of a programming language or taking you through a simple project. This is great for beginners but as soon as they start to grasp how to use the technology and have done some projects of their own, they want to read up on the more in-depth stuff.

They are specifically interested in how the masters do what they do, what is going on in their minds when they come up with the best solutions. They want to know what the masters have learned through many hours of working on complex problems.

The knowledge that engineers have in their minds is becoming far too valuable for them to keep it all to themselves. It is now more important than ever for you to share your inner workings with these aspiring engineers.

If you are a master or simply have some experience that you think other need to hear, now is the time to share that knowledge with those who seek it.

In order to carry over the knowledge that you have gathered over the years or to share some insight or process that you have recently learned, you can write a technical book about it that aspiring engineers can dive deep into when they are reading up on a specific subject.

Writing a technical book is no easy task and I take my hat off to you for even thinking of sharing your knowledge with those who need it.

I have come across an especially simple process and I have put together an opinionated guide that will make the process a little easier for you.

This is an easy to follow step by step guide that takes you through the whole publishing process so that the only thing you need to worry about is writing.

More specifically it covers:

Introduction to Leanpub an online publishing service and bookstore.
Learning the basics of Markdown and using the Leanpub In-Browser Editor.
Creating a book cover with Canva an easy-to-use online graphic design tool.
Publishing your book in the Leanpub book store.

Use this link to go to the Leanpub bookstore and get your free copy. The guide is available in PDF, MOBI, and EPUB formats.

If you just want to download the PDF version as quickly as possible here is a straight download link.

Enjoy!

Balanced Life: Don't ignore the little things

Eugene Dorfling — Sun, 18 Oct 2020 18:24:06 +0000

Life is full of responsibilities. Some are easy and some take some effort to sort out but we can't deny the fact that they exist.

We have to clean our asses, move our asses, and get some sunlight at least once a day. We have to feed ourselves and be sure that what we eat is helping our brains perform at optimal levels. We have to keep our environment clean and keep in touch with loved ones all the while trying to super focus all our energy on that project you are working on.

It is easy when we are immersed in our craft to forget about some of our responsibilities. For instance, when you start dropping balls because your code is taking priority over clean underpants. I am referring to hypothetical balls.

These hypothetical balls represent the things in our life that we spend our energy on. Some of them because our life depends on them and others just because we want to.

The thing is we can only carry so many balls at once.

And if we blow up one ball too much, we push more balls out of our grip. Once a ball starts to slip it’s hard to stop it from popping out and once it pops many often follow.

You see when you stop moving your ass for a few days, you become lazy. Your body struggles to expend the energy you are building up, drinking all that coffee while your bum is growing bigger to store that energy for later. Your muscles start to take sick leave because of depression caused by inaction. We can also say that they deteriorate if you don't like my fancy wording. But nonetheless, real depression follows soon because you can’t think straight anymore and your code doesn't make sense anymore. And it all started with skipping that one-morning workout that you normally do.

Your brain cannot solve problems like it used to because movement is what enhances oxygenated blood flow to the brain. With the consistent demand we place on our brains to come up with creative solutions in our projects, we need to feed our brains more of the good stuff. In that sense, one can almost say, what you feed the brain is directly correlated with the quality of your work. The very brain that came up with those amazing ideas last week will soon feel out of service when you start prioritizing work over a healthy lifestyle.

Similarly, when you procrastinate from your work and run around town the whole day, your works start piling up. The bigger the pile gets the more you think about how far behind you are and the less motivated you will feel to get to work.

The bigger issue here is the psychological one. Where your mind has too many different things to process all the time. When you are leaving your balls lying around unattended, they start working on calling for your attention. While you are looking at a problem that you are trying to solve, you remember that you haven't paid your electricity bill yet. Then you see that your backyard is becoming a scene out of the jungle book. You don't have any clean clothes left in your closet and your significant other half is starting to experience withdrawal symptoms because you space out all the time. Thinking of all these things you still need to do.

Your mind becomes so crowded with all the things you need to do that you simply can't think straight anymore as you try and ignore everything falling apart around you. You try to stay focused on your craft as it normally is the only thing that matters when you find yourself immersed but all these things in your backlog are starting to make you a little crazy. All your balls lying around you, some that can never be picked up again.

The only thing you can do to get yourself out of this slippery spiral is to pull up your socks and start clearing your backlog. One by one picking up the balls lying around, mowing the lawn, paying your bills. Whatever the tasks on your heap of unattended responsibilities, start ticking them off the list.

This is also a time where you cut out some old stuff that you simply aren't interested in anymore. You need to take stock of what you spend your energy on and start prioritizing. Cutting out anything that does not serve you or that you simply don’t have the time and energy to work on. I always say " Sort it out or cut it out" as some things simply can’t be sorted and you need to get rid of the distractions. This will clear up space in your mind in order to focus on what really matters.

When you delay your responsibilities, they become bigger until they are unbearable and take up so much space in your mind that you cannot focus on the task at hand. The idea is to get as close to a clean slate as possible so that you can be present in the moment and focus your energy fully on what really matters.

Here is a fun story that I heard of a monk that goes to a master and asks to be taught the way of life.

A monk said to Chao Chou, “I have just entered this monastery. Please teach me.”
Chao Chou said, “Have you eaten your rice gruel?”
The monk said, “Yes, I have.”
Chao Chou said, “Wash your bowl.”
The monk understood.

It is such a simple story, and I might have a different takeaway than others. To me when you are hungry you must eat, when you are finished you need to wash your bowl, then the next thing happens and you need to do that.

Being 100% present and focused on what you are doing, giving it everything you’ve got. This is when we produce our best work when we are in what some might call a state of "flow". But when you leave that bowl in the sink for later a part of your mind remains behind with that bowl distracting you from what you are then trying to focus on.

We are demanding more from our brains every day and the only way we can ensure it operates at optimal levels is by keeping it focused on the task at hand clear from any other distractions. So, clear up that backlog, get back to the present moment and stay there for as long as you can.

I leave you with a quote that sums this up quite nicely:

"Life is like riding a bicycle, to keep your balance you must keep moving" Albert Einstein

Practice makes your dreams come true

Eugene Dorfling — Thu, 15 Oct 2020 19:26:23 +0000

I have believed inborn talent, and that everyone will find their own, for a long time. Until I realized that, in the end, you choose what you want to master and practice enough until you reach a point of unconscious competence.

I remember having a conversation with a couple at Afrika burn. They were telling me about their dreams and what they wanted to do with their lives, so I simply advised them to go and do it. A friendly girl overheard our conversation and asked me how I can tell people to simply do what they wanted to do. “Life doesn’t work like that,” she said, “you can’t just tell people to do what they want”.

Thinking about what she said made sense at first. But then I remembered when I was told to “go for it” and did, that life in fact does work like that.

A few years ago, while I was still looking to find my talent, I complained to my cousin that my life is pretty boring. I expected more out of life at the time and could not take the repetitive nature of my life anymore. He asked me what I really wanted to do with my life, I told him and he simply said to me “so go and do it”.

You see the mere fact that someone believed in me inspired confidence in myself that I can accomplish my dreams. My cousin is a guy that travels the world with no steady income, living day to day because he wants to. Hearing him tell me that dreams do come true, you only have to believe in yourself, and do what it takes to get closer to achieving them, changed my perspective completely.

Since then, I basically understood that you don’t only have to sit with the cards you are dealt, but that you are allowed to play with them as best you can to get to the outcome you want. We are allowed to have a dream, and most of all to go for it with everything we’ve got - however impossible it sounds.

Then I realized that once you conjure up the confidence to go for what you really want; you start out in the procrastination phase. You read up on everything three times, watching as many YouTube tutorials as possible. Some of us even spend most of our money on online courses and better equipment. But we don’t put in the work.

We are avoiding practice because practice is uncomfortable. Practice essentially means controlled failure. And we do not like failure, it does not feel good at all. But much like Joseph Campbell’s quote:
"The very cave you are afraid to enter turns out to be the source of what you are looking for".

The very act of practicing your chosen craft is what will make you a truly talented master. The thing is, in the beginning, you almost fail instantly and horribly. The practice is nothing near what you have imagined it to be, you were sure it would be more fun. It looked like so much fun when the masters did it.

Unfortunately, procrastinating practice often means that we overdo the theoretical part and we start to create a picture in our minds of how it “should” be done. This causes us to lose our sense of wonder, curiosity, and joy for practicing our craft. This is also where most people quit and start believing in inborn talent again.

We become frustrated because what we produce is nothing near what we envisioned it to be. Because you have watched so many videos on the subject you can almost confidently reproduce what the masters do, in your mind. But when you practice it in reality your fingers don’t know the way yet.

You see you still need to transfer all the theory in your mind to your fingers and keep practicing until your fingers can do the task without the need for your mind to guide them.

Like when starting off playing guitar... you can picture yourself playing a Hendrix solo, just jamming along with some nice beats, you could even have a good idea of the theory sorted in your mind. But when you start playing that guitar your fingers have no idea where to go, and when you find the correct spot, your finger does not have the strength to press down hard enough for a clear sound yet. Also, your hands cannot stretch like they are supposed to, to form the shapes needed to play the chords correctly.

Even with writing and coding, you can read and study as much theory as you like, what makes you exceptional will always be focused practice. I say focussed because as my mentor would say “a lot of people have been driving for 10 000 hours and still drive poorly”, meaning merely repeating steps doesn’t necessarily make you better at them. You need to focus on improving every time you take the next step.

You should balance theory with practice in order to get better. I am still typing faster and faster every day, pressing fewer wrong keys on the keyboard, one day I will be as fast as those veteran coders. But by then keyboards will probably be old fashion and we will be using some touchless interface.

So, when you can imagine things perfectly in theory but in practice you fumble, you can assume that you have been working hard on developing the theory part of it all but you are lacking in the practical side of things. You need more practice.

That why practice is so important - it is almost as important as learning the theory but honing your skill day by day is what will lead you to mastery.

So, the whole process of reaching for the stars can be summed up as follows: You start out with unconscious incompetence. Then with curiosity, you gain awareness of your dream and reach conscious incompetence. You then start studying and learning more about it which leads to conscious competence. With focussed practice over time, you reach mastery or unconscious competence. Like tying your shoes, you don’t think about that anymore, your hand does that automatically.

Why everyone should learn how to code.

Eugene Dorfling — Wed, 14 Oct 2020 15:31:05 +0000

We now live in a world where programming languages and education has become easier for the layman to use. Knowing how to code and build programs that automate many of our daily tasks means that soon we will much rather have a computer keep track of stuff and rely way more on the computers to handle our daily chores and tasks.

UX design has become so good (with the help of AI and ML) that we will soon see interfaces where we simply think about something and the computer happily performs the task that we thought about. Yes, I am talking about a mind-reading personal assistant.

The thing is when we simply think of ice cream and poof there it is, we will never know how ice cream is made and will never be able to make one for ourselves. There are a lot of things, some more concerning than others, that will be automated and run completely by computer algorithms.

This not only means loss of control but the loss of the ability to control. We will have no idea of how things are done because we have hidden them behind a beautiful UX wall that we simply ask and receive.

We are nowhere near a perfect mind-reading personal assistant but we are fast approaching this reality.

This means that what we do now for an income is going to be taken over by software but until then the value lies in creating that software. Whatever skill you have mastered becomes way more valuable if you have the ability to transform that skill into a computer program.

We now have the choice to simply enjoy the evolving technology having no idea how it works, or be a part of the global team building it.

A world that is mostly run by computers is inevitable from my perspective so fighting it is not an option. You can either add your creative perspective by contributing or you can enjoy the chat with your assistant not knowing what really goes on behind the scenes but you trust that those who chose to build did a good job.

The point I am trying to make is that whatever career or industry you are working in, software is going to become a very large part of it soon. The part that controls things and makes decisions for us. We now have the opportunity to take whatever creativity or skills we have and transfer them into code that forms the thinking behind those decisions.

We need to translate our precious and valuable way of thinking and problem-solving skills into computer-readable programs. These programs will soon be doing our jobs all on their own and I would hope that they were trained by the very best human practitioners.

We have figured out a lot about how the world works and trained ourselves with great skills through evolving our kind. But it is now time to hand over the hard work to the computers as they become more and more powerful. I believe coding is like voting, if you choose not to, don’t complain about the outcome.

My first Computer and Lan parties

Eugene Dorfling — Tue, 13 Oct 2020 16:47:25 +0000

I think it was the PC my Dad gave me when the lighting bought him a brand new system. It’s funny how my brother and I both got our PCs that way when the lighting struck our house on Christmas eve, two years in a row. True story.

Anyway, I think it was a Pentium 4 with 2x 512mb ram, and I have no idea which graphics card. With a huge CRT screen ready for Counter-Strike 1.6 and Quake oh and Unreal Tournament. Man those were the days.

I remember playing Counter-Strike at an internet cafe in some aunties garage straight through the night. I would pitch up with some cash and a backpack filled with snacks and cooldrink that will keep me on a sugar high for the whole duration.

But when I was given my own PC I would pack up my whole kit including the CRT screen and set it up on my friend’s mom’s dining table for a kick-ass Lan party.

I was the network boffin with my Dad being a CCIE at Cisco I knew enough to set up a Lan with a switch and a few Lan cables. I could even make an x-over cable and connect two PCs straight to each other.

What knew about IP addressing was the following: It always starts with 192.168.1. and then you increment by ten starting with 10. The “main” PC would always have the IP of 192.168.1.10 and then I would carry on with .20 and .30 and so on. Oh, and the subnet mask will always be 255.255.255.0, it just has to be, otherwise, things won’t work and you won’t be playing any quake until the subnet matches exactly that.

Then the next part is where my friends would be amazed at my Matrix programming skills, I would open up the command line, which was set to green text, and start ping testing. I could even check the IP address through ipconfig all, which I did often just to make more characters run over the screen so it would look like I am coding in the matrix.

And so we enjoyed an evening of multiplayer gaming while exchanging movies and stuff with each other. Those days it took hours to copy a single movie. To copy a series you would have to leave your drive with your friend for a week.

My friends and I would also sell our old components to each other when we upgraded. I remember when a friend gave me two 1gig modules of RAM which a pressed in a little skew and broke the slot. Luckily I could still use the new 1gig module I got, which left me exactly where I started off.

My Dad then configured a VPN to which some of his friends will connect and we basically had online gaming. It was awesome to have Lan parties that did not require me to move my huge CRT monitor and all my cables anywhere. Each of us even had a Cisco IP phone on our desks which served as our discord server.

The setup was way over my head though so I could not get my friends to join in. At least my little brother and I dominated them old farts, my brother more often than I but I came in second most of the time.

Yeah not long after that, I got a steam account and things changed so rapidly that I never even stopped to think about how fast technology is evolving. I mean all of this wasn’t even 20 years ago when we had to physically cable up to play Lan games. Now you download GTA-100GB in an hour or so and start playing with a whole bunch of people, all over the world, right from the comfort of your couch. No more changing to disc 5 to install.

I wonder what this story will look like 20 years from now?