DEV Community: KiminLee

Release 04 part3: Changing the lexer algorithm

KiminLee — Fri, 11 Dec 2020 16:04:16 +0000

Read the previous blog please (this blog is written assuming readers already read the previous blog)

Before start

Previosuly, I used es6 arrow function, but after I went through the program, I realized that the program did not use arrow function at all, so I changed the new arrow function to the es5 function.

const regexTerms = () => {
function regexTerms() {
...

const resolveRegex = () => {
function resolveRegex() {
...

Also, most of the code was using let, even if the variable was actually constant. So I changed them either.

  let rawRegex = regexTerms.join("|");
  let mainRegex = new RegExp(rawRegex, "gi");
  let mainRegex = regexTerms();
  let resolverRegexes = resolveRegex();

  const rawRegex = regexTerms.join("|");
  const mainRegex = new RegExp(rawRegex, "gi");
  const mainRegex = regexTerms();
  const resolverRegexes = resolveRegex(lexerItem);

Problem

So, the main problem was there was too many loop to generate "lexer object" to be compiled after all.

Simply the algorithm looks like this,

// three levels loop
lexerResults.map(lexerResult=>{
   for(let i = 0; i < resolverRegexes.length; i++){
      ...
      Object.keys(tokens.tokens).forEach((token)=>{
       ...
      })
   }
})

So, inside the map loop, there is for loop, then inside the for there is also foreach loop. If we can solve this problem out. (For example, reduce loop), the efficiency will be enhanced.

Solution

So, essentially resolveLexerItemMetadata was doing compare first, then make the object later (which was made in another function beforehand). But, what if the process is combined into one single function, I mean, compare and make the object at the same time.

// function: creating the object.
function resolveRegex() {
  let resolverRegexes = [];
  Object.keys(tokens.tokens).forEach((token) => {
    resolverRegexes.push({
      match: new RegExp(`(${tokens.tokens[token].match})`, "gi"),
      data: {
        identifier: token,
      },
    });

// function: comparing with the given token and object
unction resolveLexerItemMetadata(lexerItem) {
  let data = {};
  const resolverRegexes = resolveRegex(lexerItem);
  for (let i = 0; i < resolverRegexes.length; i++) {
    if (lexerItem.match(resolverRegexes[i].match) != null) {
      data["identifier"] = resolverRegexes[i].data.identifier;
      data["token"] = lexerItem;

      break;
    }
  }
  return data;
}

Solution 2

/* Instead of creating an array of objects for Token to be compared with given lexerItm,
*  Find the token with the lexerItm, then return a single object
*  Previous version, let n = the num of tokens, the best case is Ω(n) runtime.
*  This version, let n = the num of tokens, the best case is Ω(1) runtime.
*/
function resolveRegex(lexerItem) {
  const foundToken = Object.keys(tokens.tokens).find((token) => {
    const matchReg = new RegExp(`(${tokens.tokens[token].match})`, "gi");
    return lexerItem.match(matchReg) != null ? true : false;
  });

Instead of creating a resolvedToken object (intermediate object), simply find the token and if found, creating a final version of object.

function parseLine(line) {
  const lexerResults = line.match(mainRegex); // The Regex result is quickly translated into an array
  let lexeredLine = [];
  lexerResults.map((lexerResult) => {
    lexeredLine.push(resolveRegex(lexerResult));
  });
  return lexeredLine;

By doing so, we can remove the resolveLexerItemMetadata function and put the process altogether.
This will results in reducing one extra loop (foreach loop).

After testing the runtime, I could see the runtime is much faster: approximately 20%.

Visit the PR

Conclusion

At first, I tried to contribute to the bigger project (even if a small change). However, It is hard to find a good one for me even though I watched lecture and tried to practice.
The simple bug is already taken by others, and lots of code I cannot understand.
The reason, I think, is I might be too afraid of failing to create PR. So personally, I score myself 80/100

Even though I could not achieve my goal(contributing to the real project like react.js, next.js ...), I learned a lot of Open Source Development community and environment. How to look for contribution, license, unit testing, git command and communicate with open source development community.

These assets are not something to disappear. This semester is to be end, but my programming experience just begin. I will keep practice the thing I have learned in this course and try to keep connecting with the community. And later I will be able to contribute to the real world project.

The last lab: git tagging, and publishing to npm

KiminLee — Sun, 06 Dec 2020 00:05:26 +0000

git tagging

Git has the ability to tag specific points in a repository’s history as being important. Git tag is essentially a kind of version control. The semantic versioning has a convention to be followed. For more information visit the site

At first, I have zero idea about git tagging. I have used branches name and commit number to track down or retrieve my previous commit. BUT never tagging my commit with version number.

So, I needed to first understand what the tagging is and how to use that. Thankfully we have a great professor who are teaching us about git.

Here is the basic instruction you to follow to add a git tag:

git tag -a <version> -m [message] : this will add the new tag to your current commit.
git tag -d <version> : this will drop the tag
git tag show : this will show the lists of the git tag

v0.9.0

Up until now, the code is not ready to be published. So I decided to call the program at that moment is version 0.9.0. Before making release 1.0.0. I needed to make couple update in the program.

The first: If your file is not inside the same directory of the program. The file cannot be read. Because the file is just passed by name only without path. So I needed to make sure that regardless where the file is, the program should read the file based on their name only.
Update the README.md and changing the program name: Since the npm package should have unique name I needed to come up with a real unique name, for this I decided to add my name to the program name kimin-url-tester.

release 1.0.0

To publish the program. First I needed to sign up the npm website. Once verifying email, we can publish our package.

After success in login, you can hit npm publish
Keep in mind! this will only happen after verifying your email.

You can see your package in your profile

The test:

What is the problem, what would be the best solution?

KiminLee — Sat, 05 Dec 2020 00:52:30 +0000

Make another lexer.js vs use the original one and update it

The issue is updating the lexer.js. And the maintainer of the open source project allowed me to modify the code structure. However, it is really good idea to create a whole new brand of lexer.js or just to remove the useless parts and replace them with new codes?

To decide it, let's take a look at the code.

What does the lexer.js really do in the program.

This program mainly have 4 parts.

tokens: A pair of keyword and value. This is the list of syntax of the program to be compiled. If the words read from a file match with one of the token, the program know that the word is meaningful syntax.

_lexer: basically convert the raw input into meaningful object which later can be transplied.

transplie : take the object from lexer.js. If all syntax are correct then make the codes to javascript codes.

syntax checker : check if the object from lexer.js is all correct if not issue the syntax errors

inside the lexer.js

So, let's explain the lexer engine again step by step:

find all of the possible token(the words predefined in the program to be compiled later, eg) 'print(), var ...')

transform the raw data into the array of the token (need to be sorted: first in first out)

compare the token the object which has identifier(the name of the token) and regex(regular expression).

If they are matched, it will create an object(token and identifier). This object will be compiled later in the transplier.js (final version of object)

step 1 to 2, I should not modify them because if they were changed, all the predefined token will be useless (needs new token system).

Also, the lexer engine has to pass the object to the transplier.js, so that the raw code can be successfuly compiled later.

So, I could only change the step 3.

The thing to be upgraded

So far, there is so much work to be upgraded like var ----> let or const. Removing global variable, reform the code structure...

In this week, I changed the structure of the code first so that the upgrade can be done well later.

I first removed all the global variable in the lexer,js

let colors = require("./colors");
let tokens = require("./tokens");


let regexTerms = [];
let rawRegex = "";
let mainRegex;
let resolverRegexes = [];

Object.keys(tokens.tokens).forEach((token) => { // Populate main regex
    regexTerms.push(`${tokens.tokens[token].match}`);
});

Object.keys(tokens.tokens).forEach((token) => {
    resolverRegexes.push({
        "match": new RegExp(`(${tokens.tokens[token].match})`, "gi"),
        "data": {
            "identifier": token
        }
    });
});

And create a new .js file called lexerToken.js. This new file will generate the intermediate token object which will help to create the final version of object (used in transpiler.js)

This commit can be found in here

After the work, now lexerToken only handle the token related job. And lexer will only handle transform the raw data into the completed object with token object.

next step

The thing is, it took long time to compile big amount of codes. So the lexer engine needs to have smaller runtime.

The main reason for too much runtime may be the too many loop. So , the next step is find the extra loop and reduce it and change the algorithm.

Release 0.4: Part1, choosing a open-source project

KiminLee — Tue, 01 Dec 2020 23:25:09 +0000

First try and failure

In the last weekend, I found the very nice project which is really huge. PWA Studio is a collection of tools that lets developers build complex Progressive Web Applications on top of Magento 2 stores. There are more than 100 contributors.

The issue was even if the filter was changed, it is not actually applied.

The person who raised this issue told that it might be because of GraphQL or any front-end issue. As I didn't know the GraphQL, I tried to find out what it is, then try to install the code and figure out what the problem is.
After spending couple days on it, I realized this was the thing that was beyond my ability. I needed to find out other project.

Small but needed to be changed a lot!

I spent an entire day to search github for this release 0.4. However, I was more careful to choose one after the failure. I didn't want to take another risk.

After all, I have decided to contribute the project that I have done before. The GreenLight has been looking for someone to update the important part of the program since that time. He wants to update the lexical system of the program.
It requires a lot of coding and changing the program a lot.
Even if the project is really small, I think this might be worth contribution.

Plan for this

The issue is making a better system for transpiling the raw code to be read by Node.js. I don't know what would be the best solution for this system. So for next couple days, I will find out what would be the best model for it.

I will keep update the changes.

Added unit-test and CI to the findBreakURL program!

KiminLee — Thu, 26 Nov 2020 02:43:46 +0000

Should we run the program every time?

After implementing all codes for this program, I need to manually run the program and if there is any bug on there. To see what lines are wrong and what function throws error, I coded console.log hundreds of times. This is really TIME CONSUMING!

Now, it is time to introduce unit tests for my program to make the process faster and more reliable. Then, the question is what is the unit tests?

One test module will make your life so much easier

Test module is only for literally testing your program. However, it focuses on one specific job rather than a whole process.
Let's imagine, you are baking a cookie. There are many ingredients to be prepared beforehand. How much sugar is needed and what type of flour should be used, something like these. It is hard to check them all at once just before put altogether. The best way is make a check list and check each of them are OK. Likewise, unit test is checking each function or component is passed.

JEST is the BEST

There are many test tool like JASMINE, ENZYME and KARMA...BUT They are so much similar. So don't take too much time to choose the tools!

For my project, I used JEST because I have used that before for my react.js project and my prof is using this tool too!

To set up
npm i jest --save-dev
This will make the jest installed as a devDependencies.

FileRead.test.js

Let's create a test file for file reading. The file reading is essentially reading a file from given path or file name. However, we cannot use the normal file system for our test. We need to somehow create a mock file system.

Thankfully, Jest already has mock file system. For more info please visit the JEST documentation

In the __mocks__/fs.js file :

const fs = jest.createMockFromModule('fs');
...
function __setMockFileData(filename, data) {
    mockFiles = {};
    mockFiles[filename] = data;
}
...
function readFileSync(filename) {
    const data = mockFiles[filename];

    if (data) {
        return data;
    } else {
        throw new Error("invalid file");
    }
}

createMockFromModule will generate a mock filesystem.
__setMockFileData will receive given filename and put the given value into the correspond filename (acting like the file and data)
readFileSync is overloaded and performed when the test file is needed to call this function.

Now, this module will generate mock file system. It is time to make a test

const { fileRead, readDirectory } = require('./fileRead');

jest.mock('fs');
const fs = require('fs');

describe('file reading test', () => {
    const filename = 'file';
    const fileData = '<p>Hello World</p>';

    beforeAll(() => {
        fs.__setMockFileData(filename, fileData);
    });

    ...

    test('reading an existing file should work', () => {
        const data = fileRead(filename);
        expect(data).toEqual(fileData);
    });
});

The test('reading an existing file should work') will be passed if fileData is equal to "

Hello World

Reading a directory is so much similar to this logic. You can also find more info here

axios_fetch.test.js

This will send a get request with axios. So before test this module, we need to create a mock request for testing. nock is a great package for this.

const { axiosFetch } = require('./axios_fetch');
const nock = require('nock');

describe('fetching test', () => {
...
    test('reading from an existing URL should work', async () => {
        const url = 'https://test.ca';

        nock(url).get('/').reply(200);
        const status = await axiosFetch(url);
        expect(status).toEqual(200);
    });
...
})

nock(url).get(root).reply(status) will make a mock request which seems valid request just for testing. Once you make a fake http URL, you can test your fetch function using with this.

However, if you use axios you need to first set your @jest-environment configuration to node (default is jsdom).

For more information visit this issue

Now, your are read to test. result below:

Add an infinite scroll function to Telescope

KiminLee — Thu, 19 Nov 2020 02:52:41 +0000

getting started

So far, the Telescope generates more posts only when the "more post" button was click. Thanks to useSWRInfinite() and IntersectionObserver, the new posts can be rendered infinitely.

Implementation

First of all, we are going to grab a node, so we need useRef hook. Also, IntersectionObserver will be declared and assigned once the page is all loaded, so we need to have useEffect

import React, { useEffect, useRef } from 'react';
...

Second, we need to find out which node should be the flag that the current post is the last post, and the new posts should rendered.

We can reuse the code from previous loadMoreButton.jsx button node.

    <Container>
      <Grid item xs={12} className={classes.content}>
        <Button ref={$buttonRef}>Load More Posts</Button>
      </Grid>
    </Container>

This node will be the reference to indicate the post is the end of the current posts.

Third, we need to make it actually happen!

useEffect(() => {
    const options = {
      root: null,
      threshold: 1.0,
    };

    const observer = new IntersectionObserver(
      (entries) =>
        entries.forEach((entry) => {
          if (entry.isIntersecting) {
            onScroll();
          }
        }),
      options
    );
    observer.observe($buttonRef.current);

    return () => {
      observer.unobserve($buttonRef.current);
    };
  }, []);

This will make the selected button node to be acted like a flag for check.

Migrating Weather application to React.js

KiminLee — Wed, 18 Nov 2020 19:23:53 +0000

Before getting started

For 1 weeks and half, I wandered in Github, finding an issue that is interesting to me. There are many people who are looking for contribution to get help, or many company to solve buggy issues. Those are all OK, but this time none of them is actually appealing to me.

SO, this time, I decided to enhance one of my project which I always wanted to be done since deploying the application, Weather App.

How is the weather?

Thanks to Google service, we can know the weather without an weather forecast or actually going outside.

But, when you wonder what the current weather is in a certain city, not your place, and if you want to find it out, Google is not a great option. It is not faster that you expected and it might be a too heavy web service.

Weather Application is a small-size web application to help search weather by a city name.

It uses Darksky API and Mapbox API to get real-time weather information.

What is the issue?

The issue is integrate the program with react.js. Which means, the app is currently only written in Node.js. It only has server-side program, and to render .html page, it relies on handlebars.

It perfectly works, but the problem is, It doesn't support responsive UI. When you view this on mobile-platform, it isn't good. Also, because it has many handlebar files, and render them, it took some time to load the pages.

These problem would be solved if the program is separated to server-side with node.js and front-end with react.js

Install React.js

To get started, we need to first install React.js.
npx create-react-app [app-name] will generate the React app in the directory. If you don't install react on your machine, it will ask for installation first.

After that you will get a sample react.js project.

Make a component structure

The application has mainly 6 parts. --- Header, Navbar, About, Search, Result, Footer ---. So, create a component tree like this.

And for each folder they have [component_name].jsx file.

Make a migrating a program from `handlebars` to `.jsx`

Main folder is a root container which will contain all components. This will use Switch and React.lazy load, and react-router-dom.

import React from "react";
import { Route, Switch } from "react-router-dom";
import "./App.css";
...
const App = () => {
...
  return (
         <div className="App">
          ....
         </div> 
   );
};

Search component:

const Search = (props) => {
  ...
  return (
    <div className={styles.main-content}>
      <p>Use this site to get your weather!</p>

      <form >
        <input className={styles.input} placeholder="Location..." />
        <button className={styles.button}>Search</button>
      </form>
    ...
    </div>
  );
};

Header Component

const Header = (props) => {
 ...
  return (
    <div className={styles.header}>
        ...
    </div>
  );
}

Like above two components, other component is implemented similarily.

What's the next?

Because of limited time, and so much works to do for this. I separate the issue into 3 more.

We need to make a REST API server ----> which will our client axios can get the request from there.
Make a different CSS style -----> So far, the program is using the same CSS style for now. Bootstrap and .module.css will be used.
Make a redux or hook to handle state and communicate with REST API. -----> this will make the app is fully implemented.

Once the above are all solved, the issue will be continued. I will update gradually once all three issues are done.

Installing and setting up eslint and prettier configuration!!

KiminLee — Thu, 12 Nov 2020 00:27:17 +0000

Introduction

If you were working alone, coding yourself and make a good project without others' contribution, you might seem OK so far, BUT NOT ANYMORE.
If you are thinking of being a open source developer, you need to collaborate with strangers via online. Trust them, and give them a freedom on the project to fix bug, improve the code. Which means you barely have a full control on your project (since the project is NOT for all yours, but for everyone).

As you have a certain preference on your coding style, others do. If everyone code with different coding-style, the project is not readable and maintainable. Also, for the new contributor, they will be too exhausted to find out what's going on the project.

There are two possible solution to solve this problem

1. Make a good documentation to give a guideline for contributors

You can choose a one great style-guideline and let them know your project is following this style. By doing so, everyone knows that they need to follow the code-style to contribute.

2. Force them to follow the style anyhow

If we can make them "have to" follow the code-style, it will save time and we can spend more effort on the real thing!
In this post, we are going to talk about the second solution

prettier

What prettier does? the prettier is a kind of code-styling-checker. The users can make their own style setting and re-organize the code. For example, if you type shift + alt + f on vscode, this will re-format your code to look nicer. If you are not using proper indent and curly bracket and others, it will do them for you. Likewise the prettier do this, but you can MAKE your style.

first you need to create configuration file

A "prettier" key in your package.json file.
A .prettierrc file written in JSON or YAML.
A .prettierrc.json, .prettierrc.yml, .prettierrc.yaml, or .prettierrc.json5 file.
A .prettierrc.js, .prettierrc.cjs, prettier.config.js, or prettier.config.cjs file that exports an object using module.exports.
A .prettierrc.toml file.

Once you create one of them, now you can make your rules on that. For example:

{
  "trailingComma": "es5",
  "tabWidth": 4,
  "semi": false,
  "singleQuote": true
}

in .prettierrc.json

You can find more info on How to create configuration files?, Options for prettier

eslint

eslint seem similar to prettier, but it will check the syntax before running the program. So if it finds some invalid syntax, it will complain about that. Setting up eslint is same with prettier

You can find info here

.vscode

Many programmers use vscode, and the vscode provide nice feature "extension". prettier and eslint is also distributed by extension. You can make contributors to install them.

In the .vscode/extensions.json

{
    "recommendations": [
        "editorconfig.editorconfig",
        "dbaeumer.vscode-eslint",
        "esbenp.prettier-vscode"
    ]
}

Example with findBreakUrl

// in .eslintrc.json
{
    "parserOptions": { "ecmaVersion": 8 },
    "rules": {
        "semi": "error",
        "quotes": ["error", "single"]
    }
}

// in .prettierrc.json
{
    "trailingComma": "es5",
    "tabWidth": 4,
    "semi": true,
    "singleQuote": true,
    "bracketSpacing": true,
    "arrowParens": "always"
}

Update: Take a API from telescope API for findbrokenURL

KiminLee — Sat, 07 Nov 2020 20:37:35 +0000

Before start

telescope is an amazing web project in Seneca. Telescope is an open source web server and client application for aggregating and presenting a timeline of Seneca's open source blogs. Telescope makes it easy to see what's happening with open source at Seneca right now.

Installation

To run the telescope, we need docker, redis and elasticsearch. As I am a window 10 home user, I needed to install WSL2-Ubuntu on my machine.

After install WSL2, go to the docker website and install the docker desktop app. After all installing all the prerequisites, we are ready to go.

Issues

However, after installing everything by following the step, you might come across some very buggy issues. One is you may not be able to run the docker as a background running sudo systemctl start docker on your subsystem. If you spent some time to figure it out, you might try this alias systemctl='/etc/init.d/docker, if it fails, that means you don't have /etc/init.d/ file on your subsytem.
As far as I know, if you are in this stage, you would have 2 options.

Just use desktop app, you can simply turn off and on using with desktop app(GUI), actually no need to use CLI "AT ALL."
Install the docker engine -Linux version- again. This may anyhow create init.d files on your subsystem, so you probably would be able to use the above command.

I chose the first option.

"Do I use Window for development? : hell NO"
-- telescope community --

Running Telescope

If you don't have issues or solved them already, CONGRATULATION! you are "really" good to go.

First you need to install redis and elasticsearch on your system. With docker, this can be managed automatically. To install and run the multiple images and make them containers you can simply do this docker-compose up --build redis elasticsearch.

After that, you can find that you are running redis and elasticsearch is running. Then you can npm start to run the backend server.

Update: enable to take `.json` from telescope API

I added one more option which is --api/-a. With this option user can pass a certain host+apiURL to communicate with them.

changes

if (arg.a) {
  console.log(chalk.bgBlueBright("*******API Request start*******"))
  axios
    .get(arg.a)
    .then(async (response) => {
      console.log(chalk.bgGreenBright("*******API Request Success*******"))
      if (arg.j) {
        response.data.map(res=>{
          await jsonFetch(path.resolve(arg.a, `./${res[id]}`));
        })
      } else {
        response.data.map(res=>{
          await fetchFunction(path.resolve(arg.a, `./${res[id]}`), arg.a);
        })
      }
    })
    .catch((error) => {
      console.log(chalk.bgRedBright("*******API Request Fails*******"))
      console.log(
        chalk.yellowBright(
          `Cannot found: ${arg.a}. Probably because of network connection problem or wrong API address`
        )
      );
    });
} else

Now, this tool can be used for any api if the api has json format URL.

find more info

Last commit for "2020 hacktoberfest"

KiminLee — Thu, 29 Oct 2020 21:06:23 +0000

getting start

For my last contribution to "2020 hacktoberfest", I chose a little simple project but really needs some help. Awesome-Adoption is a working project, finding available pets for adoption.

To get all the information for pets, this project is using a third party API PetFinder.

The issue was token is not stored in local-storage, so whenever refreshing the browser or changing the components the api request kept sending to the server.

...
export default function App() {
  const [token, setToken] = useState("");
  useEffect(() => {
    axios
      .post(
        "https://api.petfinder.com/v2/oauth2/token",
        `grant_type=client_credentials&client_id=${process.env.REACT_APP_PETFINDER_KEY}`
      )
      .then((response) => {
        setToken(response.data.access_token);
      })
      .catch((error) => {
        console.log(error);
      });
  }, []);
...

I suggested a better solution which is using redux. The token will be used across the component and it shouldn't be changed as long as the user is same. redux can keep the token as an available state for all jsx components, and it won't be removed if it is in the localstorage.

HOWEVER, this could cost a lot of time to modify the codes, and change a lot of lines of code, so I tried to make a small change and follow the original code as possible as I can.

First, I created a function for axios fetch

...
function fetchFunction(){
  axios
  .post(
    "https://api.petfinder.com/v2/oauth2/token",
    `grant_type=client_credentials&client_id=${process.env.REACT_APP_PETFINDER_KEY}`
  )
  .then((response) => {
    localStorage.setItem("token",response.data.access_token);
  })
  .catch((error) => {
    console.log(error);
  });
}
...

If there is a token in a localstorage it doesn't need to send post request, otherwise it should.

...
if (!localStorage.getItem("token")) {
  fetchFunction();
}
...
useEffect(() => {
    ...
    if(!localStorage.getItem("token")){
      fetchFunction();
    }
    setToken(localStorage.getItem("token"));
  }, []);

Here is PR

Conclusion

This 2020 hacktoberfest is the first time for me to work as a open source developer. I always tried to contribute to bigger and commercial project, but I couldn't find the reason for the bug or don't know how to fix it. Also, I felt it is really hard to find a "good" project which I can handle.

The good things for me is there are still hundreds of small projects that I can actually help! They are really looking for a small contribution, but no one really care about them. For a beginner, these project would be the "good first contribution." However, I am not so happy that I didn't contribute to the bigger project.

Is that because of my lack of skills? or Am I too afraid of getting the first step on the big project? The answer was both. I was too intimidated by a "fancy" project. At the same time, I needed to learn a more thing about "server", "testing" or other deeper advanced programming skills. Luckily, this is just a beginning!!

GreenLight : Added a syntax checker feature!

KiminLee — Mon, 26 Oct 2020 21:27:05 +0000

Introduction

GreenLight is a new simple programming language created by Marc. This new simple language can help to make CLI program much easier and faster. It takes all the good features from JAVASCRIPT and other many languages.
Variables:
var hello = "world"; // Mutable variable const constant = "light"; // Constant variable
Objects/Array:
let array = [498, 49, 299, 18]; // Declare a new array array[1] = 133; // Modify the second element of the array print(array[1]); // Print the second element of the array let object = { // Declare a new object... "key": "hello world" }; print(object>>key); // Print the value located at the key "key"
It also, provides general standard library!
sqrt(16); // => 4 Get the square root of a number log(1); // => 0 Get the natural logarithm of a number pow(2, 10); // => 1024 Raise a number to an arbitrary power exp(1); // => 2.718281828459045 Get the exponential of a number ...
More Info

Getting start

There was a one good issue on this project. It didn't catch syntax errors and handle them properly. So, GreenLight decided to improve this feature and let someone help them. Luckily, I was there for help!

Step 1: Testing

The first thing to do is always testing first! So I have run the demo hello_world.gl file.

print("Welcome to GreenLight!");
print("Hello world!");
logError("I'm an error.");

print("---Cryptography---");
print("The SHA256 hash of 'Hello World' is " + sha256("Hello World") + "!");
print("The SHA3-256 hash of 'Hello World' is " + sha3("Hello World") + "!");

print("A random number between 50 and 100 is " + randomNumberBetween(50, 100));


print("---Array Functions---");

print("- Array Difference:");
const school_subjects = ['Math', 'Geography','History'];
var approved_school_subjects = ['Geography'];
const pending_subjects = arrDiff(school_subjects, approved_school_subjects);
print("The school subjects that still need approval are: " + pending_subjects);

print("- Array Difference Symmetric:");
const female_animals = ['Duck', 'Cat','Elephant','Tiger'];
var male_animals = ['Duck', 'Dog','Elephant','Tiger'];
const animals_alone = arrDiffSymmetric(female_animals, male_animals);
print("Animals that do not have a pair are: " + animals_alone);

print("- Array Intersection:");
const fruits = ['Banana', 'Apple', 'Grape', 'Strawberry'];
var fruits_find = ['Strawberry', 'Apple', 'Watermelon'];
const fruits_found = arrIntersect(fruits, fruits_find);
print("The fruits found were: " + fruits_found);

It was a simple code file, just printed some messages. After compiling this, it showed that the compilations is OK, and ready to run

[*] Executing lexer...
[OK] Lexer finished successfully!
[*] Transpiling...
[OK] Transpilation complete!
[*] Running syntax checker & linter...
[OK] Syntax checker finished, no issues found!

and following output message.

Welcome to GreenLight!
Hello world!
I'm an error.
---Cryptography---
The SHA256 hash of 'Hello World' is a591a6d40bf420404a011733cfb7b190d62c65bf0bcda32b57b277d9ad9f146e!
The SHA3-256 hash of 'Hello World' is e167f68d6563d75bb25f3aa49c29ef612d41352dc00606de7cbd630bb2665f51!
A random number between 50 and 100 is 100
---Array Functions---
- Array Difference:
The school subjects that still need approval are: Math,History
- Array Difference Symmetric:
Animals that do not have a pair are: Cat,Dog
- Array Intersection:
The fruits found were: Apple,Strawberry

So far, it seems OK when we correctly write the code using GreenLight syntax. However, what if I intentionally break this?

So, I deleted the ')' on the first line print("Welcome to GreenLight!");. It turned out the program didn't catch the syntax error UnhandledPromiseRejectionWarning: ReferenceError:.

Step 2: Spot the exact code file that should be improved!

To make an enhancement, I needed to find out which code file is dealing with syntax checking function among the dozens of .js code files. Luckily there is a hint from Marc and code itself. When I compiled the code, it showed that [*] Transpiling... is ahead of [*] Running syntax checker & linter... That means somehow the Transpiling process will transform the .gl file to '.js' file, and then it will run the syntax checker.

Step 3: Inside `transpiler.js`

In the transpiler.js file, there is:

function runChecks() {
    return syntaxChecker.checkSyntax(code);
}

The "code" is already transformed to .js syntax. So, in this stage the "code" looks like
console.log("Welcome to GreenLight!";print("Helloworld!");greenlight_internal_log_error("I'm an error.");console.log("---Cryptography---"); ...

As we can see, console.log("Welcome to GreenLight!"; is not closed with ')'.

Step 4: Inside `syntax_checker.js`

In this file, there was no syntax checking feature. It only catched if the user-built function was properly called with call.

function checkSyntax(compiledCode) {
    let checkPassed = true;
    let splitCode = compiledCode.split(" ");
    if (!checkAsyncFunctionCalls(splitCode)) checkPassed = false;

    return checkPassed;
}

function checkAsyncFunctionCalls(code) {
    let passed = true;
    let withinString = false;
    ...
    return passed;
}

Step 4: Added bracket checking feature

It is impossible for me to added all the syntax checking functions for the "GreenLight." So, I decided to specifically deal with open and close bracket syntax checker. First of all creating a function,
checkParenthesesBracket it should return false if the bracket doesn't open and close properly.

For example)
({)} false
({[]}) true
([[]) false

The checkParenthesesBracket takes code as parameter. The code was separated by whitespace, but it wouldn't be good idea for this. So It had to be separated by ;. let splitCode = compiledCode.split(";"); As a result, code is lines of the entire code.

For each line, for (let i = 0; i < code.length; i++) must have closing bracket if the open bracket is used.

if (code[i].includes("(")) {
            let flag = true; // check '(', ')' is a pair
            code[i].split("").forEach(char => {
                if (flag && char === "(") flag = false; // needs ')'
                if (!flag && char === ")") flag = true; // needs '('
            });

same for []

if (code[i].includes("[")) {
...
}

Step 5: Testing and Wrap up

if (!checkParenthesesBracket(splitCode)) checkPassed = false; is added in the checkSyntax function right before checkAsyncFunctionCalls.

Now run the code again.

[*] Executing lexer...
[OK] Lexer finished successfully!
[*] Transpiling...
[OK] Transpilation complete!
[*] Running syntax checker & linter...
[WARN] '(' is not closed. This could cause syntax errors: console.log("Welcome to GreenLight!"
[WARN] Syntax checker found at least one issue in your code!

Now, we can catch the error!

Refactoring findBreakURL to improve maintainability!

KiminLee — Fri, 23 Oct 2020 17:28:35 +0000

Before Start

While I am participating "2020 hacktoberfest", I realized that some of the projects on github are not really easy to get start as a new contributor for them. The reason is their codes usually written by one or two contributors and didn't really think about maintainability and let others join in the project. Most of the my projects are also done by me, so to be honest, I didn't really care about the maintainability and conventions as well. findBreakURL is also one of them. However, being a open source developer is not only improving my coding skills but also learn how to interactively work with others and effectively maintain the existing codes.

So, this week, I have decided to improve the code style of findBreakURL

What should be changed?

So far, the program has 4 different .js files.

fetch.js : With axios, we will do 'header fetch' to check status code
json.js : This file will make a json style output
yargs.js : This file handle the arguments and options and read directory to find files
index.js : This is the entry for the program, will read files and following functions

Wait, What? index.js will read files? and yargs.js read directory to find files?
This looks not so consistency and maybe a little bit irrelevant for their desired functions. Maybe we can split the functions to another files.

let's start with git checkout -b refactoring

Step 0 : Make a `fileRead.js`

It will be better if we extract reading a file/directory function from index.js and yargs.js and make a separate file for the change.

Step 1 : Make a `fileRead()` function

In index.js, fileData is extracted.

...
try {
        fileData = fs.readFileSync(`${__dirname}\\${file}`, {
            encoding: "utf-8",
        });

    // wrong file name
    } catch (error) {
        console.log(file + " is a WRONG file name");
        return process.exit(1);
    }
...

Maybe we can make this part as a fileRead() function:

const fileRead = (file) => {
    let fileData = null;

    try {
        fileData = fs.readFileSync(`${__dirname}\\${file}`, {
            encoding: "utf-8",
        ...
    return fileData

After change, commit this change.
git add index.js + git commit split index.js to index.js and fileRead.js

Step 2 : Make a `fileDirectory()` function

In the yargs.js, files are declared globally. And all conditions are not inside a function. It looks better if the variable is only used inside a function and the conditions are also in the function.

Before:

let files = [];
    if (yargs.argv._.length === 1 && yargs.argv._[0] === "start") {

      // decide the option if it is -f or -a
      if (yargs.argv.a && typeof yargs.argv.f !== "string") {
        const tmpFiles = fs.readdirSync(__dirname, { encoding: "utf-8" });

        // if -a, store all files into the files variable
        files = tmpFiles.filter((file) => {
          return file.toLowerCase().endsWith(".html");
        });

      } else if (typeof yargs.argv.f === "string") {
        files = [...yargs.argv.f.split(",")];
      } else {
        console.log(chalk.yellow("Please enter filenames or -a following -f"))
      }
      return files;
    } else {
      console.log(chalk.yellow("Wrong argument: use [start]"));
    }

After:

const readDirectory = (yargs) => {
    let files = [];
    if (yargs.argv._.length === 1 && yargs.argv._[0] === "start") {

      ...

Commit the changes as extract fileDirectory(), put that into fileRead.js

Step 4 : `Split index.js` and `yargs.js`

Now let's move those functions to fileRead.js:

const fs = require("fs");
const chalk = require('chalk')

const fileRead = (file) => {
    ...
}

const readDirectory = (yargs) => {
    ...
}

module.exports = { fileRead, readDirectory }

And index.js should be used fileRead and yargs needs 'readDirectory'
index.js:

// Start testing files each
files.map((file) => {
  const fileData = fileRead(file);
...

yargs.js :

...
const files = readDirectory(yargs);

module.exports = {
  files,
  arg: yargs.argv,
};

Step 5 : Squash the commit and merge!

We have currently 4 changes. However, we only need the last finalized version of changes. We can do this using with git rebase master -i
It enable us to squash all the previous changes and make them a one single commit

pick 4710009 split index.js to index.js and fileRead.js
squash 0c01069 extract fileDirectory(), put that into fileRead.js
squash fd8e932 variables name changed
squash f9d85bf squash index.js and package-lock.json

Also, we can change the commit name with git commit --ammend:

commit d740f80e8c0e2b1d4c9d50613e5812016bcf86de (HEAD -> master, origin/master, origin/HEAD, refactoring)
Author: klee214 <klee214@myseneca.ca>
Date:   Fri Oct 23 12:25:19 2020 -0400

    Refactoring findBreakUrl to improve code maintainability

    *extract fileDirectory(), put that into fileRead.js

    *variables name changed

    *split index.js to fileRead.js and index.js
...

More information

Visit and see this

DEV Community: KiminLee

Release 04 part3: Changing the lexer algorithm

Read the previous blog please (this blog is written assuming readers already read the previous blog)

Before start

Problem

Solution

Solution 2

Conclusion

The last lab: git tagging, and publishing to npm

git tagging

v0.9.0

release 1.0.0

What is the problem, what would be the best solution?

Make another lexer.js vs use the original one and update it

What does the lexer.js really do in the program.

inside the lexer.js

The thing to be upgraded

next step

Release 0.4: Part1, choosing a open-source project

First try and failure

Small but needed to be changed a lot!

Plan for this

Added unit-test and CI to the findBreakURL program!

Should we run the program every time?

One test module will make your life so much easier

JEST is the BEST

FileRead.test.js

axios_fetch.test.js

Add an infinite scroll function to Telescope

getting started

Implementation

Migrating Weather application to React.js

Before getting started

How is the weather?

What is the issue?

Install React.js

Make a component structure

Make a migrating a program from handlebars to .jsx

What's the next?

Installing and setting up eslint and prettier configuration!!

Introduction

There are two possible solution to solve this problem

1. Make a good documentation to give a guideline for contributors

2. Force them to follow the style anyhow

prettier

first you need to create configuration file

eslint

.vscode

Example with findBreakUrl

Update: Take a API from telescope API for findbrokenURL

Before start

Installation

Issues

Running Telescope

Update: enable to take .json from telescope API

Last commit for "2020 hacktoberfest"

getting start

Conclusion

GreenLight : Added a syntax checker feature!

Introduction

Getting start

Step 1: Testing

Step 2: Spot the exact code file that should be improved!

Step 3: Inside transpiler.js

Step 4: Inside syntax_checker.js

Step 4: Added bracket checking feature

Step 5: Testing and Wrap up

Refactoring findBreakURL to improve maintainability!

Before Start

What should be changed?

Step 0 : Make a fileRead.js

Step 1 : Make a fileRead() function

Step 2 : Make a fileDirectory() function

Step 4 : Split index.js and yargs.js

Step 5 : Squash the commit and merge!

More information

Make a migrating a program from `handlebars` to `.jsx`

Update: enable to take `.json` from telescope API

Step 3: Inside `transpiler.js`

Step 4: Inside `syntax_checker.js`

Step 0 : Make a `fileRead.js`

Step 1 : Make a `fileRead()` function

Step 2 : Make a `fileDirectory()` function

Step 4 : `Split index.js` and `yargs.js`