DEV Community: Serhii

Prefer strict types in Typescript

Serhii — Mon, 06 May 2024 13:49:24 +0000

Types should be as close to reality as possible. Otherwise, they mislead (reduce truthiness in the system) and, thus, slow down development.

Usually, it's self-evident in many statically typed languages, but it's a different case with Typescript since it allows more types flexibility.

any

any may be a favorite type for many developers who migrated from Javascript recently. It declares that a type could be anything. There are some use cases for using any, but usually it's recommended to not employ it. One such example I could think of is silencing the Typescript compiler to run a script quickly. Otherwise, it detracts truth from a system and eliminates the benefits of using Typescript.

If you don't know what your data looks like, how are you going to handle it properly? Without proper handling, a system (or part of it) fails.

The other disadvantage of using any is silencing any potential errors you make when handling such data, which leads to another system failure.

const anything: any = 'word';
const result = anything * 5;

If you remove any declaration from the anything constant, Typescript tells you what is wrong exactly.

What if I don't know the exact type of data? Imagine I retrieve it from a third-party API that may change tomorrow. Or, from a non-typed npm package because it's written in Javascript. This is where we use unknown.

unknown

The useful unknown type states that a variable shape is not known and can be anything, the same point as any declares. The only difference is that unknown requires the compiler to check a type first before operating on data.

const anything: unknown = 5;
const result = typeof anything === 'number' ?
    anything * 5 :
    undefined;

The primary use case for unknown is to mark variables that are unknown and can be anything, so you must validate a type first before manipulating data.

If data shape is known, it's always better to specify an accurate type.

Narrowing

Narrowing is a process of giving types more accurate shapes. I.e., bringing them to reality, so they reflect a real data form.

To illustrate the motivation behind it, take a look at this code:

const record: Record<string, number> = {
    field: 2,
};
const result = record.field2 * record.field3; // NaN

Yes, record, is Record<string, number>, but it's not accurate enough. Record means an object with who knows how many fields and their values are of type number. In this case, a more accurate type is { field: number; }. In this example, you can skip declaring a type because the compiler can infer it automatically.

Another example of bad narrowing:

const fn: Function = (arg: number) => {
    return arg * arg;
};

fn('string'); // NaN

Yes, fn is a function, but this type doesn't specify the details (and as in the case above, you should skip declaring a type explicitly because the compiler will infer it for you automatically).

Common types to narrow

For these examples, T can be any type.

Record<string, T>: prefer an accurate object shape, if you know it. E.g.,

type MyObject = { name: string; }

T | null | undefined means you need to handle both null and undefined cases besides the presence of value. Prefer T | null, T | undefined, or merely T.
Function: prefer an exact function signature, e.g.:

type MyFunction = (arg1: number, arg2: string): string;

Partial<T>: prefer a more accurate type.

Originally published on absolyd.com

Having a type system is more productive

Serhii — Sun, 05 May 2024 11:56:19 +0000

A type system enables multiple benefits to developers. At least, it uncovers data flow complexity.

A codebase with types becomes more intuitive and connected, primarily by making data flow visible. So, at each moment of time when I work with code, I know the input and the result after transformation or some processing.

On the contrary, working with a language like Javascript provides more flexibility. However, the cost is huge: you hide data flow complexity while the problem is still there. By silencing the issue, we don't solve it, even though we can postpone encountering the consequent problems that stem from this root issue.

For example, I may send to a function invalid input without knowing it (assume this function is large and complex to understand), which results in a runtime error, that we'd like to avoid.

A type system tells what's wrong when a developer writes code. Thus, it's easier to catch problems before they arise in a production system.

The code becomes more readable since you now understand the interconnectedness between various components better. Both writing and reading code are faster.

Originally published on absolyd.com.

26 No-code Tools to Speed Up Your Idea Release

Serhii — Sat, 16 Dec 2023 08:32:00 +0000

No-code tools are a game-changer for software engineers and non-coders. They empower them to build, automate, and integrate various applications and services without writing much code, most of the time. This opens up a world of possibilities, allowing us to focus on more complex tasks while accelerating the development process for routine tasks.

Let’s take a look at some remarkable no-code tools that can enhance your productivity:

1. Zapier

Zapier is a versatile tool that lets you connect and automate tasks between different web apps without writing a single line of code. Whether it's automating emails, updating spreadsheets, or integrating your favorite apps, Zapier has you covered.

2. Integromat

Integromat is a powerful automation platform that simplifies the integration of various apps and services using visual workflows. Create complex automation scenarios with ease, all without diving into code.

3. Microsoft Power Automate

Microsoft Power Automate (formerly Microsoft Flow) offers seamless workflow automation and integration with Microsoft services and other applications. Leverage your expertise in Microsoft tools without extensive coding.

4. Airtable

Airtable combines the simplicity of a spreadsheet with the complexity of a database, enabling you to create custom workflows and applications effortlessly.

5. Parabola

Parabola is a no-code data integration tool that simplifies data manipulation and automation, freeing you from repetitive coding tasks.

6. Bubble

Bubble is a no-code platform that empowers you to build web applications with database functionality, providing a coding-free way to bring your ideas to life.

7. Adalo

Adalo offers a no-code platform for creating mobile and web apps using a visual builder, making app development accessible to non-coders.

8. AppSheet

AppSheet allows you to create no-code mobile apps directly from data sources like Google Sheets, streamlining app development.

9. OutSystems

OutSystems is a low-code development platform that combines visual development with full-stack capabilities, providing a bridge between no-code and traditional coding.

10. Kissflow

Kissflow is a no-code platform tailored for building and automating business processes and workflows, helping you optimize efficiency.

11. Quick Base

Quick Base provides a no-code application building platform to create custom business apps, making application development more accessible.

12. Retool

Retool is a low-code platform focused on building internal tools and workflows quickly, streamlining your internal processes.

13. Tonkean

Tonkean is an AI-powered no-code platform that assists businesses in automating workflows and optimizing processes, allowing you to work smarter.

14. Coda

Coda combines documents, spreadsheets, and apps into a single platform for teams, offering a comprehensive solution for collaboration and automation.

15. Stacker

Stacker enables you to build web applications from your Airtable data without writing code, simplifying application development.

16. Ninox

Ninox is a no-code database platform for building custom business applications, offering a versatile solution for your development needs.

17. Adalo

Adalo is a no-code platform for building mobile and web apps with a visual builder. It provides an intuitive interface for designing and developing apps without coding.

18. AppSheet

AppSheet allows you to create no-code mobile apps directly from data sources like Google Sheets. It simplifies the app development process by generating apps based on your data.

19. OutSystems

OutSystems is a low-code development platform that combines visual development with full-stack capabilities. While it leans toward low-code, it offers a no-code approach for certain tasks, making it versatile for different development needs.

20. Kissflow

Kissflow is a no-code platform for building and automating business processes and workflows. It offers a wide range of pre-built applications and templates to streamline various business operations.

21. Quick Base

Quick Base provides a no-code application building platform to create custom business apps. It offers a visual interface and powerful customization options for tailoring apps to your specific requirements.

22. Retool

Retool is a low-code platform focused on building internal tools and workflows quickly. While it offers some coding capabilities, its visual development environment allows for rapid app creation with minimal coding.

23. Tonkean

Tonkean is an AI-powered no-code platform that helps businesses automate workflows and optimize processes. It uses AI to intelligently orchestrate tasks and data across various systems.

24. Coda

Coda combines documents, spreadsheets, and apps into a single platform for teams. It offers a unique approach to building interactive documents and applications without traditional coding.

25. Stacker

Stacker lets you build web applications from your Airtable data without code. It simplifies the process of turning your Airtable data into user-friendly web apps.

26. Ninox

Ninox is a no-code database platform for building custom business applications. It provides robust database and app-building capabilities without requiring extensive coding skills.

Originally published at absolyd.com.

How to create a story game with Javascript

Serhii — Thu, 27 Oct 2022 11:00:21 +0000

If you don't know Javascript or what that is, don't worry! You'll be able to learn along the way or even go without it at all. What we are going to explore is an open-source tool to build interactive stories - Twine(or see its website).

It allows us to make nonlinear stories. It can be presentations, fiction, or text-adventure games. In a nutshell, Twine creates an HTML file with all the content. So, it looks like a usual web page, and the tool is the constructor. You can upload such a "website" on the web or mobile platforms, i.e., anywhere that can parse HTML.

An example of a non-linear story (interactive fiction)

If you're particularly interested in games / interactive fiction, see an extensive collection of them. So what sort of games are possible? All of them since we can add Javascript. However, the easiest way is to do text-adventure games. Why? You don't need to program anything or much. You add stories and connect them in a meaningful way using links. I.e., if you choose "A", the story "AA" will be shown, if you choose "B", the story "BB" will be shown.

See also what you can create with the tool from their docs.

An example of an actual Twine page with a story

Creating such a page takes a few minutes on the Twine website(you can do that online). You can add Javascript, videos, music, images, customize styles via CSS and do the linking and playing around. It's a great way to start. Next, I will show you a different approach if you're used to coding in any developer's editor and want version control.

Try online first, especially if your story/game is small and only needs a little custom functionality.

Why you may need Javascript

You can go without it and feel fine. But if you have custom stuff, you'll use some macros and scripts. What are they? Imagine Twine as a core, but it has various engines that do things differently. It supports four such engines(story formats) to make the creation process more accessible. Each of them varies in complexity. In this article, I'll be using SugarCube.

This story format has many built-in things you may need. For example:

Saving a game, resuming it from a save.
Various events to react to. E.g., when a story is rendered, started rendering, etc.
Macros, i.e., useful built-in blocks/functions. For example, buttons, custom links, conditional rendering, setting variables, DOM manipulations, etc.
Audio handling.
And many other valuable things.

Let's create a simple project where we want to use custom Javascript and CSS styles, but more importantly - we want to have version control! I don't use the tool's online or desktop version because I can only manage stories as files and have their versions by commit.

You'll need to install Tweego, a tool that can parse stories as files in any preferred text editor. Be aware of its limitations, though:

When writing this article, the last update of Tweego was two years ago.
Thus, you may not have all of the features from the supported story formats(e.g., Sugarcube).

Now you need to create a project folder:

$ mkdir twine-project
$ cd twine-project
$ git init

You can move the Tweego executable to this folder as well and add it to .gitignore

It's up to you how to organize files now! An example structure may look like the following:

.gitignore
README.md
bin/
src/
├─ config.tw
├─ styles/
│ ├─ menu.css
│ ├─ main.css
├─ modules/
│ ├─ menu/
│ │ ├─ menu.tw
│ │ ├─ menu.js

In the bin folder you have the Tweego executable to build the output to HTML(we'll get to that). All the story(game)-related code is under src folder. Tweego will put all Twine(.tw) files, CSS styles, Javascript scripts into one HTML. Therefore, it doesn't matter what project structure you have.

Twine format

Now, closer to the coding: what is config.tw? This is where your code will be in Twine format. Take a look at the specification. You may name this file whatever you want. It's named config for readability. There, we specify the settings for our game:

:: StoryTitle
My first game

:: StoryData
{
    "ifid": <a serial number of your game>,
    "format": "SugarCube",
    "format-version": "2.30.0",
    "start": <a name of the story that will be shown first>
}

:: StoryAuthor
<your name if you want>

<<script>>
// in case you'll need to have 3rd-party scripts
// remove this <<script>> section at all for now
importScripts(
    'https://cdn.jsdelivr.net/npm/chance'
)
<</script>>

You need to generate a serial number for your game, i.e., IFID. Read more about how to do that here. But for now, you can use 0000A000-A0E0-00C0-00B0-CF000E000D0E to skip this boring step.

format tells Tweego which story format to use. We'll use SugarCube. format-version is a version for this story format, currently supported is 2.30.0 only. However, there are newer versions(a limitation of Tweego).

start is a story that will be shown first. Let's create a file start.tw with this content:

:: StartOfMyGame

This is the first screen of my game, yay!

[[Start playing]]
[[Read about the author]]

The :: here indicates the ID of your passage(i.e., a page). It can be anything, e.g., :: start-of-my-game or :: something like this. Now that you have the ID, change your config.tw to have:

"start": "StartOfMyGame"

After the passage(page) ID, you do whatever you want. In our case, we wrote, "This is the first screen of my game, yay!", and it'll be rendered as regular text, that's it! The [[Start playing]] thing is a link to another passage(page).

To build that to HTML, run Tweego(it'll be watching for files changes):

$ ./bin/tweego -w src -o ./output/index.html

Here, we're telling it to watch the src folder and build an output HTML into the output folder as index.html. Run this command, and you'll see the HTML output in that folder. Don't forget to add output to .gitignore. Open output/index.html in a browser and you'll see something like this(with a more dark background color):

We create the links, but we also need to create such pages. So, we need to change the start.tw:

:: StartOfMyGame

This is the first screen of my game, yay!

[[Start playing]]
[[Read about the author]]

:: Start playing
<<back>>
It's another page called "Start playing".

:: Read about the author
<<back>>
I'm the author. This is my page.

We've added two more pages, so whenever you click on, for example, "Start playing", you'll be redirected to the "Start playing" passage:

We see a new link here - "Back"! <<back>> is a SugarCube macro that redirects a user to the previous passage(StartOfMyGame). It's a more convenient way of doing that than storing a navigation history each time.

We might create these two new passages in the other files or create all the game passages in one file. It doesn't matter because Tweego puts all of the files together into a single HTML file. You don't need to care about importing something!

Adding Javascript to Twine stories

Let's imagine we want to store some information about a player's choices. There are two approaches:

We may use the <<set>> macro.
We may use Javascript.

When using <<set>>:

:: StartOfMyGame

This is the first screen of my game, yay!

<<link "Start playing" "StartPlaying">>
    <<set $choice to "StartPlaying">>
<</link>>
<<link "Read about the author" "AboutTheAuthor">>
    <<set $choice to "AboutTheAuthor">>
<</link>>

:: StartPlaying
<<back>>
It's another page called "Start playing".
The choice is <<= $choice>>

:: AboutTheAuthor
<<back>>
I'm the author. This is my page.
The choice is <<= $choice>>

A few new things here:

<<link>>macro does the same as [[]], but it adds more customizability. In our case, we kept the link text, but indicated a different passage ID(StartPlaying, e.g.). Also, we can do something when a link is pressed, e.g., a <<set>> instruction below.
<<set>> macro stores a variable.
<<= $choice>> is a macro to evaluate expressions. In our case, it's displaying $choice variable we set before.

We can achieve the same using Javascript(however, it seems unnecessary complicated in this example):

:: StartOfMyGame

This is the first screen of my game, yay!

<<link "Start playing" "StartPlaying">>
    <<script>>
        State.setVar('$choice', 'StartPlaying (Javascript)')
    <</script>>
<</link>>

:: StartPlaying
<<back>>
It's another page called "Start playing".
The choice is <<= $choice>>

I removed the second passage to not duplicate the code. Here are a few new things:

We still do the scripting inside <<link>> macro. But we use <<script>> macro now. Inside of it, we use a global object State's method setVar which does the same as <<set>> in the previous example.
We still display the $choice variable not using Javascript, but we could find that HTML block using jQuery(which is built-in in SugarCube scripts), and then set the value of it to $choice, but it's unnecessary.

When you use Javascript, you have access to the story format's APIs, so it's more customizability. However, you may not encounter such complexity in your game.

That's it for now! There are more things to do in a game, of course. But you have the documentation and tools to discover and learn more independently.

Source

Next.js vs. React: what should you use?

Serhii — Tue, 25 Oct 2022 04:27:48 +0000

React is a tool for building a front end. Next.js serves the same purpose. However, both instruments offer a slightly different set of features. More precisely, Next.js gives you more than React. But it doesn't mean you should choose it every time.

A difference between Next.js and React: Next.js gives you more features

Advantages of React.js

If you don't know what React is, it's an instrument that simplifies the development of web pages. It allows you to build components(thus, separating your pages and UI elements into convenient parts) using an advantageous markup(JSX).

Angular and Vue do the same job with some implementation differences. However, these tools bring us convenience in developing the web.

With React, it's simple to start a project(use create-react-app), and we have a bundler and a compiler. The next step is to run the app with one command. "We got it. It's simple, fine, fine!"

To compare React and Next.js, let's look at what Next allows us to do.

Next.js benefits

A satisfactory tl;dr would be: " Next.js is React but with more features." Next is a framework that uses React. React is a library that gives you the pros you read above, but also it gives you freedom of choice. You may use a different compiler or bundler, or you may use it in the react-native platform. Also, you choose your set of tools: a routing system, an SCSS compiler(if you need SCSS over CSS), and so on.

Next.js gives you React but with an improved developer experience. You don't need to spend time making the choices. But if you want, you can, of course.

Let's look at the specific benefits of Next over React.

Next.js saves you time, while React gives more choices, which you may not need

Routing

Why care: you don't need to spend time choosing a router. There are many routers for React, and you may not know the best fit for your situation.

Next has the router built-in. You only need to put your pages in the src/pages folder. So, the "contact.tsx"(tsx is a JSX with Typescript) file will be available as a page at "yourAppAddress/contact". There is dynamic routes support too.

Static pages generation

Why care: if you need static pages for your app(e.g., a landing page that leads to the app), and you don't want to make them dynamic(because of the performance), you'll need to configure a server that will serve HTML pages in addition to your React app.

You can generate static pages with Next.js because it has a built-in server.

Why care: the same thing as above, except that you want to generate a page upon each request.

Static generation outputs the HTML that will be reused on each request. The Server-side rendering method will generate HTML upon each request. The former is recommended because of better performance.

Data Fetching

Why care: if you want to fetch data skipping CORS or get data from the database directly from your app or read files, you'll need a server.

Next.js data-fetching methods allow you to do all of that without configuring a server. For example, you can generate a static page using fetching local files at build time. Or, you can request a third-party server omitting CORS because Next has an intermediary - an API server.

Image optimization

Why care: you don't need to think about image optimization. I.e., how to reduce an image size, what format to choose, and how to lazy-load the images.

There are usually two apparent problems with images:

Image size. I.e., reducing a 3MB JPG into a 40KB Webp.
Loading them after a page is rendered to reduce the network load.

Next solves these problems for a developer, so they can think about the other parts of the app. All the images inside that use next/image will be optimized and lazy-loaded. Also, you may set a loading priority for each of them.

It uses SWC

Why care: you need faster build times.

Next uses SWC, which is, according to their claim, " 20x faster than Babel on a single thread and 70x faster on four cores". Such speeds are a good enhancement during build times and "hot" reloading.

Built-in ESlint

Why care: you spend time configuring ESlint.

Imagine you wrote a few lines of code and already have the lint issues warnings.

Built-in TypeScript support

Why care: you want to write in Typescript, but configuring it takes some time.

This feature is a huge dealbreaker for me because configuring Typescript takes time. Sometimes it's merely installing the package(npm install typescript), and adding the basic tsconfig.json. Still, you need to configure it correctly according to your project needs. Since you have various files in a Next.js project(stylesheets, public files, JSX, js), imagine going through them to figure out what can be ignored, what rules(configuration) to add, and then to have to break changes with the framework updates.

Next.js does it for us, yay!

Environment Variables

Why care: if you want to store some private information like secrets, API keys, etc., and you don't want to expose them in an app's bundle. You may create a server to pull that data from or use some third-party provider, but it takes time and one more place for your code.

The framework supports .env files: create a .env.local, add the secrets, and nobody will see them. You may also expose some by prefixing them with NEXT_PUBLIC_.

Scripts handling and performance

Why care: you're thinking about lazy-loading some Javascript scripts(3rd-party ones, too), loading them first, or setting any other priority.

Next.js can do all of that if you use \<Script\> instead of the standard \<script\>. Also, it can offload the scripts to a web worker using Partytown.

It has a server

Why care: you need a server environment. E.g., to make database queries.

We talked about it in the Data fetching section, but it's a good point to highlight one more time because you may need a server for not data fetching only. For example, maybe you need to do some filesystem-heavy logic and then fetch the results via Next.js API server.

So, why React, then?

Next.js has more features than I described here(those I like personally and wanted to highlight). So, we come to the question of why we need React when starting a new project.

Sometimes, you don't need such a beast for a simple use-case. For example, if you have a single static page and you don't fetch any data. Also, you don't have much dynamic logic. Thus, there's no need to bring all of the mentioned features because you need to learn the framework(though its docs are great) and the resources to support the build overhead. I.e., why configure the Next.js CI process if you can have merely an HTML page?

Choose Next.js for a better developer experience, so you don't need to spend time configuring some tools but coding.

Original article

Why do I use Next.js in my projects

Serhii — Mon, 28 Mar 2022 07:29:23 +0000

It simplifies development workflow, provides many built-in solutions and optimizations, simply put. Next.js is a React framework that employs the latter to make a developer's life easier when creating web apps.

How did I develop web apps/websites before

It's sort of a retrospective on how things were before fancy technologies. It's a short journey on how do we come up with React. Feel free to skip it and read more about Next.js in the next sections.

Pure HTML, CSS era: simple websites

An old and simple website written in HTML

When I was getting started with web development, I put simple <div> blocks into Notepad, adding some CSS styles. It's straightforward. I don't need to know any frameworks, additional technologies. A few more <div> blocks, then some text and buttons, and I got a sidebar, a menu, and a content block with text.

The problem with this approach begins when I want to add more pages: I should copy-paste those sidebar and menu codes into the new page. As a result, I should amend the code on every page if I want to change, say, the color of the button placed on every page in a header.

Not an ideal method, but it works well for a plain one-page website.

Using template engines to re-use components

A template engine like Pug allows creating atomic components to avoid code duplication

There were(and still are) tools that decreased code duplication. Tools like Pug and others are Javascript scripts that compile custom HTML syntax into standard HTML. With their help, a developer can create one component(e.g. a header with links) and put it on many pages without duplicating the component code. If I need to change the component style or structure, I alter the code in one place.

They also provide helpers to format HTML. For example, Javascript functions to capitalize text. Now it's easier to build a multi-page website. I create multiple fundamental components for my website and I re-use them on almost every page.

From websites to web apps: handling an app's state

There was a time when businesses started to replace desktop apps with web apps. The other ones wanted to automate many things for customers. They began creating websites to receive orders, for example. When we create a many-page website with custom logic to handle users' input, sending data to a server, showing errors, etc., it becomes harder to manage the state. State? We need to store, memorize what users want, what did we show/tell them, and so on.

Handling a web app's state

We may keep state on a server, but in such a way we add more complexity there. Also, it's not always convenient. For example, I want to store a lot of info about the current user and use it immediately on the next page, then forget about it. Say, it's transforming a picture into a black-and-white one via Javascript. It'd be more efficient to not load our server and make such a transformation client-side.

We could store a state, or even many states, in a browser's local storage, or IndexedDB, right? Correct. However, we then need to have a logic to update our UI if the state changes. We might update everything when anything in our state changes, but then another issue appears. Our website's responsiveness isn't fine: performance is bad.

React, for example, solves the components issue(templating), state management problem. With it, I may create optimized web apps, simplify interactions between many components. Why choose React or Vue? They ease the development of complicated web apps. Handling local(component-level) and global state is simpler. Re-using components is much simpler. These tools are libraries, not opinionated, hence not giving you plain constraints: what you should and shouldn't do. Instead, they tell you rules of good behavior - what code is better to avoid, for example, when writing components.

React: pros and cons

We talked about how it was difficult to not duplicate code in other web pages. React allows a developer to create small, large components and re-use them anywhere. We may include small components into bigger ones and include them in larger components. Thus, we diminish repeatable code and we have full control of a local component state. Hence, we optimize a web app(or, we make it worse).

Pro: it's performant because of virtual DOM

React creates an additional layer of API of DOM to optimize performance. Since the library allows you to create UI consisting of many components, it tries to update an as small number of them as possible to avoid the standard load on a browser's re-rendering. By standard, I mean not updating the entire web page when we have only a minor state change. Why re-draw all elements in DOM if we can re-draw the ones that depend on the modified data, right?

Pro: a massive community

Any great technology is backed up by a large community. If it's not large, then it's difficult to find answers on weird(or any) issues on Stackoverflow, or, to find a library to draw a simple green circle. In the Javascript world, this community is among the largest. The same applies to React, a part of it.

A large community provides many already built solutions for a web app

If I need to make a well-styled table in React, I may find it in a moment on Google. If I have hard times figuring out some peculiar behavior with the library, it's easy to find an answer.

Cons: not for me

React works well for the problems it solves. It's easy to instantiate a new project and start coding. For me, it's not difficult to learn. And, later, it speeds up the development. Then why this article about Next.js?

Next.js advantages after using React

React community created create-react-app CLI tool to instantiate a new project and see the result immediately. And I liked that for my simple web apps.

However, my next projects had public pages, a blog. A solution could be to set up a server and render the pages there, to return prepared HTML. Or, to bring other tools: a blog engine, a static site engine.

Built-in server/static rendering

Next.js cares about this and lets a developer continue writing code. It's why I didn't need to use other tools to build a blog or a regular page optimized for search engines and performance.

Why care about server rendering?

When exposing a web page to search engines, it's better to provide them HTML page without Javascript. Google, for example, can also understand Javascript there(hence a regular React app can be "understood"), but it sends a website to a rendering queue, which takes more time than processing pages when a Googlebot is reading a website right now.

Server rendering may come with a complex setup if one tries this for the first time: I need a server, at least, to render those web pages there.

The other thing intertwined with the previous one is improved performance for users(and search bots). A server renders a page and browsers show it without the necessity to compile Javascript. It comes with an additional load on the server though: it should render a page for every user(especially if the page consists of dynamic data).

We could cache such pages or make them not so dynamic. Anyway, they will be fetched and shown fast for a user. Consequently, our web vitals become better.

Next.js allows me to create a page that is pre-rendered by default. Either at build time(static generation, pages are reused for every request) or at a compile-time(SSR, hence rendering on every request). Static generation is a preferred way since it generates pages at build time. Even if I have dynamic data on the pages but I can fetch it at build time, it's an ideal way to optimize performance.

Programmatic SEO

Programmatic SEO is a technique to automate creating a lot of web pages that target almost similar keywords. They may have one search intent but it varies in details. Imagine you have an agency that lists realtors in different cities and countries: to help people who look for an apartment to find agents quickly. You make a website with the primary keyword "best realtors". However, this keyword is popular and a new website won't gain traction from Google. Hence, we may target long-tail keywords: we save the primary intent("best realtors") but specify a location. In this way, "best realtors in Kyiv" isn't as popular as "best realtors" and we target more specific intent on our web page.

However, we'd require more time to build such pages one by one, right? Most of the code could be copy-pasted, only the list with realtors differs. If we target "best realtors" in 10,000 various cities, it's a lot of work. What if we want then to target for "best agents in Kyiv"-like keywords? We alter a word, and we require additional 10,000 web pages.

Here's the thing: we have the realtors lists in various cities already, we have a web page template. With Next.js, building a lot of pages for programmatic SEO is simple. Have a list of keywords, have one code(template) and the framework will build a page for every keyword you specified.

Built-in routing

Remember how do you choose a router for a new React project? They're almost the same, have some differences, and some libraries are stable. With a Next.js project, I don't need to think about what router to use this time(maybe some other library?). It has a built-in router. And it's a simple one: you put a file in the pages folder and it becomes a page.

For example, I want to make a page with the URL <myapp>/sign-in. I put sign-in.tsx component or sign-in folder with index.tsx component there and it works.

The router provides all the necessary features: dynamic routes, nested routes, linking pages.

Built-in API server

There's pages/api folder where I may specify API endpoints. Some good use cases are:

API middlewares. When I want to alter a request(e.g. its headers) to make another one to a different server, for example.
Request resources on other domains. To avoid the CORS issue(an example project). Oftentimes, I need to request some 3rd-party resource, but I can't due to CORS. One solution is to make a server, make the request there, and then redirect it to the 3rd-party. Let's save time though. For simple requests(and even a bit complicated, Next.js can handle it too), it'd better use pages/api.

Build-in images and fonts optimization

Next.js built-in image optimization

I don't want to optimize such assets on every page I code. At the beginning of some project, it's usually not a top priority. But it's great when the framework does the 80%-like(there are other complicated optimization tricks we usually don't need) job at optimizing assets. You paste an image and it's automatically smaller, without layout shifts, and it lazy-loads!

Environment variables

There are 2 environments that are available when you code a Next.js app: a browser, a server. Environment variables in a browser are visible if one wants to find them in the bundled code. The server variables are hidden since users don't have access to remote hosts and their code.

With Next.js, I can create both env. variables without setting up webpack and installing another library to handle env. vars for a server. The framework has built-in support for them!

There's a .env file where I specify server vars, and in the same file I may specify browser vars by adding a prefix to a var - NEXT_PUBLIC_. Incredibly convenient to me.

Summary

Next.js saves me time a lot with current and new projects. I can combine a few apps seamlessly and have one codebase only. The advantages I wrote about may not apply to your use case though. If you use React now, it's worth trying: the framework adds up useful features on top of the React pros.

If you have questions or feedback, let me know.

Implementing push notifications with Firebase for Javascript apps

Serhii — Fri, 21 Jan 2022 18:39:22 +0000

These notifications we're used to on smartphones are available in browser environments too. If you know why are you here exactly, skip this part and jump directly to the solution.

And these notifications on the web that we hate because every spam site asks our permissions about sending us their spam. However, the technology itself is useful. I didn't implement web push notifications before. There was no such case, but recently I needed to notify my users about important information as soon as possible. Email is good, but a user should open an email client first. With notifications, you see the popup immediately.

I decided to write this article because I didn't find comprehensive information about implementing browser notifications for React, Next.JS, Node apps. We'll be using Firebase for this purpose, to save time ourselves by not building solutions from scratch.

Overview of browser push notifications

So as we said, it's like those mobile notifications but used in browsers mostly for spamming. It's worth noting, that you need to send them to a user if that's really what he subscribed to. Examples:

new comments under the user's post;
new message on some platform;
important information that should be handled fast;

The other non-important things may go to email.

How does it work

First, you ask a user permission to show the notifications. If you get an acceptance, your website installs a service worker that'll be handling the notifications. You also send the request to register this particular user in a messaging server, you'll be asking it to send a notification to someone.

When a messaging server registers your users, it'll send you a unique for your user token that you'll be using as an addressee to send push notifications programmatically.

You save the registration token from a messaging server. When you want to send a notification, you'll point out this token for the user you want to send a message to, thus the messaging server understands to whom to send the notification. When a user agrees to receive notifications, your website installs a service worker, it's a background script that'll be running on the user's browser. It's programmed to handle the messages from the messaging server. When it receives one, it'll assemble a notification to show to this user.

Messaging server? This is any server that knows how to communicate with your service worker. You can build it by yourself and code a service worker that will be managing messages from there. But we won't complicate our life and we'll be using Firebase.

Firebase push notifications

If we use Firebase, we don't care about the proper messaging server setup because we got covered. What we need is to code logic to ask for the notifications permissions, install a service worker and write a logic to send notifications from our app.

For further setup, you should create a project in the Firebase Console, and have config from there(a JSON file).

Front-end set up

I have a Next.js app, but this algorithm covers any app written in Javascript, it's a library- and framework-independent.

Install Firebase to your project so we can leverage simple functions rather than doing requests to FCM(Firebase Cloud Messaging) manually.

$ npm install firebase
# or
$ yarn add firebase

Find a place where do you want to ask a user about the notification permission. For example, it can be a button that says "Subscribe to browser notifications". On this button click, you'll be calling a function getFCMToken() written below:

import { initializeApp } from 'firebase/app';
import { getMessaging, getToken } from 'firebase/messaging';

// Replace these values with your project's ones
// (you can find such code in the Console)
const firebaseConfig = {
    apiKey: 'xxxxx-xxx',
    authDomain: 'xxxx.firebaseapp.com',
    projectId: 'xxxx-xxxx',
    storageBucket: 'xxxx.appspot.com',
    messagingSenderId: '00000000',
    appId: '0:00000:00000000'
};

export const app = initializeApp(firebaseConfig);
const messaging = getMessaging();

export async function getFCMToken() {
    try {
        // Don't forget to paste your VAPID key here
        // (you can find it in the Console too)
        const token = await getToken(messaging, { vapidKey: <YOUR_VAPID_KEY> });
        return token;
    } catch (e) {
        console.log('getFCMToken error', e);
        return undefined
    }
}

With this code, we initialize the Firebase library and write this getFCMToken() function. It retrieves a registration token from FCM, and it also asks a user for notification permission. If the permissions are accepted, only then it'll communicate with FCM to register this user. Otherwise, the code throws an error, which you catch in the catch block.

Then, you get an FCM token(a user's unique token in the FCM system), which you'll be using to send notifications. So you need to store it somewhere. Usually, you have a server where you may send the token and it'll save it in the database for this particular user. Otherwise, you won't be able to send notifications to users. It's required to have the Firebase Admin SDK, which is available on server environments.

There are some exceptions though. In some cases when you want only to subscribe users to your notifications like in a newsletter, you may not store the FCM tokens. Firebase has them and you can send the notifications manually from the Console. But it's not possible to send them automatically(programmatically) because you can't differentiate users(you don't have the tokens).

And the last thing is to have a service worker that will handle the notifications from FCM. Create a file that'll be available on the root of your web app, the file named firebase-messaging-sw.js. It should be accessible on https://yourwebsite.com/firebase-messaging-sw.js. Its contents:

// It's a static script file, so it won't be covered by a module bundling system
// hence, it uses "importScripts" function to load the other libs
importScripts('https://www.gstatic.com/firebasejs/8.2.0/firebase-app.js');
importScripts('https://www.gstatic.com/firebasejs/8.2.0/firebase-messaging.js');

// Replace the values with yours
const firebaseConfig = {
    apiKey: 'xxx',
    authDomain: 'xxx',
    projectId: 'xxx',
    storageBucket: 'xxx',
    messagingSenderId: 'xxx',
    appId: 'xxx'
};

firebase.initializeApp(firebaseConfig);

const messaging = firebase.messaging();

// Not necessary, but if you want to handle clicks on notifications
self.addEventListener('notificationclick', (event) => {
    event.notification.close()

    const pathname = event.notification?.data?.FCM_MSG?.notification?.data?.link
    if (!pathname) return
    const url = new URL(pathname, self.location.origin).href

    event.waitUntil(
        self.clients
            .matchAll({ type: 'window', includeUncontrolled: true })
            .then((clientsArr) => {
                const hadWindowToFocus = clientsArr.some((windowClient) =>
                    windowClient.url === url ? (windowClient.focus(), true) : false
                )

                if (!hadWindowToFocus)
                    self.clients
                        .openWindow(url)
                        .then((windowClient) =>
                            windowClient ? windowClient.focus() : null
                        )
            })
    )
})

That's all on the front-end side! You can test your button, on the press, it should ask your permission(a browser asks, to be precise) to send you notifications. When you allow it, you should see an FCM token(console.log it somewhere)

Sending notifications from the server

In my case, it's a Node.js server and we'll be installing the SDK for it, but the general principle is the same for other languages/platforms.

$ npm install firebase-admin

You also have a configuration for the backend in the Console. It differs from the client-side one because it has a private key that you need to sign your notification, which will be sent to FCM. Put this firebase.json file(it's called service account) somewhere to be accessible from code, you may put it as an environment variable.

Then you should initialize the Firebase library on a server start(or later if you want to control the flow). I've put this logic into a separate file:

import admin from 'firebase-admin';

import serviceAccount from './config/firebase.json';

export function init() {
    admin.initializeApp({
        credential: admin.credential.cert(serviceAccount),
    });
}

On a server start, call init() and Firebase is ready to serve you.

I won't cover storing the FCM tokens, I'm sure you know how to do it and it's not the purpose of this article. So, given you have an initialized Firebase on the server(from the last paragraph) and you have a user's FCM token, you're ready to send push notifications to a user's browser! It looks like this:

import { getMessaging } from 'firebase-admin/messaging';

// I use Typescript, you may not, but types will help you
// to understand what data structures FCM expects.
// It's an internal structure though, firebase-admin has
// good typings in the library
interface Message {
    title: string;
    body: string;
    requireInteraction?: boolean;
    link?: string;
}

// Use this function to send push notifications to a specific user
export async function sendFCMMessage(fcmToken: string, msg: Message): Promise<string> {
    try {
        const res = await getMessaging().send({
            webpush: {
                notification: {
                    ...msg,
                    icon: 'https://your-website.com/favicon.png',
                    requireInteraction: msg.requireInteraction ?? false,
                    actions: [{
                        title: 'Open',
                        action: 'open',
                    }],
                    data: {
                        link: msg.link,
                    },
                },
            },
            token: fcmToken,
        });
        return res;
    } catch (e) {
        console.error('sendFCMMessage error', e);
    }
}

Now, some details on the notification payload. Firebase supports various platforms, here I use the webpush field for my payload. FCM supports other fields:

interface BaseMessage {
    data?: {
        [key: string]: string;
    };
    notification?: Notification;
    android?: AndroidConfig;
    webpush?: WebpushConfig;
    apns?: ApnsConfig;
    fcmOptions?: FcmOptions;
}

I've tried to use notification as a general-purpose one, but I had issues with clicking on notifications, a browser didn't handle clicks(the service worker had the click handler). Plus, there were problems with showing icons on notifications. It's better to use webpush if you target desktop users. An FCM token can be for various platforms: Android, iOS, web.

Inside webpush, there are title and body that correspond to a notification's title and body. There's icon if you want your notification to have an icon. Put a publicly accessible image you want to be shown. Set requireInteraction to true if you don't want the notification to be closed after a few seconds, it should wait for the user's explicit reaction.

There's a custom link field inside data, it's aimed for the service worker to be read and handle the click on notifications correctly.

I don't know about all browsers, but my browser(Brave) doesn't handle the default notification click on the whole area, there should be custom buttons. I define them in the actions field. It seems it doesn't matter what buttons with actions I put, the "Open" button(action) will open the URL from data.link property I send.

Summary

Web push notifications aren't difficult to implement if you can use Firebase. Also, it's easy to send messages to various devices on different platforms. Only obtain an FCM token client-side(on a mobile, web, or desktop), and send notifications from a server.

Source

How to do text classification with Javascript

Serhii — Sun, 21 Nov 2021 11:23:13 +0000

A few months ago I tried to find information about doing some natural language processing with Javascript. There was not much of it. Most of the time, I stumbled upon tutorials on how to do so in Python. I'm writing this article in the hope to help someone do the same with Javascript. At least, try to. Javascript ecosystem is large, yet machine learning is mostly done in Python. For some custom(complicated) cases, you maybe will decide not to use Javascript. I'll explain why you may do so.

I want to point out that I'm not a machine learning engineer. I'll cover simple cases with no deep explanations of the underlying algorithms.

There are manageable cases when you may try JS packages that do the classification. In some others, and if you understand ML concepts, you may create custom models using TensorFlow.js.

My case seemed simple. I wanted to classify potential business problems(opportunities) for my Reddit advanced search tool. I'll tell you soon how it went, once we cover the tools. Let's start with simple cases.

Natural.js

It's a package for Node.js that helps deal with natural language. It has many useful built-in helpers. For example, it can do a sentiment analysis from the box and without any setup. Let's install it:

$ npm install --save natural

Easy sentiment analysis, right?

const { SentimentAnalyzer, PorterStemmer } = require('natural');

const analyzer = new SentimentAnalyzer("English", PorterStemmer, "afinn");
const result = analyzer.getSentiment(["I", "love", "cakes"]);

console.log(result); // 0.66

Yes, it's easy. PorterStemmer is a transformation function that converts words to theirs stems. To their original form, simply put. We pass an array of words to getSentiment function, but we may use built-in tokenizers to do so automatically.

Where is a promised text classification, Lebowski?

I wanted to show the simplicity of usage, without even training some complex algorithms. Now let's see how it deals with text classification.

The package supports the Naive Bayes classifier and logistic regression. They work differently, so try each one and see what fits your case better.

const { BayesClassifier } = require('natural');

const classifier = new BayesClassifier();

classifier.addDocument('buy our limited offer', 'spam');
classifier.addDocument('grow your audience with us', 'spam');
classifier.addDocument('our company provides a great deal', 'spam');
classifier.addDocument('I like to read books and watch movies', 'regular');
classifier.addDocument('My friend likes to walk near the mall', 'regular');
classifier.addDocument('Pizza was awesome yesterday', 'regular');

classifier.train();

console.log(classifier.classify('we would like to propose our offer')); // spam
console.log(classifier.classify('I\'m feeling tired and want to watch something')); // regular

Usually, you need a lot of examples. With a small amount of them, any method you choose(this library or a custom model) will output not the best results. Pay vast attention to your data, it's a major element in text classification. Maybe Natural.js will cover your case and you can finish the reading. If you need a more custom setup(if you think so, review your data again), read further.

Brain.js

This library helps you to build neural networks. Natural works with more simple algorithms. Neural networks are many algorithms that work as one, simply saying. They reflect the behavior of biological neurons that are great at recognizing patterns.

Now you can customize the algorithms. Specifically, you can build your own neural networks architectures - specify how many layers you need, activation functions, learning rate, and other parameters. This is where it gets trickier. There are no "gold rules" at building neural net architectures. The process greatly varies from a use case. We may use the default options in the cases like defining a color from RGB params:

const brain = require('brain.js');

// Build a default neural net
const net = new brain.NeuralNetwork();

// This is where we specify our data: input and the result(output)
// the data is an array of examples(input and output).
// And then the network trains on them.
net.train([
  // we tell it: if "r" from RGB scheme is 0.03, and "g" is 0.7
  // then the output should be "black"
  { input: { r: 0.03, g: 0.7 }, output: { black: 1 } },

  // notice that we skip some values from RGB, in this case we
  // missed "g"
  { input: { r: 0.16, b: 0.2 }, output: { white: 1 } },

  // here we point out all the RGB values
  { input: { r: 0.5, g: 0.5, b: 1.0 }, output: { white: 1 } },
]);

// This is how we run the network to get a prediction
const output = net.run({ r: 1, g: 0.4, b: 0 }); // { white: 0.81, black: 0.18 }

It's a powerful way to build such a network without understanding the underlying concepts, data normalization. Just point out a few examples and you're done. However, in reality, you need more examples for better precision.

Transforming text to numeric vectors

Now we're talking about data normalization. For text classification, we need to transform the text into numeric values because Brain.js doesn't have custom data transformation flow for regular neural nets, yet you may try it for LSTM, for example. Why convert strings to numbers? Neural networks training is a process of many math calculations, which require numbers, not other data types. You might use raw strings, they would be converted to their numeric representations, however, not to the format you(and the algorithms) probably want. What these "algorithms" do is figure out the patterns of the input to build a function that can calculate the output based on the input. So it's important how you do this transformation.

The first option , you may propose, is to convert every character to their numeric order in the alphabet. For instance, "a" is 0, "b" is 1, "c" is 2 and so on. Thus, we will have 26 possible values for every character. It means, the word "car" can be represented as [2, 0, 17]. In this case, if your task is to classify text with many sentences, your input dimensionality becomes 2D, which isn't fine, because input should be 1D. We might flatten the 2D array, but then it gets delicate. It means, the text like this "I want apples" converts to "iwantapples"(and then to a numeric 1D vector). It may be fine, yet we're not sure the network recognizes a pattern there to classify correctly.

The big problem with such an approach is that every character is seen by a net independently , not as a word. Thus, "car" is [2, 0, 17], and the resulting function(a set of functions that process the input) may "think" it's almost the same as "bar" - [1, 0, 17]. It doesn't think, of course, but the pattern says so. Thus, it's difficult to retrieve any context, we just perceive every character independently.

The second option is to do the same, but for words. In reality, we retrieve context mainly from words, not by characters separately. Such an approach also simplifies the calculations: we don't need to convert 2D input into 1D and a neural network gets fewer numbers to process, which is a performance boost. To convert words to a number, we should figure out what numbers to assign to them. You may create examples of text you will be training on, tokenize it into words(omitting punctuation because it doesn't add context), make a dictionary of these words, where every one of them gets an ordering number. It's like adding words to a Set and their number is an order in which they appear in it. E.g. if I have a text "I want apples.", my dictionary is ["i", "want", "apples"], where the word "I" will be assigned to 0, "want" to 1, and "apples" to 2.

We may optimize this approach by also stemming words to their root form, e.g. "apples" become "apple" because the net doesn't need to know(except the use cases where your task is to classify singular or plural forms) whether it's a singular or a plural form, it's better to have a numeric representation for a word abstraction - apples("apple", "apples").

It's the most simple method to vectorize text. Though, it also has problems. In cases, where you need your neural net to "figure out" the context by looking for a set of words, it's difficult because in the example above, "I" and "want" are placed as neighbors (0 and 1 accordingly), but they aren't similar, they mean different things. For example, "car" and "automobile" mean the same but can be represented as 14 and 8233 with this approach. Thus, your model may derive different results based on whether your examples have synonyms.

The third option is to use pre-generated vectors. The ones that were generated by processing a lot of texts and deriving which words are similar, and which ones are different. Thus, for example, a vector for "car" may be [0.45, 0.78, 0.97, 0.34, 0.87], and for "automobile" it may be [0.49, 0.73, 0.98, 0.33, 0.88]. As you noticed, they're not single numbers, but vectors for every word. Thus, you get a 2D array for the whole text. I'd suggest you go with pre-generated vectors such as GloVe.

Getting back to Brain.js

Now you know how to convert strings to vectors, you can use the library to help you. It has various types of pre-defined neural networks. The one we saw before is the feedforward neural net with backpropagation. This is where things get delicate too, again - in choosing the right network type. A feedforward net is a simple one that takes an input, does some calculations-transformations, and returns the results. It sees every input independently, it doesn't have a memory. It means it can't derive context from multiple words. If your task requires so, you better choose recurring neural nets such as RNN or LSTM(see the Brain.js details on them).

TensorFlow.js

This is a path where you decided you require more custom setup. This is a Javascript version of a powerful machine learning framework for Python. It allows you to build any models or use already created ones by the community. However, they don't have much. And their functionality of converting Python models to JS ones and vice versa doesn't work well enough yet.

The code may look like this:

const tf = require('@tensorflow/tfjs-node');

const data = {
    // assume we already have vector representations of the text examples
    inputs: vectorRepresentations,
    // imagine we have such 3 classes
    output: [0, 0, 2, 1, 2, 1, 0, 1],
}

// tensors are TensorFlow vectors to simplify the internal
// processing for the library
const inputTensors = tf.tensor(data.inputs);
const outputTensors = tf.tensor(data.outputs);

const model = tf.sequential();

// 1st layer: a 1d convolutional network
model.add(tf.layers.conv1d({
    filters: 100,
    kernelSize: 3,
    strides: 1,
    activation: 'relu',
    padding: 'valid',
    inputShape: [MAX_WORDS_LENGTH, GLOVE_VECTOR_DIMENSIONS],
}));

// transform 2d input into 1d
model.add(tf.layers.globalMaxPool1d({}));

// the final layer with one neuron
model.add(tf.layers.dense({ units: 1, activation: 'sigmoid' }));

// here are some tuning, read in the TF docs for more
model.compile({
    optimizer: tf.train.adam(LEARNING_RATE),
    loss: 'binaryCrossentropy',
    metrics: ['accuracy'],
});

// print the model architecture
model.summary();

// train the model
await model.fit(inputs, answers, {
    // the default size, how many inputs to process per time
    batchSize: 32,

    // how many times to "process", simply put
    epochs: EPOCHS,

    // the fraction of the inputs to be in the validation set:
    // the set, which isn't trained on, but participates in calculating
    // the model's metrics such as accuracy and loss
    validationSplit: 0.2,

    // shuffle inputs randomly to have a different starting seed every time
    shuffle: true,
});

// save the model to load in the future and run classifications
await model.save('file://./data/models/myFirstModel');

Here we built a model to do text classification for 3 pseudo-classes(0, 1, 2). We used a 1d convolutional network for the 1st layer. TensorFlow allows you to specify any amount of layers you want, set training epochs, validation split, choose different ML algorithms, activation functions for every layer, and many other options. Though, we need to know how to build ML models. If we don't, we may add anything, tune parameters, and won't receive good results.

I went to TensorFlow.js for more customizability but spent months on adjusting a lot of stuff and didn't get great results. I learned many things along the way, but still, I'm not an ML engineer, so it's better(faster) to use models built by professionals and not create your own wheel. But if it's for fun, why not! Then, let's understand the code I wrote.

I chose this architecture due to its performance: convolutional networks are faster for text processing and also they process input in a kind of context. They're mainly used in computer vision because they process input matrices, not just 1d arrays of numbers. So, for example, if you get an image, 100x100 px, a convolutional network may process 5x5 pixel window per time. Thus, some noise and details can be classified correctly. For text, it's almost the same - we need to take multiple words in a batch and don't process them independently. Thus, simplifying a model's job in recognizing patterns.

I chose GloVe vector representations, so my input data was a 2D array of numbers, where every subarray was a word representation. The kernelSize parameter in a convolutional network is responsible for the "sliding window" - those 5x5 pixels to process per time. In my case, I specified kernelSize to 3. It means the network processes 3 vectors(3 words) per time. The filters param tells how many neurons you want. strides means how many "steps" to take per once when moving the "sliding window". For example, for the text "I want to eat apples tomorrow", the first batch is ["i", "want", "to"], the second batch is ["want", "to", "eat"], the 3rd is ["to", "eat", "apples"] , and so on. So, it moves by one word per time to the right.

General learnings

I spent some time with Natural.js, then Brain.js, and TensorFlow. I went to the last one for custom configuration and spent a lot of time building custom models. It'd be better to use an already built model for text classification. However, I didn't find a good way to transform Python TensorFlow models into Javascript ones, that's why in the end I switched to a Python setup with HuggingFace. But my task wasn't so straightforward. I wanted to classify potential people's problems and pains: where someone hates using something or complaining about things.

There were some things I learned while building custom models with tensorFlow.js I wish I knew earlier. Write your experiments in a logbook. You'll be building various models with various hyperparameters and it becomes difficult to recall what worked well for you and what didn't. Also, don't forget about the test set(assuming you have a validation one too).

There is a lot of things to mention about building ML models. Here are some I highlighted in my logbook. I hope it saves someone's time in narrowing the search when troubleshooting.

When to stop training. If a validation loss starts increasing. It should be similar to but slightly higher than a training loss. If it's lower or almost equal to a training loss, a model requires more training. If training loss is reducing without an increase in validation loss then again keep doing more training.

You have 1.0 accuracy. In most cases, if you have 100% train accuracy, you've probably greatly overfitted. Or, a model recognized a "false" pattern in your data.

Overfitting? A big topic. Here's some reference(not mine, but I can't find the source):

If validation loss >> training loss you can call it overfitting.
If validation loss > training loss you can call it some overfitting.
If validation loss < training loss you can call it some underfitting.
If validation loss << training loss you can call it underfitting.

A higher than training loss validation loss means overfitting, a model learned the patterns that happen to be true in training data, but they're not in real-world data.

If you have a too powerful model(the one that has too many parameters and not much training data, see model.summary()), review it again and simplify, because for me some of the models memorized the data, and thus, have greatly overfitted.

Another evidence of overfitting is that your loss is increasing, Loss is measured more precisely, it's more sensitive to the noisy prediction if it's not squashed by sigmoids/thresholds (which seems to be your case for the Loss itself). Intuitively, you can imagine a situation when the network is too sure about output (when it's wrong), so it gives a value far away from the threshold in case of random misclassification.

Accuracy or loss fluctuates.

some portion of examples is classified randomly , which produces fluctuations, as the number of correct random guesses always fluctuates (imagine accuracy when coin should always return "heads"). Basically, sensitivity to noise (when classification produces random results) is a common definition of overfitting

The training loss at each epoch is usually computed on the entire training set. The validation loss at each epoch is usually computed on one minibatch of the validation set , so it is normal for it to be noisier.

Take care of your batch size. Sometimes it needs to be adjusted:

Smaller batch sizes give noisy gradients but they converge faster because per epoch you have more updates. If your batch size is 1 you will have N updates per epoch. If it is N, you will only have 1 update per epoch. On the other hand, larger batch sizes give a more informative gradient but they convergence slower and increase computational complexity.

Typescript tutorial for Javascript developers

Serhii — Fri, 05 Nov 2021 06:55:11 +0000

I wrote an article on why to use Typescript if you're still in doubt about whether to use it in your projects. In short, Typescript allows you to write maintainable code. Javascript works well for prototyping but becomes frustrating when you return to a project again. Suddenly, you see blocks of code where you can't understand what kind of data passes there.

In this article, I want to introduce you to Typescript while playing with code. Thus, you see it in action and don't overflow with an unnecessary theory. I recommend playing with the code you'll meet here in Typescript Playground.

Imagine you want to count items, which have the field meta that contains the field countable set to true, in an array. If an item doesn't have meta.countable, we don't count it.

function getAmount (arr: any[]) {
    return arr.filter(item => item.meta.countable === true).length;
}

Typescript array type with anything in there
Why are you using any? It's not OK! It's for an example. Don't use any in such cases. Read more about it later in the article.

We see the any keyword near the arr argument, that's new to us! I guess you already know what it is. This way we tell TS that arr is of any[] type. It literally means any Javascript type in an array. I.e. arr is an array and every item of it is of type any. It safe you from writing such code:

// Argument of type 'string' is not assignable to parameter of type 'any[]'
getAmount('string');

// Argument of type 'number' is not assignable to parameter of type 'any[]'
getAmount(29);

The compiler ensures you should pass exactly what you've pointed out as an arr argument for the getAmount function. What if you need to pass several types, for example, an array and a string? And if arr is a string, then return 0. A weird case, but imagine you work on a legacy system that uses this function in many places, so somewhen you may get a string.

function getAmount (arr: any[] | string) {
    if (typeof arr === 'string') {
        return 0;
    }
    return arr.filter(item => item.meta.countable === true).length;
}

getAmount('55'); // now it's possible to pass a string
getAmount([{ meta: {countable: true} }]);

| means "or". Thus, arr can be an array of anything(any[]) or a string. Refer to this page for more everyday types in Typescript.

The compiler is smart enough to even infer a return type of getAmount.

// function getAmount(arr: any[] | string): number
function getAmount (arr: any[] | string) {
    // because we always return a number
    // 0 or arr.length(filtered
}

Type inferring for a function that always returns a number

Sometimes, Typescript can't infer a type because of ambiguity. Usually, it's a good practice to explicitly indicate a return type of a function.

function getAmount(arr: any[] | string): number {
    // ...
}

// Syntax: don't forget this
// |
function myFunction(arg: any): boolean {/* function body */}
// |
// a type this function returns

Now you know how to write functions and point arguments and return types! In most cases, that's what you need. All other code is still Javascript. With more types. However, let's dive deeper and highlight more complicated cases and what things to avoid.

Someone may pass anything in an array:

function getAmount(arr: any[]): number {
    // ...
}

getAmount([5, "string", {}, () => {}]); // no error

That's not what we expect. TS works well in this case, we specified any[], so what problems? Don't use any if there's no real need for it. It's easier to pass any than describing an advanced type, but that's what Typescript is for. Don't shoot yourself in a foot in the future.

Typescript objects

We may want to replace any[] with object[] and it would work as we pass objects there, right? Correct, but an null and a function are also objects. It's not what we expect either. Don't use object, try to narrow types.

interface Item {
    meta?: {
        countable?: boolean;
    }
}

function getAmount (arr: Item[]) {
    return arr.filter(item => item.meta?.countable === true).length;
}

getAmount([
    {}, {meta: {countable: true}}
]); // 1

Now it works as expected. We specified a separate interface for a possible array element. Interfaces and types allow you to create your own types using basic Typescript types. Some examples:

// is also called "type alias"
type Hash = string;

// interface are "object" types and allow us
// to specify an object immediately
interface Person {
    name: string;
    isOkay: boolean;
};
// it's the same as using a type alias
type Person = {
    name: string;
    isOkay: boolean;
};

Types and interfaces

Let's start implementing a booking tickets service to dive deeper into these types and interfaces. We want to have the possibility to book a ticket for a person.

type Person = {
    name: string;
}

type Ticket = {
    from: string;
    to: string;
    person: Person;
}

function bookTicket (from: string, to: string, person: Person): Ticket {
    // some procesing
    return {
        from,
        to,
        person,
    };
}

bookTicket('Paris', 'Mars', {name: 'Joey'});

The code seems okay. However, we can book a ticket to Mars using the function, but we don't fly to Mars yet. What we may rectify in our code to adjust to reality? We could add validation for from and to fields inside the function, but we also can do this with TypeScript. For example, we could list possible locations we're flying to and from.

type AvailableLocation = 'Paris' | 'Moon' | 'London';
type Person = {
    name: string;
}
type Ticket = {
    from: AvailableLocation;
    to: AvailableLocation;
    person: Person;
}

function bookTicket (from: AvailableLocation, to: AvailableLocation, person: Person): Ticket {
    // some procesing
    return {
        from,
        to,
        person,
    };
}

// Error: Argument of type '"Mars"' is not assignable to parameter of type 'AvailableLocation'
bookTicket('Paris', 'Mars', {name: 'Joey'});

We narrowed possible options for locations. Thus, eliminated cases when we can write code that calls the function with invalid locations like "Mars" or "Andromeda Galaxy". We listed multiple allowed options via "or" operator - Paris | Moon. We might be using enums for this purpose too:

enum Locations {
    Paris,
    Moon,
    London,
}

type Ticket {
    from: Locations;
    to: Locations;
    person: Person;
}

bookTicket(Locations.Paris, Locations.Moon, {name: 'Joey'});

There are differences in using types and enums, I won't cover them this time, but you may refer to this page for the details.

As you might notice, somewhere I used interface for an object type and then declared another one via type. Use what you like more for such cases or use based on your project code guidelines. For more information about the difference, read here.

Using `Record` to type objects

Sometimes you have generic objects, where a key is always string(and it's always a string, if you want to use other values, use Map instead) and a value is always string too. In this case, you may define its type as follows:

type SomeObject = {
    [key: string]: string;
}

const o: SomeObject = {key: 'string value'}

There's another way to do the same using Record<keyType, valueType>:

type SomeObject = Record<string, string>;
// it means an object with string values, e.g. {who: "me"}

It's something new here: generics, computed types to re-use the existing ones. Let's re-create the Record type:

type Record<Key, Value> = {
    [key: Key]: Value;
}

Thus, if we want to create an object, we don't need to write such signatures every time. So, an object with number values is as simple as:

const obj: Record<string, number> = {level: 40, count: 10};

We may need more complex types, for example, to represent the state of our API requests. Imagine you have a global state where you put all the API data. Thus, you know where to show a loader, when to remove it, and to show relevant data.

type StateItem = {
    isLoading: boolean;
    response: Record<string, unknown> | null;
};
type State = Record<string, StateItem>;

const state: State = {
    getInvoices: {
        isLoading: false,
        response: null,
    },
};

Do you see the inconveniences here? We might narrow a type for state keys: it's a string, but we want to be sure we put valid API request names there. The second thing is the unknown I put for the response(an object with unknown values), yet it's still better than any, because you should determine its type before any processing.

type APIRequest = 'getInvoices' | 'getUsers' | 'getActions';
type BaseResponse = {isOk: boolean};
type GetInvoicesResponse = BaseResponse & {data: string[]};
type GetUsersResponse = BaseResponse & {data: Record<string, string>[]};
type GetActionsResponse = BaseResponse & {data: string[]};
type StateItem = {
    isLoading: boolean;
    response?: GetInvoicesResponse | GetUsersResponse | GetActionsResponse;
};
type State = Record<APIRequest, StateItem>;

// Type is missing the following properties from type 'State': getUsers, getActions
const state: State = {
    getInvoices: {
        isLoading: false,
        response: {isOk: false, data: ['item']},
    },
};

Let's disassemble some pieces of the above:

APIRequest type is a list of possible requests names. Narrowing types are for the better. See the error comment near the state const? Typescript requires you to specify all the requests.
BaseResponse represents a default and basic response, we always know that we receive {isOk: true | false}. Thus, we may prevent code duplication and re-use the type.
We made a type for every request possible.

While it's better than it was before, but we could do even better. The problem with these types is that response is too generic: we may have GetInvoicesResponse | GetUsersResponse | GetActionsResponse. If there are more requests, there is more ambiguity. Let's employ generics to reduce duplicate code.

type BaseResponse = {isOk: boolean;};
type GetInvoicesResponse = BaseResponse & {data: string[]};
type GetUsersResponse = BaseResponse & {data: Record<string, string>[]};
type GetActionsResponse = BaseResponse & {data: string[]};
type StateItem<Response> = {
    isLoading: boolean;
    response?: Response;
};
type State = {
    getInvoices: StateItem<GetInvoicesResponse>;
    getUsers: StateItem<GetUsersResponse>;
    getActions: StateItem<GetActionsResponse>;
};

It's more readable and safe to specify every request separately, thus there's no need to check state.getInvoices.response on every response type possible.

Don't use any type. Prefer unknown. In such a way, you should be checking the type before doing any further operations with it.

type Obj = Record<string, unknown>;

const o: Obj = {a: 's'};
o.a.toString(); // Object is of type 'unknown'

Prefer Record<string, T> over object, which can be null, any kind of object, a function. T refers to a generic type.
Narrow types where possible. If it's a few strings you use often, probably they can be combined in one type(see the example about API requests state).

type GoogleEmail = `${string}@gmail.com`; // yet it's still a string

const email1: GoogleEmail = 'my@gmail.com';

// Type '"my@example.com"' is not assignable to type '`${string}@gmail.com`'
const email2: GoogleEmail = 'my@example.com';

It's a new thing here: template types. Any email is a string, but if you can narrow a type, then why not(it's an example, sometimes it's an overhead).

Other use cases you may encounter

Generics in functions

You saw generics, it's a powerful way to re-use the code, the other examples include functions:

type Response<T> = {
    isOk: boolean;
    statusCode: number;
    data: T;
}

async function callAPI<T> (route: string, method: string, body: unknown): Response<T> {
    // it's a pseudo-fetch, the real API differs
    const response = await fetch(route, method, body);
    // some manipulations with data

    return response;
}

So, the syntax is function <name>:<type> (args) {}. You may use T(or other names for a generic, or, a few of them) inside a function too.

Specifying types for readability

Imagine you work a lot with variables that are strings, but it's hard to understand which is what type exactly. For example, when dealing with OAuth tokens.

type AccessToken = string;
type IdToken = string;

Both tokens are JWT strings, but sometimes it's useful to understand the context.

function callProviderEndpoint (token: AccessToken) {}
function decodeUserInfo (token: IdToken) {}

So, the syntax is function <name>:<type> (args) {}. You may use T(or other names for a generic, or, a few of them) inside a function too.

Type assertions

There are cases when you need to cast(transform to for the compiler) a type to another one. For example, when a library method returns object and you know it's not useful, you need a more narrow type. You may write const result = libResult as Record. as allows you to transform a type into a desired one(if it's possible). The easiest cast is for any types: the compiler doesn't know anything about a value, so it trusts you. There are cases when you'd want to cast something into any for compatibility, but often it's laziness to write correct types. Invest type into writing good(corresponding to reality) types.

You may also do casts like follows:

const response = <MyCorrectType>libResponse;
// the same as
const result = libResponse as MyCorrectType;

Some general questions one may ask

Should I learn Typescript?

Definitely. I presume you're already familiar with Javascript, which is simple and fast to prototype. Typescript adds type safety and readability. Your app's logic becomes more predictable. Read more about reasons to use Typescript.

How to learn Typescript?

Read the documentation about the basic types, or this article. Practice the examples by yourself and go code! Install the environment(many frameworks have their already prepared Typescript ecosystem for you to install and use instantly) and make things happen. It's okay you don't understand some things or you're tired of the compiler errors. It's getting easier.

Summary

I didn't cover all the things in the article. The mentioned above should be enough to spark your interest and learn the basics that cover most cases you'll encounter. Learn as you go. The theory might seem to be complicated and boring, it's okay. Write me on Twitter if you need more details or help.

Original

What are the common questions should be answered better?

Serhii — Tue, 17 Aug 2021 11:27:03 +0000

Hi! I've created the general programming FAQ page where I tried to cover frequently asked questions beginners ask.
For example, how to find a job, how to start learning, etc.

I want to supplement the page.

What the questions you asked or wanna ask that should be covered better? Maybe with simpler explanations?

How to learn Javascript easily

Serhii — Fri, 13 Aug 2021 14:32:34 +0000

tl;dr search for "learn javascript theory" in Google and pick any website. Go repeat what they teach in Codepen. Repeat for every theory section.

So you're prepared to learn something exciting! The first thing you need: a theory on how those things work. Only basic things such as syntax and programming fundamentals. Like what? Understand why do we need programming it at all. What things are possible, what aren't

Step 0. A place where to write and execute code. To see the result of your code. Go to Codepen and create a pen. Or choose another code editor. This is where you'll be writing programs while studying.

Step 1. Choose any website where you can learn about the language syntax. Google "javascript tutorial", "learn js" and so. The source doesn't matter at this point, pick any.

This is a great resource.

Get familiar with how to write code:

how to create a variable;
how to make a function;
how to call a function;
how to read function arguments, handle them, and return something useful;

Congrats on this milestone! You now can write basic code. Note that it isn't so straightforward to write it though in the beginning. It's okay you can't follow tutorials. Try to write basic things. Get used to the style and then go further.

Don't go to the next step until you code the practice tasks in the tutorials you followed. It's important! Don't skip the tasks!

Step 2. Learn about data structures and when to use them. There are primitive structures such as numbers, strings, booleans. And there are more complex types such as arrays, objects. You should:

understand when to use arrays and objects;
know the difference between the two;
write a lot of code using various data structures;

Don't go further if you aren't comfortable with writing Javascript code. Functions, operations with objects, arrays. If you don't understand some part, it's okay and it doesn't mean you're dumb. I was trying to learn to code a lot of times and it was hard. Go back and read the theory, then solve the practical tasks. If you can't deal with them, go to Stackoverflow and see solutions. Understand how they work.

Most of the time if you don't understand something, it's because you skipped a section. Or, didn't read it carefully. Or, the information isn't simple enough. Try following a different source. See a video tutorial. Still, finding it difficult? Write me.

Step 3. Deciding your niche in applying Javascript. With JS, you can program:

front-ends(interactive visual representation). It's websites with logic. You need HTML, CSS to create visual blocks and JS to say a browser what to do. For example, when you press a button, JS handles the press and does something useful. Sends your data to a server to save in a database;
back-ends(server-side code: Node.JS). This is where you can handle the data a website sends you. Store it in a database, or send it somewhere else. Or, transform it and save it;
mobile apps. With React Native framework you can do such, isn't that cool?
desktop apps;

Step 4. When you decide what path you want to take, learn the specifics of that environment.
For browsers(front-end path) you need to learn DOM. I.e. how to interact with visual elements on a website page. Think what projects you'd want to create - a personal website, an online store. Or something else. And create it! Make only a super simple and tiny version. Find a theory you will need when you encounter unknowns. E.g. how to find all buttons on a page and handle clicks on every one of them.

If you prefer server-side coding, you should learn Node.JS. It's a thing that helps you to run Javascript in a server-side environment. Learn how to:

create a basic HTTP server;
handle the HTTP requests. E.g. when calling GET /api/hello to return "Hi!";Then practice. Pick up the Express library. It simplifies creating code on a server. Make some HTTP routes, do some transformations of data you receive.

And, of course, create an interesting project! A server for an online store website? A Discord bot?

For mobile and desktop apps, the strategy is the same. Learn specifics about the frameworks(React Native for mobiles or Electron for desktops). Create pet projects.

Summary

The most important thing in learning Javascript fast enough is to practice a lot. Don't skip it. Thus you learn applicable things that solidify what you've been reading a lot.

Focus on one area. Solidify your knowledge in one niche, then go further after understanding the basics.

Follow me on Twitter

Reasons to use Typescript

Serhii — Sat, 05 Jun 2021 13:15:26 +0000

Typescript is a superset of Javascript. It's Javascript with type definitions and the code will be checked when compiling.

Why to adopt TS if ...

Javascript is fast to prototype

We can code fast with Javascript. The language is simple and allows us to do a lot of things. A prototyping time can be amazingly short. A downside of this is code readability. If we need to prototype something quickly to understand if the idea is viable, we skip thinking about architecture, writing documentation, tests. Eventually, we have a working piece. It has some bugs, but they can be fixed relatively fast. The piece is doing its intended job, we add more functionality to it, but some problems start to emerge.

To add a simple functionality, it's hard to understand whether some service passes you an object with 20 fields, a string, or it may not pass anything at all. It's difficult to fathom API contracts between code pieces you wrote! By writing more console.log's you see the contracts and adjust the code to new requirements.
After some time, your colleague asks a team's help to assist in debugging "what these objects are". Next month a manager asks why it takes so long to modify a tiny bit of the functionality. The company even hired 5 additional developers for the last week. In the next week, the CEO along with CTO decided to rewrite the product from scratch.

Could the situation be different if the team adopted Typescript? Not really. But TS could help to eliminate bugs initially and with the next releases. Plus, it would take less time to modify existing code. Why? Doesn't TS add more time to write, because you should think about types, write them down? We'll get back to this question.

Typescript eliminates problems when you write the code

IDEs have great TS support, so they will highlight potential errors you encounter. But, what kind of errors? Imagine you construct an object, convert it to some other form, pass it to a chain of 5 functions and then send the result to UI. It's easy to follow the object properties when constructing it in the beginning. It's difficult to follow a contract further. The next function can remove necessary fields from the object, another one will add some new ones, etc. With Typescript, you code how the object should look like, what properties are required, what are optional, and this contract will be followed. Otherwise, you'll get compile errors(and errors highlights before that).

Another situation: there is a function that should be returning a number, but returns undefined. The function is complicated and we can't exactly know where the problem occurs(i.e. why it can't handle 10).

import {calculateXfromNum} from './otherService';

// there is a function from other service
// it calculates some number from other number
// this function should return number, at least we expect that
// judging by the code
const num = calculateXfromNum(10); // num is undefined

// this function expect a number
doStuffWithResult(num); // error

It improves code readability

Have you struggled with situations where you didn't know what to expect from functions, 3rd-party services responses? Like in the example above. Types allow you to see what other functions take as arguments and what they'll return. Have you seen the console.log debugging process? To understand what form an object has, people log it to the output a few times along the function execution.

You could use debugger also, it may help more. But why make things so complicated? Isn't it enjoyable to know what this class method returns from the first glance at the code?

The subsequent question you may derive from the reasons above may be:

What about the time we spend on documenting types and writing more code? The coding won't be so fast then.

It accelerates the development process

How is that possible?
If you think you will document all the types when using TS, thus it slows you down, that's not the case. Typescript is smart enough to infer types from code. In cases where it can't do that, you need to indicate a type explicitly. For example:

const n = 10; // n is number

// here, "num" can't be inferred, because you can pass anything here
// TS can't know that
function fn(num) {
    return num + 10;
}

// if you point that "num" is number, TS knows the returning type
// will be a number too
function fn2(num: number) {
    return num + 10;
}

Still, even if Typescript can infer types in simple cases, a developer should write complex types, so how the development becomes faster?

How much time do you think developers in mid/large-size JS projects spend on debugging to understand what an object will be returned by some function? I think a lot. We don't write code for machines, but humans. The code should be understandable. You take a glance and you get a basic idea of what's going on.

Learning Typescript syntax takes a few minutes to start coding if you already understand Javascript. What you need to know is how to declare type/interface. Then, you need some time to get used to advanced cases.

The additional development time will go to defining types, which isn't so time-consuming. At least, it takes much less time than:

asking a colleague where to find documentation about a service;
asking a colleague who wrote that;
asking the person who wrote that to explain what the form of an object is;
writing comments about object type after debugging it;
rewriting the comments because you didn't expect other use cases;
Save yourself and your team many human hours of reading and understanding the code.

How to migrate to Typescript

I'll provide points for you to decide is it worth adopting at all, and how simple the process is.

It's easy to learn

Understand how to declare types/interfaces and you're good to go. Basically, it's:

type Human = {
    name: string;
    age?: number;
}

const me: Human = {name: "Sergiy"};

While you write more code, you'll get to know many other things along the way.

It's easy to start right away

If you have a Javascript project already, all it takes for you to start writing TS code is to have Typescript installed, maybe some tsconfig configuration, and to set the --allowJs flag, so you can mix TS with JS code. As you code more, you'll add more types, and maybe you convert all the codebase to TS. At least, you can TSify crucial parts of the code right now.

However, you'll need to compile the project to Javascript. An additional build step in your workflow.

If you're convinced to switch or migrate to Typescript, there are some thoughts to ponder about. As a person who leads a team, you should be responsible to explain to the developers why do you choose TS, what are pros and cons. Otherwise, programmers may think of it as a tool that slows them down. You may encounter situations when a developer write something like that:

function parse(payload: any, options: object): any {
    // ...
}

While it's a valid code, it doesn't add more readability. A developer may find using any is a good thing to save time. The idea the developers should understand is that if something is unknown, it's better to figure out the actual type and document it.

Almost the same situation with wide types like object: the data may be an object with any properties, a function, an array. The more narrow type, the better.

Summary

Use Typescript if you need better code readability, hence development speed up in the long run. It takes a little time to get started with TS if you know Javascript. You can integrate TS in your Javascript project without rewriting JS parts right away.

Typescript is a tool that should be regarded as such and not as a thing that solves abstract problems like "improving the code".

Follow me on Twitter and see my other articles.