DEV Community: Maxim

Why AI is failing at giving good advice

Maxim — Fri, 26 Apr 2024 08:55:42 +0000

TLDR: ChatGPT generates responses based on the highest mathematical probabilities derived from existing texts on the internet. Popular advice (for various reasons) is seldomly good, nor (by definition) uniquely applicable, nor (mostly) founded on actual experience. You are probably better off taking advice from a real person who can empathize and knows what they are talking about.

When you ask ChatGPT a question, something highly interesting happens:

ChatGPT, which has previously consumed half or more of the internet to build its language model, will translate your question into a mathematical representation of numbers (e.g., a vector).

I don't know in detail how they do it, and I am sure there are some layers in between and around that serve some specific purpose, but I understand that if you google the phrase "How are you?", you can statistically expect a certain range of words and sentences in the results around it. Most sentences following the question will probably sound like "I'm good, thanks" or "Doing great, how about you?". Whereas if you search the internet for all occurrences of "Integrated circuit", you will usually find a very distinct set of words and sentences nearby, like "silicon semiconductor," "MOS transistor," or "the voltage requirement is 0.6V".

With so much base data, you can assign a mathematical value (or direction) to every word, change it when it appears together with other words (context), and compute an entire, unique direction for a continuous piece of text.

Realize the following (exemplified): Anyone who ever had success on the internet writing articles (or anything else) in the broader sense of the universe just pressed a particular combination of buttons on their keyboard, and then more biological masses in the world started reading the outcome than other produced texts.

In a strange but very scientific way, when you ask ChatGPT a question, it tries to compute the exact combination of letters, words, and sentences based on their previously computed values that it thinks you are looking for. Astonishingly enough, that is often a highly useful response in the real world.

But this approach has problems, especially when you try to give someone good, specific advice:

The outcome is, by definition, mathematical. It's probability, applied to man-made text. The most propagated (related) text on the internet will likely be repurposed in its own words to answer anything that you ask. Essentially, that means it might give you a mashed answer as you would get from the X first results on Google, but it will fill in contextual gaps from other places and make it more applicable to your specific input.

If most internet texts said the sky was yellow, ChatGPT would say so, too. Similarly, suppose you ask ChatGPT the infamous question, "How can I make money online quickly?". In that case, you will get a shallow, unhelpful response (that will often stay unhelpful even if you drill down into specifics).

This is not to say that everything is particularly "wrong" (although some points are, according to most people's experience); it is just paraphrasing those online bubbles of drop shippers, BuzzFeed listicles, and affiliate boards.

For example, almost everyone who has succeeded with YouTube or affiliate marketing will tell you neither is quick. It takes years of work, dedication, and a fair pinch of scientific user analysis.

Even if you ask it to walk you through making money step by step, it fails: In March 2023 (2 days after the release of version 4), a tweet caught fire that documented the usage of ChatGPT as a business owner, giving precise directions to make money (starting with $100):

// Detect dark theme var iframe = document.getElementById('tweet-1636107218859745286-359'); if (document.body.className.includes('dark-theme')) { iframe.src = "https://platform.twitter.com/embed/Tweet.html?id=1636107218859745286&theme=dark" }

It did make some money, but with millions and millions of views and even mainstream news covering the endeavor, I am hesitant to attribute the generated income to ChatGPT. The updates died out quickly, and two weeks later, the official confirmation was posted that the project (and apparently, the site) was sunsetted. It's not what a successful attempt to make money looks like in my world.

A Large Language Model can provide accurate answers if fed the correct base context (superseding the general knowledge base) and if you ask the right questions. But even then, you need to find that respective chatbot and the questions you must ask to get helpful answers (although the latter may apply to many human conversations, too).

At the current state of the internet, there is almost any educational information and advice already out there in some form, freely accessible to everybody, more than anyone could ever take action on in their lifetime. Today, the value of providing information is about more than just delivering it; it's about delivering the right information to the right people the right way. And LLMs fail at the former.

The bottom line is that AI is not yet capable of what a good teacher or mentor can do: giving actually good, uniquely applicable, empathizing advice. It's much better at explaining things.

PS.: This article was peer-reviewed and approved by ChatGPT. I ignored its suggestion to add examples where it gave helpful advice because that's against my agenda statistically, with enough advice given, you will just randomly run into occasions where it gave good advice.

PPS.: A big discourse was recently sparked by Pieter Levels, who built a mental therapist Telegram bot with AI. This article has been sitting in my drafts for almost a year now and has absolutely nothing to do with that (I feel that discussion is more ethic-, accountability-, and risk-based anyway, as opposed to this article's core message). The timing of this article's publication just after his tweet is coincidental.

PPPS.: If you enjoyed this article, please consider heading to my site and subscribing to my newsletter 🧜.

Twitter lists for developers

Maxim — Thu, 30 Jul 2020 21:44:12 +0000

I've recently discovered the use of Twitter lists and decided to create some useful ones for devs.

If you think someone is missing in a list (or you have an idea for another cool list), please comment :)

DevNews - A list with mostly official technology accounts. You'd be surprised, how often you can find real gems in their timelines!

Bonus point: There may occur occasional job offers from these accounts even before accounts from the DevJobs list retweet them

DevJobs - A list with Twitter accounts which tweet jobs all day long. You'll find something for freelancing and company positions here.

DevConfs - All the major development conference accounts in one place.

AI/ML/NLP - No surprises here. If you're interested in artificial intelligence, machine learning, or natural language processing, you will love this list.

React.js - Thought leaders and people deeply involved in React.js development itself.

Wrap providers elegantly using withProvider HoC

Maxim — Sun, 12 Jul 2020 00:00:00 +0000

If you use React contexts, you probably have created a wrapper component before, which includes just the context Provider. You do that because you need the context data inside some component, but for it to be available, the Provider can't be rendered in the same component. It needs to be rendered somewhere above.

Here is a pattern that I use from time to time. It allows me to work with React contexts in a very compressed manner (I mean in the same file). Let's look at an example:

import React, { useContext, useState } from "react";

const CounterContext = React.createContext();

const CounterProvider = ({ children }) => {
  const [count, setCount] = useState(0);

  return (
    <CounterContext.Provider
      value={{
        count,
        increment: () => setCount((prevCount) => prevCount + 1),
      }}
    >
      {children}
    </CounterContext.Provider>
  );
};

const App = () => {
  const { increment, count } = useContext(CounterContext);

  return (
    <div>
      <h2>{count}</h2>
      <button onClick={increment}>Increment</button>
    </div>
  );
};

// 👇 without this 🤡, useContext in App will return undefined.
const WrappedApp = () => (
  <CounterProvider>
    <App />
  </CounterProvider>
);

export default WrappedApp;

If we create a simple higher-order component, we can use that instead of arbitrarily creating provider wrappers like WrappedApp.

const withBasicProvider = (Provider) => (Component) => (props) => (
  <Provider>
    <Component {...props} />
  </Provider>
);

// ...

export default withBasicProvider(CounterProvider)(WrappedApp);

This is a very naive implementation and will simply wrap a component with the passed argument, so you can omit creating a provider wrapper component manually.

But what if we have more than one context, which we want to use? We can reduce any number of given providers to incrementally wrap the passed component.

I'm using reduceRight just, to preserve the order of passed providers (e.g. the CounterProvider wraps DarkModeProvider wraps App):

export const withBasicProviders = (...providers) => (WrappedComponent) => (props) =>
  providers.reduceRight((acc, Provider) => {
    return <Provider>{acc}</Provider>;
  }, <WrappedComponent {...props} />);

// somewhere here `DarkModeProvider` has been added, too.
// It really doesn't matter how it looks in detail. This
// HoC is just about handling providers. In the end there
// is a CodeSandbox link for you to play around with
// actually working code.

// ...

export default withBasicProviders(CounterProvider, DarkModeProvider)(App);

Now, to make this HoC truly useful, let's see how to pass props to their respective providers. I am sure you can implement that functionality in various ways, but I feel the following is quite neat.

If there are any props that you want a provider to receive, you can simply pass an array. The HoC will check if the current provider item is an array. If it is, withProviders will use the object, that the second item is supposed to be, as props. If it isn't, it behaves just like withBasicProviders before - it just wraps the previous element.

export const withProviders = (...providers) => (WrappedComponent) => (props) =>
  providers.reduceRight((acc, prov) => {
    let Provider = prov;
    if (Array.isArray(prov)) {
      Provider = prov[0];
      return <Provider {...prov[1]}>{acc}</Provider>;
    }

    return <Provider>{acc}</Provider>;
  }, <WrappedComponent {...props} />);

// ...

export default withProviders([CounterProvider, { start: 5 }], DarkModeProvider)(App);

If you want to play around with some working code, here's a CodeSandbox demo.

(This is an article posted to my blog at maximzubarev.com. You can read it online by clicking here.)

How to create a Gatsby.js transformer plugin

Maxim — Wed, 26 Feb 2020 17:32:29 +0000

If you are missing a feature in Gatsby.js, this is the starting point to integrate it.

This article is a guide on how to create a Gatsby.js transformer plugin, even more specifically a plugin for a plugin (mainly gatsby-transformer-remark, but it works with gatsby-plugin-mdx, too). You can inspect the code of the finished plugin from this tutorial here.

What can you do with a Gatsby.js plugin?

Before jumping into the code, let's investigate what you can do with a Gatsby.js plugin. For this, I want to remind you what Gatsby.js is: A static website builder, similar to Jekyll or Hugo.

The magic happens when it fetches your desired content dynamically and integrates it into static files based on your configuration. You can set up the layout to your liking and wrapper components with React upfront and then plug in the content.

As simple as that. But then also, plugins can do things to your content. For example, if your content happens to be in Markdown, you want to transform it to HTML to output it on your website.

That's pretty much the whole process (with more details and mechanisms put in place along the process, but this is already a very light yet accurate description).

So there are mostly two main tasks Gatsby does for you, and so you can hook into each of those:

Fetching content from somewhere in a specific format.
Alter the content.

That gives you the possibility to create two types of plugins. Their concept and API are distinct, so practical experience from writing the one type of plugin does not strictly transfer to the other.

1. Source plugins

A source plugin has only one job: To connect your Gatsby.js build process to data, which usually is external, but can sometimes be internal, too.

Compared to the second type of plugins, source plugins are quite complex code-wise, or at least much more comprehensive by default.

Here is the official guide on creating source plugins.

2. Transformer plugins

A transformer plugin takes an input (usually some content data) and transforms it wholly or in parts.

Here is the official guide on creating transformer plugins.

Plugin example: `gatsby-remark-color-highlight`

I have created a plugin that searches your remark content for color hex codes and wraps them into an element to make the color visually appear.

// Takes this
<p>Lorem ipsum #abcdef and foo bar.</p>

// and transforms into this
<p>Lorem ipsum <span style="background-color: #abcdef">#abcdef<span> and foo bar.</p>

In this case, it's a plugin for another transformer plugin (gatsby-transformer-remark). That means, that after gatsby-transformer-remark has hooked into the delivered content and transformed a Markdown string into an AST (Abstract Syntax Tree), we are going to write a middleware basically, that takes this AST and further transforms it as we need it.

This fact again makes the plugin and tutorial not exactly transferrable to source or transformer plugins (although it's closer to the latter). The docs page on creating a Remark plugin is immensely helpful to our operation.

Let's start already:

$ yarn init # and fill out the questionnaire

Because we want to develop the plugin live with a Gatsby site, let's connect both. Change into the directory of your Gatsby project and adjust your gatsby-config.js. You can set the path to your plugin locally for now.

module.exports = {
  plugins: [
    {
      resolve: require.resolve(`../local/path/to/gatsby-remark-color-highlight`),
    },
  ],
};

Add some dev packages to make development pleasant and a few more utility libraries to juggle the data and colors.

$ yarn add --dev babel-preset-gatsby-package @babel/core cross-env
$ yarn add unist-util-visit color

The following is the signature of the function that we are going to export from the src/index.js.

module.exports = (
  { markdownAST }, // AST tree
  pluginOptions,
) => {
  // further code goes here

  return markdownAST;
});

unist-util-visit is a function, that visits each node in the AST tree (which is what we are working with here) with a custom callback. In the callback, you can mutate data to your liking. The end goal of the plugin is to return the mutated markdownAST.

On a related note, here is a list of more, possibly useful AST tree utility packages.

Before going into the visit callback, we can test each node and only visit the nodes that meet our conditions. For that, you can pass an unist-util-is compatible test function to visit. We can use a regular expression to check for the existence of color hex codes in the node value.

For this tutorial, we are going to check on text nodes only, because any node is going to boil down to its text content in any event. Take `foobar`, which is an inline code tag in Markdown. In the AST, this results in a node with typeinlineTagand a single child in thenode.childrenproperty, which is atext node on its own.

The only exceptions to this rule are HTML nodes. But those are a whole different story, and we are not going to include them, because you can't use a regex to check (think <i style="color: #ffffff">text</i>, which has no readable color code when rendered in the browser).

const visit = require("unist-util-visit");

module.exports = ({ markdownAST }, pluginOptions) => {
  // thanks to https://stackoverflow.com/a/1636354/744230
  const hexCodeRegex = /#(?:[0-9a-fA-F]{3}){1,2}/gim;

  const test = (node, n) => node.type === "text" && hexCodeRegex.test(node.value);

  visit(markdownAST, test, (node, index, parent) => {
    // `node` can only be only a node with a color hex code inside
  }

  return markdownAST;
}

Before we start mutating the AST, let's add a few helper functions to build a new color AST node and generate the HTML for its value. Also, let's create plugin options so that the user can affect the output to some degree:

const visit = require("unist-util-visit");
const Color = require("color");

module.exports = ({ markdownAST }, {
  wrapperElement = "code",
  className = "color-highlight",
}) => {
  // thanks to https://stackoverflow.com/a/1636354/744230
  const hexCodeRegex = /#(?:[0-9a-fA-F]{3}){1,2}/gim;

  const test = (node, n) => node.type === "text" && hexCodeRegex.test(node.value);

  const buildNodeHtml = color =>
    `<${wrapperElement} class="${className}" style="background-color: ${color}; color: ${
      Color(color).isLight() ? "#000" : "#fff"
    }">${color}</${wrapperElement}>`;

  const buildColorNode = color => ({
    type: "html",
    children: [],
    value: buildNodeHtml(color),
  });

  visit(markdownAST, test, (node, index, parent) => {
    // `node` can only be only a node with a color hex code inside
  }

  return markdownAST;
}

In the next step, we are going to deal with the actual replacement of hex code colors. This part is mostly autonomous, vanilla JavaScript. The only thing to be aware of is the data structure of AST nodes, as we split up nodes at the hex color, create a new node for whatever came before the color code, add a newly constructed color node and finish up with whatever came after the hex color.

const visit = require("unist-util-visit");
const Color = require("color");

module.exports = ({ markdownAST }, {
  wrapperElement = "code",
  className = "color-highlight",
}) => {
  // Thanks to https://stackoverflow.com/a/1636354/744230
  const hexCodeRegex = /#(?:[0-9a-fA-F]{3}){1,2}/gim;

  const test = (node, n) => node.type === "text" && hexCodeRegex.test(node.value);

  // helper function, returns an HTML string with the color code wrapped in a tag
  // with background color
  const buildNodeHtml = color =>
    `<${wrapperElement} class="${className}" style="background-color: ${color}; color: ${
      Color(color).isLight() ? "#000" : "#fff"
    }">${color}</${wrapperElement}>`;

  // helper function, creates an object with a structure that resembles an AST node
  const buildColorNode = color => ({
    type: "html",
    children: [],
    value: buildNodeHtml(color),
  });

  // `node` can be only a node with a color hex code in the value because of `test`
  visit(markdownAST, test, (node, index, parent) => {
    // Split the node value at any occurrence of a color hex code. Each element of `parts`
    // is a substring of the original node value. No element can include color codes,
    // because those are the separator
    parts = text.split(hexCodeRegex);

    // Get all the occurrences of hex codes. Note, that the length of this array
    // is always `parts.length - 1` (unless the node value doesn't have a
    // hex color anywhere, which we omit with `test`)
    matches = text.match(hexCodeRegex);

   // This is an array with all the nodes, that, joined together, results in the
    // desired output. Essentially, we want to fill it alternating with
    // non-hex-code-content nodes and our newly created hex-code nodes alternating
    const replacementSiblings = [];

    // Loop through all the parts.
    for (let i = 0; i < parts.length; i++) {
      // Clone the node, because we want to use the same for the split content nodes
      // around the hex codes
      const nodeCopy = JSON.parse(JSON.stringify(node));

      // Assign the first node of the split
      nodeCopy.value = parts[i];

      // Clean up properties not required properties, which are not transferrable from
      // the original node, therefore polluting the object
      delete nodeCopy.position;
      delete nodeCopy.indent;

      // This statement is purely optional. The plugin would work just as well without.
      // It's just to keep the AST clean, e.g. if we split the string
      // "hex code at the end #123456" with `hexCodeRegex` as a separator, it results
      // in `["hex code at the end ", ""]`. But why fill the AST with useless nodes, that
      // do nothing?
      if (nodeCopy.value !== "") {
        replacementSiblings.push(nodeCopy);
      }

      // Make sure matches[i] exists. You could check with `if (matches[i])` just as well.
      // I am just utilizing the knowledge of `matches.length === parts.length - 1`
      if (i < matches.length) {
        replacementSiblings.push(buildColorNode(matches[i]));
      }
    }

    // `splice` removes the node at the current index (which is the node we are
    // visiting), and replace it with all the items we have assembled in
    // `replacementSiblings`
    parent.children.splice(index, 1, ...replacementSiblings);

    // This is a crucial line. The callback can override the internal index of `visit`,
    // thereby overriding what node it is going to visit next. Because we just
    // replaced 1 node with 1 + x (where x === replacementSiblings.length) nodes, if
    // we don't skip those newly injected sibling nodes, `visit` is going to run into an
    // infinite loop by visiting the sibling node that you just created, create new
    // siblings again, visit those, and so on.
    return index + replacementSiblings.length - 1;
  }

  // We return the mutated `markdownAST` here.
  return markdownAST;
}

And that's it. You've successfully written a plugin for Gatsby.js. You can inspect and study the actual code, which also considers HTML nodes, on GitHub.

Further tips

Helpful resources

I've found it incredibly helpful to study the source code of other plugins to learn and understand how to write my Gatsby.js plugin. It might not tell you explicitly what to do, and blind copy-pasting likely won't work, but it's the fastest and most effective of learning.

It also requires the ability to read and digest uncommented code by yourself, which can be easy or hard depending on how complex the code is.

Debugging and logging

I noticed it is quite tedious to debug your plugin because the build process takes up some time (about 20 seconds for each run in my case). And you are required to restart Gatsby.js each time you want to see the output because the building process runs only once upfront.

On that note, I noticed that for some reason, Gatsby cache silently swallows my console.logs on the second and subsequent runs, so I have to prepend each run with gatsby clean

$ gatsby clean && gatsby develop
# instead of just
$ gatsby develop

Why order of plugins matters (sometimes)

This tip is usually only important for transformer plugins, including the one we've created here.

Because such plugins change the content (or in this case the node tree of your content) and run in the same order as defined in the gatsby-config.js, one plugin may not receive the same markdownAST as another. For example, if you would like to transform hex codes in code snippets produced by gatsby-remark-vscode, you need to put gatsby-remark-color-highlight after.

That is because you don't want to suggest to gatsby-remark-vscode that your added wrapper element is user content and, therefore, should be escaped in the snippet. Instead, you want to wrap the content in the HTML produced by gatsby-remark-vscode.

On other occasions, plugins, which are a prerequisite for another plugin, need to be put to the front.

Conclusion

When you want to learn more about plugin development for Gatsby.js, the best way is to have a look at their plugin authoring guide and related docs, as well as on the source code of existing plugins, like:

gatsby-remark-autolink-headers (transformer plugin)
gatsby-source-wordpress (source plugin)

Thanks for reading. This tutorial has been originally posted on my blog. You can sign up for my newsletter for more in-depth articles about frontend technologies.