DEV Community: Vrite

I Published This with Drag and Drop using Vrite

Arek Nawo — Tue, 12 Mar 2024 17:33:07 +0000

For many developers, blogging and technical writing play a key role in building their portfolio, sharing their projects, and for some — even in their day-to-day work. That’s why developer-centric platforms like DEV and Hashnode, and even more general ones, like Medium, are full of interesting technical content. The only problem with technical writing is in the actual writing and publishing process.

For blogs, the editing experience varies from platform to platform, and things like cross-posting require a lot of effort (and copy-pasting) to adjust your content to each platform’s unique quirks.

On the other hand, more complex technical writing (when working on e.g. product documentation, or knowledge bases) often involves custom content that’s unique to your publication and isn’t supported in any kind of writing tool or CMS.

You’re left with Markdown which, while being pretty good for basic needs, is very limited, and usually requires various extensions and supersets to fit your needs (MDX, Markdoc, etc.). Also, MD might not be great for everyone (especially when writing longer pieces) and doesn’t provide the same experience as a modern WYSIWYG editor like Notion.

These reasons (and many others) are why I decided to create Vrite - an open-source developer content platform.

Vrite (v0.4)

With tons of built-in features, including:

Content management dashboard (with Kanban and Table views);
Embedded Monaco Editor with Prettier formatter;
Hybrid WYSIWYG editor with real-time collaboration;
Vector search (with AI question-answering);
Bi-directional Git Sync;

Vrite is meant to be the most versatile and extensible CMS-like platform for all your technical content needs.

Extension System

With the latest v0.4.x, among many other features and improvements, the focus is on a new extension system.

An extension system is a key feature to make sure Vrite is able to handle pretty much any technical writing use case. From improving publishing (and cross-posting) to extending the capabilities of the WYSIWYG editor, a powerful extension system is the future of Vrite.

The goal is to make creating extensions not much more difficult than creating e.g. a React component, while making sure the extensions are sandboxed and secure.

To give you a sample of the extension API, here’s a snippet from a “first-party” Dev.to extension:

/** @jsx createElement */
/** @jsxFrag createFragment */
import {
  Components,
  createView,
  createTemp,
  createFunction,
  createRuntime,
  ExtensionConfigurationViewContext,
  ExtensionContentPieceViewContext,
  createElement,
  createFragment,
} from '@vrite/sdk/extensions';

type Config = {
  // ...
};
type ContentPieceData = {
  // ...
};
export default createRuntime<Config>({
  onUninstall: createFunction(async ({ client }) => {
    // ...
  }),
  onConfigure: createFunction(async ({ client, config, spec }) => {
    // ...
  }),
  configurationView: createView<ExtensionConfigurationViewContext<Config>>(
    ({ use }) => {
      const [apiKey] = use('config.apiKey');
      const [organizationId] = use('config.organizationId');
      const [autoPublish, setAutoPublish] = use('config.autoPublish');
      const [contentGroupId, setContentGroupId] = use('config.contentGroupId');
      const [requireCanonicalLink] = use('config.requireCanonicalLink');
      const [draft] = use('config.draft');

      // ...

      return (
        <>
          <Components.Field
            type="text"
            color="contrast"
            label="API key"
            placeholder="API key"
            bind:value={apiKey}
          >
            Your Dev.to API key. You can generate one in the [settings
            page](https://dev.to/settings/extensions), under **DEV Community API
            Keys** section
          </Components.Field>
          {/* ... */}
        </>
      );
    },
  ),
  contentPieceView: createView<
    ExtensionContentPieceViewContext<Config, ContentPieceData>
  >(({ config, token, extensionId, notify, use, flush }) => {
    // ...
  }),
});

Speaking of which… this entire post was written in Vrite and published with a simple drag and drop using the Dev.to extension!

If you’re interested in trying it out for yourself, check out Vrite Cloud and the official docs (if you’re wondering, yes — they were also written in Vrite, and published via Git Sync).

Using The Dev.to Extension

Start by signing into Vrite.

Then go to the Extensions side panel.

Installation

From the Extensions side panel, in the Available section, find Dev.to and click Install.

Configuration

The Dev.to extension has a few options you have to configure to fully activate it.

API Key

To retrieve your Dev.to API key, go to the bottom of the Extensions section in the DEV Settings: https://dev.to/settings/extensions

Here, add a description and Generate API Key. Once the new key is available, copy and paste it into the extension’s configuration.

Organization ID (optional)

If you want to publish the posts as an organization, you have to provide your Organization ID.

You can find it by going to your DEV dashboard (https://dev.to/dashboard) then switching from Personal to your organization view using the selection menu above the posts list and copying the organization ID from the URL:

https://dev.to/dashboard/organization/[ID]

Content Group ID

The content group you want to automatically publish from. Once a content piece is moved directly to this content group, it’ll trigger a Webhook that’ll auto-publish the content piece on Dev.to.

You can copy the content group ID from the content group menu in either the dashboard or the explorer panel.

Require Canonical Link

When checked, the auto-publishing won’t trigger if the content piece doesn’t have a canonical link assigned. Useful when cross-posting the content to e.g. Dev.to and your own blog.

Auto-publish

Whether to enable auto-publishing for all content pieces by default.

Draft

Whether the article should be marked as a draft on the Dev.to platform (not publically visible).

Usage

With auto-publish enabled, once ready, simply move (drag and drop) the content piece from e.g. Drafts content group to the one configured e.g. Published. You should see the article published on Dev.to shortly after.

The extension also provides a content piece view (available from the Extensions section of the Content piece side panel.

From here, you can customize the Auto-publish and Draft per the given content piece, and also set a Series name to associate the article with, once it’s published on DEV.

Finally, with or without the auto-publishing enabled, you can easily publish “manually” using the Publish button.

The Future of Vrite

Currently, the extension system is only available on the Vrite Cloud, which is a paid product with a free trial available.

The extensions will be opened for self-hosting once the API is stabilized and all (or most) of the necessary features are implemented (i.e. permission control, an official extension store, versioning, and development tools).

Apart from the extension system, with features like sharing, and built-in front-ends in the pipeline, the future of Vrite looks exciting!

WYSIWYG for MDX?! Introducing Vrite's Hybrid Editor

Arek Nawo — Wed, 18 Oct 2023 16:01:57 +0000

You might be familiar with so-called What You See Is What You Get (WYSIWYG) or Rich Text editors. They’re the core part of popular apps such as Notion, allowing users to both see and edit the formatting and content of the document.

WYSIWYG editors are great for creating all kinds of content but, in some cases, they might be limiting, allowing you to use only a small set of formatting options and elements supported by the editor.

That’s especially problematic in niche content categories, like technical/developer content, where content requirements might differ, between companies, teams, and even individual projects.

That’s why formats like Markdown (MD) and MDX (MD with support for JSX) are so popular for use cases like documentation, knowledge bases, or technical blogs. They allow you to use any kind of custom formatting or elements and then process the content for publishing. On top of that, they’re great for implementing a docs-as-code approach, where your documentation lives right beside your code (i.e. in a Git repo).

However, writing Markdown files is a very different experience compared to the WYSIWYG approach. It can be challenging for beginners or less technical users, while also getting increasingly difficult to manage as the content base grows. My latest project — Vrite — is meant to solve this problem.

Vrite Hybrid Editor

Vrite is an open-source developer content platform, featuring extensible editing experience, content management tools, and powerful APIs. It’s intended as an all-in-one, collaborative solution for product documentation, technical blogs, and knowledge bases.

With a collaborative WYSIWYG editor being a core part of the experience, I was thinking for a long time about how to create an experience that still has the power and customizability of MDX when needed. That’s how I came up with Vrite’s new Hybrid Editor.

To be fair, rather than being a whole new editor, in the latest v0.3, the Vrite editor was slightly redesigned and got a new block content type — Element — which enables the use of JSX-like blocks right in the editor, leading to a part-markup, part-WYSIWYG editor.

The Element block can have a type (e.g. Card) and a set of props (like title or options). You can edit it by clicking the opening tag. When the block is active and editable, its syntax will be highlighted.

Elements can be either wrappers (containing content) or simple block nodes. You can switch between those two “modes”, similar to how it’s done in JSX/MDX — by changing the closing bracket of the first tag. /> is for self-closing, while > — for wrapping.

Behind the scenes, Vrite processes the content and makes it accessible in ProseMirror-based JSON format, including the type and all the props of the Element block.

This format makes it easy to further process the content — whether that’s for use with other features of Vrite or when accessing it through the API for publishing.

MDX and GitHub Sync

With the Element block enabling the use of more custom content, it also empowers other features of Vrite — such as Git sync — with new possibilities.

Git Sync

When your existing documentation or technical content is stored in MD files in your Git repo, or you want to implement a docs-as-code approach, with your repo being the single source of truth — Git sync allows you to sync the content from your repo with Vrite.

This way you can still use all of Vrite’s features (e.g. content management, publishing extensions, semantic search, etc.), while also being able to commit your changes to the repo, resolve conflicts, etc. Basically, Git sync turns Vrite into a technical content editor, with additional features on top.

Currently (v0.3.2), Git sync only integrates with GitHub, with other providers planned for later on.

Syncing MDX

Git sync works on top of Input/Output Content Transformers — functions that can “transform” the content to and from the Vrite’s JSON format and whatever’s in your repo. This is quite versatile, and — in theory — should be able to support all kinds of formats, decoupling the format you use from the tooling and features you have available.

Vrite comes with a GitHub-Flavored Markdown (GFM) transformer built-in. It supports basic Markdown formatting, with the addition of GFM extensions, like task lists or tables.

To extend that, Vrite supports “remote” transformers — essentially processing the content on the backend, communicating via batched POST requests.

That’s how the new MDX extension works — it registers a new transformer, that processes your content through endpoints, such as POST https://extensions.vrite.io/mdx/input.

So, to use Git sync with your MDX content:

From the Extensions side panel → Install the MDX extension;
From the Source control side panel → Configure GitHub as a provider (selecting your repo and branch, path to sync, and MDX as a transformer; Check out this blog post to learn more on this configuration;
After the Initial sync, check out your content and verify if it was synced correctly.

Limitations

The MDX transformer can handle much more than the built-in Markdown transformer:

It supports YAML frontmatter, with properties like title, description or slug being synced with Vrite’s metadata.
It supports the Element block, converting it to MDX, as expected.
It uses a special <Import> Element to contain the import statements from the MDX file (for better back-and-forth syncing).

That said, with both the Element block and MDX extension being new features in a project that’s still in Beta, there are also some limitations you have to consider:

For the Element block, you can only use values that are JSON-parseable. So, e.g. passing JSX element as a prop won’t work for now.
Element is a block and there’s no support for custom inline formatting as of yet. Thus, if you use inline JSX components in your MDX (e.g <Icon/>, <InlineBadge>...</InlineBadge>, etc.) — they’ll be omitted.
If you’re extending MDX for custom use cases, the Git sync isn’t guaranteed to work. The MDX transformers support only more popular customizations (such as title or metadata for codeblocks);

The end goal with MDX support is to extend the editor to be fully customizable (both inline and block-wise) and provide more documentation and details for users to create custom transformers, to support all kinds of use cases.

Bottom Line

I fully intend to make Vrite a viable, go-to solution for all developer content needs — one that’s both open-source and extensible, while still providing an integrated, all-in-one experience.

The Element block is an important milestone towards this goal. With custom inline content and an entire extension system — for extending the editor with WYSIWYG components built on top of custom elements — coming soon, there’s still a lot more in store.

If you’re interested in the project, consider leaving a star on GitHub 🌟.

🤖 AI Search and Q&A for Your Dev.to Content with Vrite

Arek Nawo — Thu, 24 Aug 2023 16:45:58 +0000

No one can deny that ChatGPT brought Large Language Models (LLMs) into the public spotlight. While LLMs are not perfect, when you think about it, the ability to ask a wide variety of questions and get an answer in seconds is mind-blowing. 🤯

The only thing left is to somehow connect it with your own data and provide a new context for the LLM to source answers from. This is where text embeddings, vector databases, and semantic search come in. However, depending on your use case, implementing the entire stack necessary to power an “AI search” or a Question-and-Answer (Q&A) type of interface might be quite a challenge… but it doesn’t have to be.

With the latest update, Vrite — an open-source technical content management platform I’m working on — now has a built-in search and a Q&A feature to find answers to all the questions related to your content. Both inside Vrite — via a new command palette — and outside — via an API. ✨

This can be used to easily build a new kind of search experience for your blog or to provide answers to user’s questions in your product docs.

To give you a fun example of that, we’ll go through a process of importing content from the Dev.to API to Vrite to search through it, and then see how you can easily implement a semantic search on your own site using Vrite APIs.

Setting-up

Let’s start by getting into Vrite. You can use the hosted version (free while Vrite is in Beta) or self-host Vrite from the source code (with better self-hosting support coming soon)

To import your Dev.to content collection to Vrite, it’d be best to do so in a dedicated workspace. In Vrite, you can use workspaces to separate different projects or teams. To create a new one, from the sidebar go to the Workspace section.

From here, you can both create and switch between different Workspaces. Create one for your Dev.to blog and switch to it.

With the dedicated workspace ready, you’ll have to create a new API token — both in Vrite and Dev.to — to use in the import script.

To get one in Vrite, go to the Settings side panel → API section → click New API token. From here you’ll have to configure the details and permissions for the new token. Make sure to select Write permission for both Content pieces and Content groups, as these will be necessary to import the content.

Once you create it, store the token in a safe place - you won’t be able to see it again.

To get an API key from Dev.to, go to the Settings → Extensions → DEV Community API Keys section → provide a description and click Generate API Key.

You will be able to see your API Key at any time, though you should still keep it secure.

Importing Content From Dev.to

With API tokens ready, it’s time to prepare an import script.

With Node.js (v18 or newer) and NPM installed, initialize a new project, install Vrite SDK, and create the primary import.mjs file.

npm init -y
npm install @vrite/sdk
touch import.mjs

Inside import.mjs, let’s first create a function to fetch your articles from Dev.to.

const VRITE_API_TOKEN = "...";
const DEV_API_KEY = "...";

const getDevArticles = async (perPage = 1000) => {
  const response = await fetch(
    `https://dev.to/api/articles/me/published?per_page=${perPage}`,
    {
      headers: {
        accept: "application/vnd.forem.api-v1+json",
        "api-key": DEV_API_KEY,
      },
    }
  );
  const data = await response.json();
  return data.reverse();
};

From v18, Node.js provides a fetch() API, similar to web browsers, which makes handling network requests much easier. Use it with the proper URL and headers to make a request to the User's published articles endpoint.

The Dev.to API implements pagination, with the max value being 1000, so a single request should be enough to retrieve all the articles for most (if not all) users.

To actually import the content to Vrite, let’s create a separate function.

import { createClient } from "@vrite/sdk";
import { gfmInputTransformer } from "@vrite/sdk/transformers";

// ...

const importToVrite = async (numberOfArticles) => {
  const articles = await getDevArticles(numberOfArticles);
  const client = createClient({
    token: VRITE_API_TOKEN,
  });

  const { id: contentGroupId } = await client.contentGroups.create({
    name: "My Dev.to Articles",
  });

  for await (const article of articles) {
    const {
      title,
      body_markdown,
      published_at,
      cover_image,
      canonical_url,
      url,
    } = article;
    const { content } = gfmInputTransformer(body_markdown);
    try {
      await client.contentPieces.create({
        contentGroupId,
        title,
        cover_image,
        canonicalLink: canonical_url || url,
        content,
        coverUrl: cover_image,
        date: published_at,
        members: [],
        tags: [],
      });
      console.log(`Imported article: "${title}"`);
    } catch (error) {
      console.error(`Could not import article: "${title}"`, error);
    }
  }
};

The .mjs extension in newer versions of Node.js allows out-of-the-box use of ESM import syntax, which we use to import Vrite SDK and gfmInputTransformer.

Vrite SDK provides a few built-in input and output transformers. These are functions, with standardized signatures to process the content from and into Vrite. In this case, gfmInputTransformer is essentially a GitHub Flavored Markdown parser, using Marked.js under the hood.

In the importToVrite() function, we first retrieve articles from DEV using the mechanism discussed before and initialize the Vrite API client. From there, we create a new content group for housing the content and loop over the imported articles using for await to create new content pieces from them.

The created pieces include the transformed content and some additional metadata sourced from Dev.to, to easily identify individual pieces.

With that, all you have to do is call the importToVrite() function with the number of your latest Dev.to articles to import and watch it go! Here’s the entire script:

import { createClient } from "@vrite/sdk";
import { gfmInputTransformer } from "@vrite/sdk/transformers";

const VRITE_API_TOKEN = "...";
const DEV_API_KEY = "...";

const getDevArticles = async (perPage = 1000) => {
  const response = await fetch(
    `https://dev.to/api/articles/me/published?per_page=${perPage}`,
    {
      headers: {
        accept: "application/vnd.forem.api-v1+json",
        "api-key": DEV_API_KEY,
      },
    }
  );
  const data = await response.json();

  return data.reverse();
};
const importToVrite = async (numberOfArticles) => {
  const articles = await getDevArticles(numberOfArticles);
  const client = createClient({
    token: VRITE_API_TOKEN,
  });

  const { id: contentGroupId } = await client.contentGroups.create({
    name: "My Dev.to Articles",
  });

  for await (const article of articles) {
    const {
      title,
      body_markdown,
      published_at,
      cover_image,
      canonical_url,
      url,
    } = article;
    const { content } = gfmInputTransformer(body_markdown);
    try {
      await client.contentPieces.create({
        contentGroupId,
        title,
        cover_image,
        canonicalLink: canonical_url || url,
        content,
        coverUrl: cover_image,
        date: published_at,
        members: [],
        tags: [],
      });
      console.log(`Imported article: "${title}"`);
    } catch (error) {
      console.error(`Could not import article: "${title}"`, error);
    }
  }
};

importToVrite(20);

Search and Q&A in Vrite Dashboard

With the content now in Vrite, let’s go back to the dashboard and see how to use the command palette to search right in Vrite.

When coupled with Vrite support for collaboration, the built-in search and command palette can serve as a great tool when using Vrite as an internal knowledge base.

To open the palette use ⌘K (on macOS), Ctrl K (on Windows or Linux), or the search button in the dashboard’s toolbar.

The command palette has 3 modes:

Search — the default, provides results as you type;
Command — can be enabled by typing > in empty search or by clicking the Command button in the bottom-right corner; Allows quick access to various actions available in the current view; You can move back to the search mode by using Backspace in empty input;
Ask / Q&A — can be enabled by clicking the Ask button in the top-right corner; Type in your question and click Enter to request an answer;

Try searching for any term and see results from your various Dev.to blog posts appear.

Vrite indexes entire sections of your content pieces, identified by the title and a set of headings. This allows the LLM to extract the most semantic meaning from the content, which enables vector search to provide better search results for your queries. So, the better you structure your posts the better the search results will be.

You can also try the Q&A mode, asking any question that there should be an answer for in your content. Upon submission, the prompt is sent together with the context, for GPT-3.5 to generate an answer, which is streamed back to the command palette.

Personally, I was quite impressed with how well the Q&A turned out. Even answers that would require reading through several pieces were generated accurately in seconds. Still, you should keep in mind that this won’t always be the case.

Search and Q&A via Vrite API

Now, searching through Vrite’s command palette is nice, but the real fun begins when you get to implement this search and Q&A experience on your own blogs and docs via Vrite API.

First, you’ll have to “proxy” Vrite API searches via your own backend or serverless functions, due to CORS and security considerations (especially if your token has powerful permissions). To do so, you’ll have to access Vrite API from Node.js.

Search

First, make sure to have an API token with at least Read access to Content pieces. With that, you can use the search() method of the API client to retrieve the results.

import { createClient } from "@vrite/sdk";

const VRITE_API_TOKEN = "...";

const search = async (query) => {
  const client = createClient({
    token: VRITE_API_TOKEN,
  });

  const results = await client.search({ query });

  console.log(results);
};

search("Dev.to");

The search result is an array of objects, each containing:

contentPieceId — ID of the related content piece;
breadcrumb — an array of the title and headings leading to the section;
content — plain text content of the section;

You can process or send these results directly to the frontend as a JSON array.

Q&A

Q&A is a bit more difficult. Due to the slow response associated with the time GPT-3.5 needs to generate an answer, the /search/ask endpoint is implemented via Server Sent Events (SSEs) to stream the answer to the user, allowing them to see the first tokens as soon as they’re ready.

Vrite SDK doesn’t support SSE streaming just yet, so, for now, you’ll have to implement this yourself. Use the eventsource library or similar to connect with the endpoint and stream the answer.

const streamAnswer = async (query) => {
  try {
    const source = new EventSource(
      `https://api.vrite.io/search/ask?query=${encodeURIComponent(query)}`,
      {
        headers: { Authorization: `Bearer ${VRITE_API_TOKEN}` },
      }
    );

    let content = "";

    return new Promise((resolve, reject) => {
      source.addEventListener("error", (event) => {
        if (event.message) {
          return reject(event.message);
        } else {
          source.close();
          return resolve(content);
        }
      });
      source.addEventListener("message", (event) => {
        content += decodeURIComponent(event.data);
      });
    });
  } catch (error) {
    console.log(error);
  }
};

streamAnswer("What is Dev.to?").then((answer) => {
  console.log(answer);
});

The example loads the entire answer and then resolves the Promise. You can use this method directly, but the response time for each request will be counted in seconds.

To provide a better User Experience (UX), you’ll likely want to forward the events coming from Vrite API, through your backend to the frontend, where the user will see the first tokens of the answer appear much faster. The implementation of this will depend on your backend framework, but the general approach is to write a text/event-stream response as the data comes in. Here’s a good overview of the general process.

I’m working to document and support this process better in the coming weeks.

Bottom Line

While the AI search itself is really great, the best part about Vrite is that the search is only a small fraction of a greater whole. With the Kanban content management, WYSIWYG technical content editor, Git sync, and extensions for publishing to platforms like Dev.to with just drag and drop — we’ve only scratched the surface of what’s possible!

Now, Vrite is currently in Beta and there are still bugs to be resolved, and new features to be added and fleshed out. If you want to help and support the project, leave a star on GitHub and report any issues or bugs you encounter. With your support, I hope to make Vrite the go-to, open-source technical content platform 🔥

🔥✍️ Notion-like Experience for Your GitHub Content

Arek Nawo — Tue, 22 Aug 2023 17:54:25 +0000

A vast amount of technical content — technical blogs, product docs, and others — lives inside Git repos — and for a good reason. Aside from Git’s version control, platforms like GitHub provide tons of integrations and additional features, to easily host and automate various workflows. That’s especially important for efficient content delivery for blogs, or for docs to be kept in sync and right beside the documented codebase.

That’s why, while working on Vrite — an open-source technical content platform — I knew a good integration with Git is crucial for many workflows. It’s a must-have feature, that has to be implemented right for technical writers to benefit from it.

So, after weeks of research and development, I’m excited to share with you what I’ve come up with, and how you can use Vrite to edit your GitHub content.

The Various Types of Git Integrations

Naturally, the idea of integrating Git into a Notion-like, modern, collaborative, WYSIWYG editing environment is nothing new. However, not all Git integrations are created equal.

Some Content Management Systems (CMSs) similar in functionality to Vrite are Git-based, meaning they fundamentally rely on Git to work. While that implies great Git integration, it also comes with various limitations.

Vrite leans more towards headless, API-first CMS, storing its content in a database, and making it accessible via API for users to build custom frontends on top of. This means that Git becomes a separate layer that has to be integrated into existing tooling.

CMSs and tools like knowledge bases approach this in various ways. Most commonly I’ve seen a one-way integration, meaning either your Single Source of Truth (SSOT) is a Git repo and every change has to be immediately committed to it or your SSOT is the actual app and you can commit changes in batches, with little to no regard for content or updates that happen in the repo. There seem to also be other solutions, like periodic sync, where a forceful sync happens every once in a while.

To me, all these approaches seemed inferior to what developers have at their disposal in code editors and IDEs — including the ability to commit and pull at any time, resolve merge conflicts when necessary, switch between branches, stage certain changes, etc. I wanted to recreate at least a part of that in Vrite by integrating with various Git providers via their APIs — starting with GitHub.

Vrite Git Sync

So, how does Git sync currently work in Vrite? Let me take you through a quick overview.

Setting Up Sync with GitHub

You can use Vrite via the hosted version (that’s free while in Beta) or self-host it from the open-source repo (though good support for self-hosting is still in the works).

When inside the dashboard, open the Source control side panel.

From here, you’ll have to first select and configure a provider. For now, only GitHub is supported, with other major providers (GitLab, BitBucket, and Gitea) to come later.

To configure GitHub you’ll first have to select a repo and authorize access to it. To do so, first, authenticate with GitHub, and install the Vrite GitHub app. Only accounts and repos authorized in the GitHub App’s installation will be available for selection.

After you’ve selected the repo and branch to sync, you can optionally configure the mapping settings. These dictate how to map the repository’s content to Vrite.

Base directory — indicates which directory to sync with, so you can only sync e.g. /docs directory. It points to the root (/) directory by default.
Match pattern — a glob match pattern that, applied relative to the base directory, will only sync files with matched filenames. By default, it matches all .md files.

Once you configure GitHub provider, and Save the configuration, you’ll have to perform the initial sync. That’s when the existing content from GitHub will be synced into Vrite, creating content groups for all the folders, content pieces for synced files, and indexing it all. Depending on the amount of content you have, this can take a while.

Managing Content In Vrite

Once the sync is complete, you’ll see a new content group appear, with a name corresponding to your repo.

Now, a word on managing your Git content in Vrite.

In Vrite, your content is organized into content pieces — containing the actual content and related metadata, and content groups — grouping multiple content pieces together. In the Vrite dashboard’s primary view — Kanban — these are displayed as individual draggable columns and cards inside them.

Kanban is great for managing the content production process, but it doesn't necessarily represent the way your Git repo is structured — with nested folders and all. That’s why, to provide a good experience for the new workflow with Git, Vrite now supports nested content groups.

To navigate those, you can use the folder icon, next to the content group’s name. Click it to get inside the content group, and drag it onto the other folder icon to move one group into the other.

The same dragging and clicking mechanism works with the breadcrumb that will appear in the toolbar. Simply click a content group (or the workspace icon) to get back to the certain nesting level, or drag a folder icon from the Kanban onto one part of the breadcrumb to move it.

Why use the folder icon for that? I think it’s a more intuitive way to manage group nesting while dragging entire columns is still reserved for rearranging the kanban.

But that’s not all. If you prefer a more folder-like approach to managing your content, you can always switch to the new list view, similar to the one you’ve experienced in GitHub. The same rules apply here, with the additional option to also drag individual content pieces onto the groups’ folder icons to move them. This feature is to-be-added in the Kanban view.

One more thing to note is in regard to filename mapping. Vrite supports various metadata for content pieces, including title and slug, but these didn’t seem appropriate to use for filename mapping. What if you want to use the title for something meaningful, or the slug for publishing on your blog, docs, etc.? That’s why Vrite added a new metadata field called Filename, which is specifically meant to aid in use cases like Git integration, assigning a meta filename to your piece.

However, with the increased number of metadata fields, I’ve decided to keep some (including filename) disabled by default, to allow you to better focus on the metadata that matters to you. So, if you use Git, you’ll likely want to go to the Settings → Metadata section and enable the fields that matter to you.

Pull, Commit & Resolve Changes

Knowing how to manage content in Vrite, let’s go back to the Source control panel and see how to sync your edits with Git.

First — commits. Go to the editor and edit a piece or two. You should see the changed files reflected in the Changes section shortly after. In case you want to commit the changes, simply provide a Commit message and commit.

The commit should show up in the repo as verified and authored by vrite-io[bot]. I decided against attributing changes to individual users, as Vrite enables you to add team members and collaborate on the content in real time, making it difficult to differentiate who actually made the change.

Now, when you try to commit after a change has been done to the repo from outside your Vrite account, you’ll receive a “Pull required before committing changes” error notification. To resolve this, you’ll have to first pull the latest changes.

To do so, simply use the button in the Pull section. Vrite will fetch all commits made since the last pull (or the initial sync) and sync the content of files changed in those commits. It can be done entirely automatically, or there can be a conflict. In this case, you’ll have to resolve it first, before you can sync the changes.

All conflicted files will be listed in the Pull section. Click on one of them to start resolving the conflict.

You’ll see a Monaco Editor-powered change editor. The content incoming from the Git repo is on the left, while the current content in Vrite is on the right. You can make changes in the editor on the right - this will ultimately become the result content. Once you’re done, click Resolve. If there are no other conflicts, you should now be able to pull the latest changes.

Future Improvements

With that, you now know pretty much all there is to know about GitHub sync in Vrite. It’s quite simple and still not as feature-rich as you can find in code editors or IDEs, but it gets the job done, better than quite a few other CMS and knowledge-base tools.

That said, there’s definitely room to improve. I’d like Vrite to be the “go-to” editor/CMS for blogs and docs, and this necessitates some additional features. In the short term, I’m looking to add:

Custom remote transformers — to allow developers to provide custom functions for processing content in and out of Vrite;
Stage and undo — to better organize the commits from Vrite;
Custom blocks — to extend the content that can be synced with and edited in Vrite, to support more use cases, like complex, custom docs;

In the long-term:

Support for other providers;
Easy switching between branches;
Custom WYSIWYG editor for conflict resolution;
Access to your Git version history right from Vrite;

Bottom Line

Syncing the content from GitHub with Vrite allows you to have a great WYSIWYG editing experience for your technical content, while still having access to all the features of Git and GitHub. In addition to that, you get to benefit from all the features of Vrite, like easy content management, powerful API with Webhooks, and… an AI search, though that’s a story for another day.

In case you’re interested in Vrite, leave a 🌟 on GitHub, as well as any feature requests or issues you may have, to support the project. ❤️

Vrite Editor: Open-Source WYSIWYG Markdown Editor

Arek Nawo — Tue, 18 Jul 2023 17:58:10 +0000

I’ve created an open-source, minimalistic WYSIWYG Markdown editor to easily create and export Markdown and HTML content on the go, without any sign-up required!

Here’s “Why?”, “How?”, and other questions answered!

The “Why?”

In software development, Markdown is basically omnipresent. You can find it in READMEs, powering software blogs, and serving as one of the primary content formatting methods when Rich Text Editing (RTE) is required.

Markdown is awesome! But, when writing 1000 words+ articles, I quickly feel the need for a better experience. For years, I’ve used StackEdit — an open-source, in-browser Markdown editor — for editing all kinds of long-format Markdown text. That said, given my recent experience with WYSIWYG editors, I thought I could do something better.

That’s how I created Vrite Editor — an open-source Rich Text Editor, focused on good user experience and technical writing.

The Stack

No good tool is built without using good tools, and Vrite Editor is no different. Before getting into WYSIWYG editors, I extensively researched available RTE frameworks, that could provide the tooling and functionality I was looking for. Ultimately, I picked TipTap and underlying ProseMirror — IMO, the best tools currently available for all kinds of WYSIWYG editors.

Aside from that, I used:

Solid.js — the super-fast UI framework;
UnoCSS — for styling with Tailwind-like atomic CSS;
Vite — build tool for putting it all together;

The results came out pretty nicely! 🌟

Features & Behind the Scenes

Vrite Editor comes with many features to make editing a breeze! You can check them all out in the official documentation. That said, there are a few unique ones that I feel are worth diving deeper into.

Code Editor

Something that I always felt was lacking in WYSIWYG editors (even the top ones, like Notion) is good support for code editing. Granted, technical writing is a niche use-case and you can get by with simple syntax highlighting and some copy-pasting. That said, I wanted to try something else — I wanted to integrate a full-blown code editor.

By referencing the ProseMirror docs, forwarding the editor state back and forth, and adjusting the layout, I managed to integrate Monaco Editor — the web editor extracted from VS Code — together with Prettier (for code formatting) right into the Vrite Editor (I know, that’s a lot of editors in one place 😅).

Granted, this is not VS Code in your content editor, but this way you get great syntax highlighting for many languages, while also having formatting and auto-completion for at least a few — like JavaScript or HTML.

Table Editor

The “whole grail of WYSIWYG editing” (as stated in the TipTap docs) took a while to get right. While TipTap provides extensions for supporting Table formatting, the UI is a different story. Table formatting UIs are notoriously difficult to get right. That’s why I was quite happy when I finally felt like I built something that was… alright.

When your cursor is inside the table, you’ll see a bottom menu appear, which you can use e.g. insert and remove rows, columns, and headers. On the other hand, when you select some cells, you’ll see a menu to e.g. merge, split, or delete them. It’s not Excel-level but, for an inline table, I think it’s pretty good.

Markdown Pasting

Handling Markdown pasting was a surprising challenge. Naturally, you’d expect an editor that works with Markdown to be able to parse and properly load the Markdown you pasted, so some work had to go into that.

TipTap provides input rules to handle so-called “Markdown shortcuts” while typing, and paste rules for offering the same parsing functionality while pasting. The problem is, paste rules work line-by-line, which means parsing something like a blockquote, codeblock, or a list, just isn’t possible.

To handle pasting block Markdown content like this, I had to tap into ProseMirror and implement a custom mechanism (though somewhat based on TipTap’s paste rules), detecting starting and ending points of the blocks and parsing them with Marked.js.

Thankfully, with this done, most Markdown can be pasted freely into the editor and should be parsed just fine!

Mobile Support

The other thing that's quite hard to get right in a web-based WYSIWYG editor, is mobile support.

Initially, I thought that a good editing experience on the mobile platform can only be achieved with a native mobile app, so why bother trying to get it right on the mobile web?

Well, ultimately I tried to add mobile support to Vrite Editor anyway, and, to my surprise, it turned out pretty good!

There were many things that needed optimization — and some that still do. From the general layout, toolbar, and the UI of block elements to formatting and block content menus — everything had to be in some way optimized or completely changed for mobile.

Now, there’s still some work to be done here. Monaco editor doesn’t work well for mobile, and some “optional” features weren’t yet optimized (like link previews or comments).

Still, with the work already done, Vrite Editor is usable and actually quite comfortable on mobile.

By the way, I did write this section on my phone. 😊

Markdown Exports

Now, with the editor working, the last important part was to output Markdown. TipTap provides methods to retrieve both HTML and ProseMirror JSON representations of the content. I decided to use the JSON (since it’s easier to work with) and created functions to transform it into various outputs. I called these “Content Transformers”.

Using Content Transformers I created an “Export” menu, outputting the content in HTML, GitHub Flavored Markdown (GFM), and ProseMirror JSON (e.g. for custom processing) to a Monaco Editor, with options to download or copy it.

While there might be better ways to do this, this one provides a lot of flexibility and makes sure your content can be properly processed to Markdown or any other format you need.

Vrite CMS

In case you enjoy using Vrite Editor, you might be interested in a larger project that it’s a part of — “Vrite CMS”, or simply “Vrite”.

Vrite is a headless CMS for technical content, providing editing and management tools to work with content like technical product documentation, programming blogs, etc. Basically, it’s my answer to (what I consider) a lack of good tooling focused on this kind of content.

Beyond just the Vrite Editor, you get additional features, like:

Kanban dashboard for content management;
API and Webhooks;
Extension system for easy publishing to popular platforms;
Real-time collaboration;
and more!

Since Vrite (and Vrite Editor for that matter) is currently in Public Beta, new features and improvements are in active development. The best way to try it out right now is through the hosted version at app.vrite.io (free while in Beta) with better self-hosting support in the works.

Bottom Line

In any case, if you’re interested in technical writing, definitely check out Vrite Editor and Vrite as a whole. Also, leave a 🌟 on GitHub, as well as any feature requests or issues you may have, to support the project. ❤️

How I put ChatGPT into a WYSIWYG editor

Arek Nawo — Mon, 19 Jun 2023 20:44:18 +0000

With all the hype going on, AI (or rather Machine Learning (ML) and Large Language Models (LLMs) are everywhere. Personally, I might not use ChatGPT (and similar alternatives) much, but I sure do rely on likes of GitHub Copilot (for intelligent autocompletion in VS Code), or Grammarly (for editing my blog posts) every day.

I think we’re still quite a few breakthroughs away from AGI and current technology won’t be enough to get us there (thankfully or not). That said, we’re already deep into the times of “AI-enhanced” apps, where the apps at the top may not have the best AI systems, but they integrate them in the best possible way.

That’s why it was an interesting process, exploring OpenAI’s API and trying to integrate it into the Rich Text Editor (RTE) of Vrite — my open-source headless CMS.

Extending the WYSIWYG Editor

For those unfamiliar Vrite, in a nutshell, is a headless CMS for technical content, like programming blogs or software docs. It can be viewed as two apps in one — Kanban dashboard for content management and WYSIWYG editor for writing, with additional dev-friendly features like embedded code snippet editor and formatter.

The latest big addition to Vrite is an early extension system, to easily build integrations and extend what Vrite can do. This, to me, seemed like the perfect way to introduce ChatGPT into the editor — as an extension.

Block Actions

To be able to use the extension system for integrating ChatGPT into the editor, a new API had to be introduced. I called it Block Action API since its specifically meant for adding quick actions to the editor, that operate on top-level content blocks, like paragraphs, headings, or images, as is highlighted below:

With Block Actions API, extensions can read the JSON content of the active block and update it with HTML-formatted content, just like it’s done in Vrite API (on one end, parsing JSON output is easier while, on the other, HTML is more suitable to transform content into).

From the UI side, Block Actions are displayed as buttons on the side of the actively-selected block. They can either invoke an action directly on click or — like with ChatGPT — open a dropdown menu to prompt the user for more details.

The buttons had to be absolutely positioned, which required both a custom TipTap extension and tapping deeper into the underlying ProseMirror (both libraries powering the Vrite editor).

The process basically came down to figuring out the position and size of the block node, given a selection of an entire top-level node or just its child node (source code):

// ...
const BlockActionMenuPlugin = Extension.create({
  // ...
  onSelectionUpdate() {
    const { selection } = this.editor.state;
    const isTextSelection = selection instanceof TextSelection;
    const selectedNode = selection.$from.node(1) || selection.$from.nodeAfter;

    if (!selectedNode) {
      box.style.display = 'none';
      return;
    }

    const { view } = this.editor;
    const node =
      view.nodeDOM(selection.$from.pos) ||
      view.nodeDOM(selection.$from.pos - selection.$from.parentOffset) ||
      view.domAtPos(selection.$from.pos)?.node;

    if (!node) return;

    const blockParent = getBlockParent(node);
    const parentPos = document
      .getElementById('pm-container')
      ?.getBoundingClientRect();
    const childPos = blockParent?.getBoundingClientRect();

    if (!parentPos || !childPos) return;

    const relativePos = {
      top: childPos.top - parentPos.top,
      right: childPos.right - parentPos.right,
      bottom: childPos.bottom - parentPos.bottom,
      left: childPos.left - parentPos.left,
    };

    let rangeFrom = selection.$from.pos;
    let rangeTo = selection.$to.pos;

    box.style.top = `${relativePos.top}px`;
    box.style.left = `${relativePos.left + parentPos.width}px`;
    box.style.display = 'block';

    if (isTextSelection) {
      try {
        const p = findParentAtDepth(selection.$from, 1);
        rangeFrom = p.start - 1;
        rangeTo = p.start + p.node.nodeSize - 1;
      } catch (e) {
        box.style.display = 'none';
      }
    }

    // ...
  },
});

Replacing Editor Content

The second part involved handling the actual process of replacing the block’s content with the newly provided one. The trickiest thing to figure out here was to get the correct range (the start and end position in ProseMirror) of the block node. This was necessary to then use TipTap’s commands to properly replace the range.

If you’ve taken a closer look at the last code snippets — the code for that was already there. The block’s range was updated, together with the Block Action UI positioning, on every selection update.

The actual replacement of the range with new content was much easier to do. All there was to it was converting the HTML to Schema-adherent JSON and involving proper commands (source code):

// ...
const replaceContent = (content) => {
  unlock.clear();
  setLocked(true);
  if (range()) {
    let size = 0;
    const nodeOrFragment = createNodeFromContent(
      content,
      props.state.editor.schema
    );

    if (nodeOrFragment instanceof PMNode) {
      size = nodeOrFragment.nodeSize;
    } else {
      size = nodeOrFragment.size;
    }
    props.state.editor
      .chain()
      .focus()
      .insertContentAt(
        range()!,
        generateJSON(content, props.state.editor.extensionManager.extensions)
      )
      .scrollIntoView()
      .focus()
      .run();
    setRange({ from: range()!.from, to: range()!.from + size - 1 });
    computeDropdownPosition()();
  }
  unlock();
};
// ...

The replaceContent() function could then be called remotely, from the extension’s sandbox, by sending a proper message to the main frame.

To enable use-cases like ChatGPT integrations, where the content will be updated (i.e. replaced) multiple times in a row before the process is finished, the function also locked the editor for a short time of the function being called and updated the range, and UI positioning on every call. But why was this required?

Integrating With OpenAI’s API

The process of integrating OpenAI’s API is pretty well-documented in its official docs. Given that an official SDK is provided, the entire process can be done in just a few lines of code:

async ({ ctx, input }) => {
  const configuration = new Configuration({
    apiKey: ctx.fastify.config.OPENAI_API_KEY,
    organization: ctx.fastify.config.OPENAI_ORGANIZATION,
  });
  const openai = new OpenAIApi(configuration);
  const response = await openai.createChatCompletion({
    model: 'gpt-3.5-turbo',
    messages: [{ role: 'user', content: input.prompt }],
  });
};

Now, all that is true, but only if you’re willing to wait what often is +20s for a single response! That’s a lot for a single request. Nothing from changing the server location to optimizing the request by limiting max_tokens or customizing other parameters worked. It all comes down to the fact that current-day LLMs (those on the level of GPT-3 at least) are still rather slow.

With that said, the ChatGPT app still manages to be perceived as fairly fast and responsive. That’s thanks to the use of streaming and Server-Sent Events (SSEs).

Streaming ChatGPT Response

The chat completion and other endpoints of OpenAI’s API support streaming through Server-Sent Events, essentially maintaining an open connection through which the new tokens are sent as soon as they’re available.

Unfortunately, the official Node.js SDK doesn’t support streaming and requires you to use workarounds to get it working, resulting in much more code required, just to connect with the API (source code):

async ({ ctx, input }) => {
  const configuration = new Configuration({
    apiKey: ctx.fastify.config.OPENAI_API_KEY,
    organization: ctx.fastify.config.OPENAI_ORGANIZATION,
  });
  const openai = new OpenAIApi(configuration);
  const response = await openai.createChatCompletion(
    {
      model: 'gpt-3.5-turbo',
      stream: true,
      messages: [{ role: 'user', content: input.prompt }],
    },
    { responseType: 'stream' }
  );
  ctx.res.raw.writeHead(200, {
    ...ctx.res.getHeaders(),
    'content-type': 'text/event-stream',
    'cache-control': 'no-cache',
    connection: 'keep-alive',
  });

  return new Promise<void>((resolve) => {
    const responseData = response.data as unknown as {
      on(event: string, data: (data: string) => void): void;
    };

    responseData.on('data', (data) => {
      const lines = data
        .toString()
        .split('\n')
        .filter((line) => line.trim() !== '');
      for (const line of lines) {
        const message = line.replace(/^data: /, '');
        if (message === '[DONE]') {
          ctx.res.raw.end();
          resolve();
          continue;
        }
        try {
          const parsed = JSON.parse(message);

          const content = parsed.choices[0].delta.content || '';

          if (content) {
            ctx.res.raw.write(`data: ${encodeURIComponent(content)}`);
            ctx.res.raw.write('\n\n');
          }
        } catch (error) {
          console.error('Could not JSON parse stream message', message, error);
        }
      }
    });
  });
};

On top of that, you also have to support streaming on your end, between your API server and web client which, in the case of Vrite, meant integrating SSEs with Fastify and tRPC. Not the cleanest solution, but pretty stable nonetheless.

From the frontend (the extension sandbox to be precise), a connection with the new streaming endpoint has to be established and incoming data — correctly processed (source code):

import { fetchEventSource } from "@microsoft/fetch-event-source";
// ...

const generate = async (context: ExtensionBlockActionViewContext): Promise<void> => {
  const includeContext = context.temp.includeContext as boolean;
  const prompt = context.temp.prompt as string;

  let content = "";

  context.setTemp("$loading", true);
  window.currentRequestController = new AbortController();
  window.currentRequestController.signal.addEventListener("abort", () => {
    context.setTemp("$loading", false);
    context.refreshContent();
  });
  await fetchEventSource("https://extensions.vrite.io/gpt", {
    method: "POST",
    headers: {
      "Content-Type": "application/json",
      "Accept": "text/event-stream"
    },
    body: JSON.stringify({
      prompt: includeContext ? `"${gfmTransformer(context.content)}"\n\n${prompt}` : prompt
    }),
    signal: window.currentRequestController?.signal,
    async onopen() {
      return;
    },
    onerror(error) {
      context.setTemp("$loading", false);
      context.refreshContent();
      context.notify({ text: "Error while generating content", type: "error" });
      throw error;
    },
    onmessage(event) {
      const partOfContent = decodeURIComponent(event.data);

      content += partOfContent;
      context.replaceContent(marked.parse(content));
    },
    onclose() {
      context.setTemp("$loading", false);
      context.refreshContent();
    }
  });
};

The EventSource Web API for handling SSEs (built into most modern browsers) unfortunately supports only GET requests, which was quite limiting when a POST request with larger body JSON data was required. As an alternative, you can use the Fetch API or a ready library like Microsoft’s Fetch Event Source.

Again, with streaming enabled, you’ll now receive new tokens as soon as they’re available. Given that OpenAI’s API uses Markdown in its response format, a full message will need to be put together from the incoming tokens and parsed to HTML, as accepted by the replaceContent function. For this purpose, I’ve used the Marked.js parser.

Now, with each new token, the larger response is being built up. Every time a new token comes, the full Markdown is parsed and the content updated, making for a nice “typing-like effect”.

While this process does have some overhead, it’s not noticeable in use, while the Markdown simply has to be parsed with each new token, as it may contain e.g. the closing of the code block, or the end of the formatted segment. So, while this process could potentially be optimized, it wouldn’t lead to any recognizable performance improvement in the majority of cases.

Finally, worth noting is the use of AbortController, which can be used to stop the stream at any time the user chooses to. That’s especially great for longer responses.

Bottom Line

In general, I’m very happy with how this turned out. Data streaming, nice typing effect, and good integration with the editor’s existing content blocks thanks to Markdown parsing — all came together to create a compelling User Experience.

Now, there’s certainly room for improvement. The Block Actions API, as well as the Vrite Extensions as a whole still have a lot of development work ahead of them before they can be created by other users. Other UI/UX improvements to consider, like operating on multiple blocks at once (e.g. for additional context for ChatGPT) and displaying UI inline (much like Notion AI) not to obscure the view are just a few examples of what I was considering. That said, it’ll take some more time to implement these ideas well.

Vrite is much more than just a GPT-enhanced editor. It’s a full, open-source CMS focused on technical content like programming blogs, with a code editor, API, Kanban management dashboard included and easy publishing integrations included. So, if you’re interested in trying it out and possibly using it to power your blog, definitely check it out!

🌟 Star Vrite on GitHub — https://github.com/vriteio/vrite
🐞 Report bugs — https://github.com/vriteio/vrite/issues
🐦 Follow on Twitter — https://twitter.com/vriteio
💬 Join Vrite Discord — https://discord.gg/yYqDWyKnqE
ℹ️ Learn more about Vrite — https://vrite.io
📕 Vrite documentation — https://docs.vrite.io

Next-Level Technical Blogging with Dev.to API

Arek Nawo — Tue, 13 Jun 2023 19:49:08 +0000

Having a platform like DEV at the center of your technical blog is a great option. With over 1 million users, Dev.to is one of the largest developer communities there is, welcoming technical writers and readers at all levels.

That said, things are complicated when it comes to writing and delivering content on DEV. With its minimal editor, you’ll likely have to do a lot of copy-pasting and manual adjustments to get your post from your editor to DEV and other platforms you might want to cross-post to, like Hashnode.

Thankfully, DEV’s underlying platform — Forem — has a great API you can leverage to both publish and fetch blog posts from DEV with ease. Let me show you how…

Getting Started With Forem API

Currently, there are 2 versions of the Forem API available — version 0 and version 1. They differ a bit in the request format but provide similar endpoints. You can use either one of them (both are documented in the official docs), though it’s recommended to use the newer v1.

To switch between different versions, you have to provide proper HTTP headers. For v1, these are:

API-Key: [your-api-key]
Accept: application/vnd.forem.api-v1+json
Content-Type: application/json

While for the v0:

API-Key: [your-api-key]
Accept: application/json
Content-Type: application/json
User-Agent: Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; .NET CLR 1.1.4322)

The key difference is in the Accept header — you have to specify application/vnd.forem.api-v1+json to enable the newer version. On top of that, for v0, you have to provide a valid User-Agent header, as otherwise your request will result in 403 Forbidden error. The provided value is only an example, that I verified to work.

From this point on, I’ll be referencing only v1, though all the endpoints described are the same in both versions.

With all this made clear, all you need now is an API key.

Getting Your API Key

To get your API key you have to go to the very bottom of the Extensions tab of your Dev.to account’s Settings page. There you’ll see the DEV Community API Keys section:

Enter a custom description and Generate API Key. You should see a new API key appear on the list, from where you can copy it.

Fetching Articles From DEV

Now, there’s a lot you can do with this API and its endpoints, which are all pretty well-documented. So, rather than serving as the second documentation, I’d like to discuss two of probably the biggest use cases for this API.

The first one is to use Dev.to as your content hub, making DEV the center of your programming publication, while also serving the content on your own website, like a custom blog or portfolio page.

The most important endpoint for this use case would be GET /api/articles/me, which allows you to retrieve all of your published articles. Here’s how you can use it:

interface DEVArticle {
  type_of: string;
  id: number;
  title: string;
  description: string;
  published: boolean;
  published_at: string;
  slug: string;
  path: string;
  url: string;
  comments_count: number;
  public_reactions_count: number;
  page_views_count: number;
  published_timestamp: string;
  body_markdown: string;
  positive_reactions_count: number;
  cover_image: string;
  tag_list: string[] | string;
  tags?: string[];
  canonical_url: string;
  reading_time_minutes: number;
  user: {
    name: string;
    username: string;
    twitter_username: string | null;
    github_username: string | null;
    user_id: number;
    website_url: string | null;
    profile_image: string;
    profile_image_90: string;
  };
  organization?: {
    name: string;
    username: string;
    slug: string;
    profile_image: string;
    profile_image_90: string;
  };
}
interface GetArticlesOptions {
  pages?: number;
  perPage?: number;
}
const getArticles = async (
  options?: GetArticlesOptions
): Promise<DEVArticle[]> => {
  const articles: DEVArticle[] = [];
  const pages = options?.pages || 1;
  const perPage = options?.perPage || 30;

  for (let page = 1; page <= pages; page += 1) {
    const response = await fetch(
      `https://dev.to/api/articles/me?per_page=${perPage}&page=${page}`,
      {
        method: 'GET',
        headers: {
          'api-key': 'your-api-key',
          accept: 'application/vnd.forem.api-v1+json',
          'content-type': 'application/json',
        },
      }
    );
    const json = await response.json();

    articles.push(...json);
  }

  return articles;
};

getArticles({ pages: 3, perPage: 100 }).then((result) => {
  console.log(result);
});

The API is CORS-enabled, meaning you’ll have to use the getArticles() functions from your backend. For making the actual request, you can use the fetch() function, available since Node.js v18. For older versions of Node.js, you can use a fetch()-compatible library like node-fetch.

Converting Markdown to HTML

Once you have the article data, you’ll likely have to process its body_markdown to a format required by your website, like HTML. There are many tools you can do this with — here’s an example using Marked.js:

const markdownToHTML = (markdown: string): string => {
  const regex = new RegExp(/^.*{%\s?(.+?) (.+?)\s?%}.*(?:\n|$)/);
  const getEmbedUrl = (embedType: string, input: string): string => {
    switch (embedType) {
      case 'youtube':
        return `https://www.youtube.com/embed/${input}`;
      case 'codepen':
        return input.replace(/\/pen\//, '/embed/');
      case 'codesandbox':
        return `https://codesandbox.io/embed/${input}`;
      default:
        // etc...
        return '';
    }
  };
  const embedExtension = {
    name: 'embedExtension',
    level: 'block',
    start(src) {
      return src.match(/^.*{%/)?.index;
    },
    tokenizer(src) {
      const match = regex.exec(src);

      console.log('tokenizer', src, match);
      if (match) {
        return {
          type: 'embedExtension',
          raw: match[0],
          embedType: match[1].trim(),
          input: match[2].trim(),
          tokens: [],
        };
      }
    },
    renderer(token) {
      return `<iframe src="${getEmbedUrl(
        token.embedType,
        token.input
      )}"></iframe>`;
    },
  };
  marked.use({ gfm: true, extensions: [embedExtension] });

  return marked.parse(markdown);
};

For the most part, DEV uses a standard Markdown format, which is easily handled by Marked.js and its built-in GitHub Flavored Markdown support. For embeds (i.e. liquid tags), you can easily extend Marked.js using its Custom Extension system. Above is an example of an extension converting liquid tags to <iframe> elements, creating proper URLs for embeds like YouTube, CodePen, or CodeSandbox. You should be able to easily extend it to handle the embeds you use.

If you want to go beyond fetching articles, Forem API provides various other endpoints I encourage you to explore. Here are a few examples:

Retrieve your profile details;
Fetch comments for the article;
Filter articles based on tags;

Publishing on DEV via API

An arguably more popular use case for an API like Forem’s is simplified or automatic publishing.

In general, there’s a lot of copy-pasting when writing a technical article — between various code and text editors, publications like DEV or Hashnode, and potentially even a CMS powering your custom blog. Going through this whole process for every single article can be tiring.

That said, with some automation and different APIs, you can simplify this process to a high degree. In the case of Forem API, you can use the POST /api/articles to first publish an article on DEV and then update it as needed with PUT /api/articles/{id}. Here’s an example:

interface DEVArticleInput {
  title?: string;
  body_markdown?: string;
  published?: boolean;
  series?: string | null;
  main_image?: string | null;
  canonical_url?: string | null;
  description?: string;
  tags?: string[];
  organization_id?: number | null;
}

const publishArticle = async (
  article: DEVArticleInput
): Promise<DEVArticle> => {
  const response = await fetch('https://dev.to/api/articles', {
    method: 'POST',
    headers: {
      'api-key': 'your-api-key',
      accept: 'application/vnd.forem.api-v1+json',
      'content-type': 'application/json',
    },
    body: JSON.stringify({
      article,
    }),
  });
  const json = await response.json();

  return json;
};
const updateArticle = async (
  articleId: string | number,
  articleUpdate: DEVArticleInput
): Promise<DEVArticle> => {
  const response = await fetch(`https://dev.to/api/articles/${articleId}`, {
    method: 'PUT',
    headers: {
      'api-key': 'your-api-key',
      accept: 'application/vnd.forem.api-v1+json',
      'content-type': 'application/json',
    },
    body: JSON.stringify({
      article: articleUpdate,
    }),
  });
  const json = await response.json();

  return json;
};

With these two functions, you can easily publish and update your articles as needed, e.g.

publishArticle({
  title: 'Hello, world!',
  body_markdown: '# Hello World\nThis is my first post published through Forem API!',
  published: false,
}).then((article) => {
  console.log(article);
});

// Later on
updateArticle('1234567', {
  tags: ['javascript', 'typescript', 'react', 'node'],
  series: 'Hello World',
  canonical_url: 'https://example.com',
}).then((updatedArticle) => {
  console.log(updatedArticle);
});

Now, you’ll need to convert your content input to a DEV-compatible Markdown format, doing a somewhat inverse operation to what was demonstrated earlier with Marked.js. This will likely require some custom processing to best fit your use case.

Now, when properly implemented, such publishing automation can greatly improve your publishing experience. Still, this likely won’t help you escape copy-pasting between your editors.

That said, if you’re looking for something that could provide a great experience across your entire technical writing process — from writing and coding to content management and publishing — then you might be interested in my latest project — Vrite — open-source headless CMS for technical content.

Easy Publishing With Vrite

Vrite combines multiple tools and concepts that are familiar to all developers, like:

Kanban dashboard for content management;
Modern rich text editor with built-in code editor and Prettier code formatter;
Content delivery API with dedicated JavaScript SDK;

in the form of headless CMS. It’s a unique combination that, in my opinion, has the potential to provide the best experience across the entire technical writing space. For more details, you can check out the official guide.

Publishing via Dev.to Extension

Staying on the topic of Dev.to, Vrite’s latest feature — Extensions — make integrating Vrite with DEV as easy as clicking a few buttons, allowing you to easily publish and update content from Vrite to DEV both manually and automatically.

Right now, Vrite Extensions enable easy integration with platforms like DEV, Hashnode or Medium and extend the capabilities of Vrite editor with the power of OpenAI’s GPT-3.5, with a lot more to come.

Setting up an extension is easy and can be done from the Extensions side panel:

You can check out this blog post for a more detailed introduction to Vrite Extensions.

Bottom Line

There’s a lot you can do with Dev.to’s API. Between fetching and publishing content it’s really versatile and can be used to handle a lot of custom use cases.

That said, if you’re interested in an even better technical writing experience, definitely give Vrite a try and let me know what you think about it and how I can make it better!

🌟 Star Vrite on GitHub — https://github.com/vriteio/vrite
🐞 Report bugs — https://github.com/vriteio/vrite/issues
🐦 Follow on Twitter — https://twitter.com/vriteio
💬 Join Vrite Discord — https://discord.gg/yYqDWyKnqE
ℹ️ Learn more about Vrite — https://vrite.io
📕 Vrite documentation — https://docs.vrite.io

Start programming blog in minutes with Astro and Vrite

Arek Nawo — Tue, 23 May 2023 20:02:35 +0000

Running a programming blog is a great way to hone your skills, build a personal brand and expand your portfolio. However, it certainly requires a lot of effort to both get started and to keep going. You not only need to have the programming knowledge and writing skills but also the ability to make quick and sure decisions.

Developers (being developers) will often spend a lot of time picking their tech stacks, rather than building the blog or creating content. Now, while this is certainly beneficial for learning about new tools, sometimes you just need to ship! 🚢

Thus, more recently, for any kind of static or content-heavy websites, like a landing page, blog, etc. I’ve defaulted to Astro.

Astro

Now, if you haven’t used Astro before - you’re missing out. It’s a web framework for building really fast SSG or SSR-powered websites.

It does so through island architecture, by only loading the JS code of your components when required — e.g. when the component becomes visible or the browser is idle — and shipping static pre- or server-rendered HTML in all other cases.

On top of that, it integrates really well with your existing stack, including various UI frameworks, like React, Vue, or my recent favorite — Solid.js — for building the interactive parts of your UI.

Vrite — CMS for Technical Content

With Astro handling the front end, you still need some kind of data source for your blog. While a few Markdown files can do the job, I don’t consider that the most scalable or comfortable setup.

Unfortunately, most CMSs won’t be ideal either, as writing about code is a very specific, rather niche use case. Goodies like code highlighting, formatting, or proper Markdown and Keyboard shortcuts support aren’t guaranteed. Not to mention, the additional requirements you might have when working in teams, like real-time collaboration or proper content management tools.

All these reasons taken together are essentially why I created Vrite, which is basically a CMS for technical content. With everything above and more included — like code editor, Prettier integration, Kanban dashboard, and more.

Vrite and Astro

Now, Vrite is open-source and currently in Public Beta. This means that, while there might be bugs, you can already do quite a lot with it. For example, you can easily integrate it with the DEV platform via its API. However, by combining it with Astro, you can take it a step further!

With its great performance and support for Solid.js (the framework Vrite is built with), Astro is already powering the Vrite landing page and blog — and does so really well!

This great pairing inspired me to create Vrite’s first dedicated integration — one for Astro.

So, with Astro, Vrite, and easy integration between the two, it’s possible to get a blog up and running in minutes! Let me show you how…

Building a Blog With Astro and Vrite

Source code here: https://github.com/areknawo/start-programming-blog-in-minutes-with-astro-and-vrite

Since Vrite is open-source and based on Node.js, you’ll soon be able to self-host it pretty easily if you want to. However, until I document this process and stabilize Vrite, the best way to try it out is through a hosted instance at app.vrite.io. So, start by signing in for an account:

When you’re in, you’ll see a Kanban dashboard:

You can read more about how the content is organized in Vrite in my previous blog post. However, for now, all you need to know is that individual columns are called content groups — meant for organizing your content — while the cards inside — content pieces — are containing the actual content with related metadata.

Create a few content groups to represent your content production process (e.g. Ideas, Drafts, Published) by clicking New group, and then create your first content piece by clicking New content piece in the content group of choice.

With a content piece now created and selected, you’ll see a side panel, containing all its metadata. Inspired by code editors like VS Code, with which most developers are quite familiar, the resizable side panel is where you’ll edit metadata, configure all available settings, manage your team, and more.

Once the content piece is selected (opened in the side panel and highlighted), you can move to the editor by clicking the Editor button in the side toolbar.

Vrite editor is focused on providing the best technical writing experience possible. Thanks to many features, like:

Modern, clean UI/UX with WYSIWYG editing experience;
Markdown and keyboard shortcuts;
Built-in code editor with code highlighting, autocompletio,n and Prettier formatting (for supported languages);
Support for most of the GFM (GitHub-Flavored Markdown) formatting options and content blocks;

it’s easy to get started with and can cover vast majority of use cases in technical blogging.

To customize the editor to the needs of your blog, you can go to Settings → Editing experience to enable/disable various formatting options and content blocks and provide your own Prettier config for code formatting.

Now, to integrate with Vrite you’ll need an API token. To get it, head to Settings → API → New API token.

Here, you can customize the token name, description, and permissions. It’s recommended to only use necessary permissions, which for a personal blog would likely mean read access to content pieces and content groups (to retrieve the content), tags, and profile to build tag-based lists and provide some profile info about you (which you can configure in Settings → Profile).

Save your API token and keep it safe. Now it’s time to create an Astro-powered blog!

Start by creating a new Astro project:

npm create astro@latest

When prompted, choose "Use blog template” as this will get you started the fastest.

Now, cd into the new projects and install Vrite JS SDK:

npm install @vrite/sdk

The SDK provides all the tools you need to interact with Vrite’s API, including the API client, and Content Transformers. I’ve already covered those in a previous blog post. However, you don’t actually have to think about them when working with Astro — thanks to the dedicated integration!

First, create a .env file to house your Vrite API token and an ID of the content group to publish content from:

VRITE_ACCESS_TOKEN=
VRITE_CONTENT_GROUP_ID

To get the ID of the content group, in the Kanban dashboard, open the context menu and Copy ID:

Then, in the astro.config.mjs import and configure the integration, by first loading the ENV variables and then providing the Vrite plugin in the integrations array:

import { defineConfig } from 'astro/config';
import mdx from '@astrojs/mdx';
import { vritePlugin } from '@vrite/sdk/astro';
import sitemap from '@astrojs/sitemap';
import { loadEnv } from 'vite';

const { VRITE_ACCESS_TOKEN, VRITE_CONTENT_GROUP_ID } = loadEnv(
  import.meta.env.MODE,
  process.cwd(),
  ''
);

export default defineConfig({
  site: 'https://example.com',
  integrations: [
    mdx(),
    sitemap(),
    vritePlugin({
      accessToken: VRITE_ACCESS_TOKEN,
      contentGroupId: VRITE_CONTENT_GROUP_ID,
    }),
  ],
});

Additionally, for the best experience, add the Vrite SDK types to your tsconfig.json file:

{
  "extends": "astro/tsconfigs/strict",
  "compilerOptions": {
    "strictNullChecks": true,
    "types": ["@vrite/sdk/types", "astro/client"]
  }
}

Now the integration is configured and ready to go! It’s time to use it in the code.

Inside src/pages/blog/index.astro file let’s display a full list of existing blog posts:

---
// ...
import { getContentPieces } from "virtual:vrite";

const posts = await getContentPieces({ limit: "all" });
---

<!DOCTYPE html>
<html lang="en">
    <!-- ... -->
  <body>
    <Header />
    <main>
      <section>
        <ul>
          {
            posts.map((post) => (
              <li>
                {post.date && <FormattedDate date={new Date(post.date)} />}
                <a href={`/blog/${post.slug}/`}>{post.title}</a>
              </li>
            ))
          }
        </ul>
      </section>
    </main>
    <Footer />
  </body>
</html>

All it takes is a getContentPieces() function call to retrieve all the content pieces from the configured content group, in the order seen in the Kanban dashboard. Here we retrieve all the available content pieces (by providing { limit: “all” }, though, for bigger blogs, you’ll likely want to use pagination, which is also supported by this utility.

The function itself comes from the virtual:vrite module that provides a few other utilities like this, plus a fully-configured API client to make your work with Vrite and Astro a breeze. Additionally, (if you configured your tsconfig.json), it’s fully typed, providing great DX!

With that, you should now see the list of the content pieces on the website:

What about the actual content? Well, for this you should move to src/pages/blog/[…slug].astro:

---
import BlogPost from "../../layouts/BlogPost.astro";
import { getStaticPaths, ContentPiece, Content } from "virtual:vrite";

export { getStaticPaths };
type Props = ContentPiece;

const contentPiece = Astro.props;
---

<BlogPost
  title={contentPiece.title}
  description={contentPiece.description || ""}
  pubDate={contentPiece.date ? new Date(contentPiece.date) : new Date()}
  heroImage={contentPiece.coverUrl}
>
  <Content contentPieceId={contentPiece.id} />
</BlogPost>

In Astro, you can use dynamic routes, together with rest parameters, i.e. […slug] to generate pages from an external source, like a CMS. The parameter can be used to match and then fetch the correct data in the getStaticPaths() function (when in the SSG mode).

This is such a common use case that the Vrite integration implements this function for you! Simply make sure your route has a slug parameter, re-export the function from virtual:vrite module and — you’re done!

The rest of the file is used to render the blog post. All the data of the content piece is available via Astro.props (which you can strongly-typed by adding type Props = ContentPiece;). You can then supply this data to a BlogPost layout provided by the template. Finally, to render the content, use the Content component provided by the integration, and supply it with the content piece ID.

With that, your blog post should now be visible under the generated URL:

The Content component automatically renders your blog post to HTML and uses Astro’s Code component for displaying and highlighting the code snippets, though you’ll have to npm install shiki for it to work:

Now you have a blog basically ready to go. If you want to customize the integration further, you can always tap into the underlying Vrite API client to e.g. retrieve your Vrite profile. In src/pages/about.astro:

---
import Layout from "../layouts/BlogPost.astro";
import { client } from "virtual:vrite";

const profile = await client.profile.get();
---

<Layout
  title={profile.fullName || "About me"}
  description="Lorem ipsum dolor sit amet"
  pubDate={new Date("May 23 2023")}
  updatedDate={new Date("May 23 2023")}
  heroImage="/placeholder-about.jpg"
>
  <p>{profile.bio}</p>
</Layout>

That said, it’s worth noting that the profile data in Vrite is a bit limited right now and you’ll likely want to add some more information right into the page.

When you’re done, simply run npm run build and have Astro generate a super-fast static blog for you!

Bottom line

While I might be a bit biased, I think this is one of the fastest and easiest CMS integrations out there. Unless you’ve experienced some bugs — mind you Vrite is still in beta and you should report those on GitHub — with some styling, you’ll have a blog in no time! Additionally, if you ever need to scale, the Vrite and Astro combo easily supports pagination and SSR, with little to no additional setup!

Now, there’s still a lot more you can do with Vrite even now, that wasn’t covered in this blog post. For example — you can set up a Webhook to automatically rebuild and redeploy your static blog when you add a new content piece to the group (you can get some inspiration for that from this blog post). You can also invite your team to collaborate on content in real-time or cross-post using Webhooks and Content Transformers. Documentation and more content on all of that is coming soon.

Overall, if you’re running a programming blog or are a technical writer in need of better tools, I think Vrite is (or will soon become) what you were looking for. Either way, if you’re interested, definitely check out Vrite at app.vrite.io and follow me on this journey for better technical tooling!

🌟 Star Vrite on GitHub — https://github.com/vriteio/vrite
🐞 Report bugs — https://github.com/vriteio/vrite/issues
🐦 Follow on Twitter for the latest updates — https://twitter.com/vriteio
💬 Join Vrite Discord — https://discord.gg/yYqDWyKnqE
ℹ️ Learn more about Vrite — https://vrite.io

Better blogging on Dev.to with Vrite - headless CMS for technical content

Arek Nawo — Fri, 19 May 2023 17:06:08 +0000

With technical writing becoming increasingly popular - thanks in part to platforms like DEV or Hashnode, I found it interesting that the tooling in this niche is still lacking. You often have to write raw Markdown, jump between different editors and use many tools to support the content production process.

That’s why I decided to create Vrite - a new kind of headless CMS meant specifically for technical writing, with good developer experience in mind. From the built-in Kanban management dashboard to the advanced WYSIWYG editor with support for Markdown, real-time collaboration, embedded code editor, and Prettier integration - Vrite is meant to be a one-stop-shop for all your technical content.

With the release of the Public Beta earlier this week, Vrite is now open-source and accessible to everyone - to help guide the future roadmap and make the best tool for all technical writers!

DEV API

A CMS - especially a headless one - can only do so much without a connected publishing endpoint. In the case of Vrite, thanks to its API and flexible content format, it can be easily connected to anything from a personal blog to a GitHub repo or a platform like Dev.to.

Dev.to is an especially interesting option, since the API of the underlying platform - Forem - is well-documented and easily available. So, let’s see how to connect it with Vrite!

Getting Started With Vrite

Given that Vrite is open-source, you’ll soon be able to self-host it. That said, I’m still working on proper documentation and support for this process. For now, the best way to try out Vrite is through a free “cloud” version at app.vrite.io.

Start by signing up for an account - either directly or through GitHub:

When you’re in, you’ll be greeted by a Kanban dashboard. This is where you’ll be able to manage all your content:

At this point, it’s worth explaining how things are structured in Vrite:

Workspace - this is the top-most organizational unit in Vrite; it’s where all your content groups, team members, editing settings, and API access are controlled; A default one is created for you, though you can create and be invited to as many as you want;
Content Groups - the equivalents to columns in the Kanban dashboard; They basically group all the content pieces under one label, e.g. Ideas, Drafts, Published.
Content Pieces - where your actual content and its metadata - like description, tags, etc live;

Let’s say you want a completely new workspace for your Dev.to blog as you plan to publish unique content there. To create one, click the Switch Workspace button in the bottom-left corner (hexagon) and then New Workspace.

You need to provide a name and optionally - a description and logo. Then click Create Workspace and select the new workspace you’ve created from the list:

Back in the dashboard, you can now create a few content groups to organize your content by clicking New group. When that’s done, you can finally create a new content piece by clicking New content piece at the bottom of the column of your choice.

With a new content piece, you can view and configure all its metadata in the side panel. In Vrite, almost everything, aside from creating and managing content happens in this resizable view. This way you can always keep an eye on the content while editing metadata or configuring settings.

Now, click Open in editor either in the side panel or on the content piece card in the Kanban (you can also use the side-menu button) to open the selected content piece in the editor.

This is where the magic happens! Feel free to explore the editor while writing your next great piece. Vrite synchronizes all the changes in real time and supports many of the formatting options available in modern WYSIWYG editors. On top of that, you also get an advanced code editor for all your snippets with features like autocompletion and formatting for supported languages:

Connecting with DEV

When you’ve finished writing your next piece, it’s time to publish it! For convenience, the Vrite editor provides an Export menu where you can get the contents of your editor in JSON, HTML, or GitHub Flavored Markdown (GFM) for easy copy-pasting. However, to get a more proper auto-publishing experience, you’ll likely want to use Vrite API and Webhooks.

The intended workflow looks like this:

Drag and drop content pieces to the publishing column;
Send a message to the server via Webhooks;
Retrieve and process the content via the Vrite API and JS SDK;
Publish/update a blog post on Dev.to;

For this tutorial, I’ll use Cloudflare Workers as they’re really fast and easy to set up, but you can use pretty much any other serverless provider with support for JS.

Start by creating a new CF Worker project:

npm create cloudflare

Then, cd into the scaffolded project to wrangler login and install Vrite JS SDK:

wrangler login
npm i @vrite/sdk

To interact with the SDK, you’ll need to have an API token. To get it from Vrite, go to Settings → API → New API token:

It’s recommended to keep the permissions of the API token to the necessary minimum, which in this case means only Write access to Content pieces (as we’ll actually be updating the content piece metadata later on). After clicking Create new token you’ll be presented with the newly-created token. Keep it safe and secure - you’ll only see it once!

Additionally, to publish the content on Dev.to via its API, you’ll need to get an API key from it as well. To do so, go to the bottom of the settings in your DEV account and click Generate API Key:

Now, add both tokens to the Worker as environment variables via wrangler.toml:

name = "autopublishing"
main = "src/worker.ts"
compatibility_date = "2023-05-18"

[vars]
VRITE_API_TOKEN = "[YOUR_VRITE_API_TOKEN]"
DEV_API_KEY="[YOUR_DEV_API_KEY]"

Upon the event, Vrite sends a POST request to the configured target URL of the webhook with additional JSON payload. For our use case, the most important part of this payload will be the ID of a content piece that was just added to the given content group (either by drag and dropped or by being created directly)

Let’s finally create our Worker (inside src/worker.ts):

import { JSONContent, createClient } from '@vrite/sdk/api';
import { createContentTransformer, gfmTransformer } from '@vrite/sdk/transformers';

const processContent = (content: JSONContent): string => {
  // ...
};

export interface Env {
  VRITE_API_TOKEN: string;
  DEV_API_KEY: string;
}

export default {
  async fetch(request: Request, env: Env, ctx: ExecutionContext): Promise<Response> {
    const payload: { id: string } = await request.json();
    const client = createClient({ token: env.VRITE_API_TOKEN });
    const contentPiece = await client.contentPieces.get({
      id: payload.id,
      content: true,
      description: 'text',
    });
    const article = {
      title: contentPiece.title,
      body_markdown: processContent(contentPiece.content),
      description: contentPiece.description || undefined,
      tags: contentPiece.tags.map((tag) => tag.label?.toLowerCase().replace(/\s/g, '')).filter(Boolean),
      canonical_url: contentPiece.canonicalLink || undefined,
      published: true,
      series: contentPiece.customData?.devSeries || undefined,
      main_image: contentPiece.coverUrl || undefined,
    };

    if (contentPiece.customData?.devId) {
      try {
        const response = await fetch(`https://dev.to/api/articles/${contentPiece.customData.devId}`, {
          method: 'PUT',
          headers: {
            'api-key': env.DEV_API_KEY,
            Accept: 'application/json',
            'content-type': 'application/json',
            'User-Agent': 'Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; .NET CLR 1.1.4322)',
          },
          body: JSON.stringify({
            article,
          }),
        });
        const data: { error?: string } = await response.json();

        if (data.error) {
          console.error('Error from DEV: ', data.error);
        }
      } catch (error) {
        console.error(error);
      }
    } else {
      try {
        const response = await fetch(`https://dev.to/api/articles`, {
          method: 'POST',
          body: JSON.stringify({ article }),
          headers: {
            'api-key': env.DEV_API_KEY,
            Accept: 'application/json',
            'content-type': 'application/json',
            'User-Agent': 'Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; .NET CLR 1.1.4322)',
          },
        });
        const data: { error?: string; id?: string } = await response.json();

        if (data.error) {
          console.error(data.error);
        } else if (data.id) {
          await client.contentPieces.update({
            id: contentPiece.id,
            customData: {
              ...contentPiece.customData,
              devId: data.id,
            },
          });
        }
      } catch (error) {
        console.error(error);
      }
    }

    return new Response();
  },
};

What’s going on here? We start by initiating the Vrite API client and fetching metadata and the content related to the content piece that triggered the event. Then, we use this data to create an article object that’s expected by the DEV API and use it to make a request.

In Vrite, in addition to strictly-defined metadata like tags or canonical links, you can also provide JSON-based custom data. It’s configurable both from the dashboard and through the API, making it a great storage for data like, in this case, DEV article ID, which allows us to determine whether to publish a new article or update an existing one (using a custom devId property). The same mechanism is applied for retrieving the name of the series the article should be assigned to on DEV, which can be configured from the Vrite dashboard using a custom devSeries property.

Worth noting is that, for requests to DEV API, we’re passing a generic User-Agent header - it’s necessary to make a successful request without 403 bot-detection error.

Content Transformers

You might have noticed that the body_markdown property is set to the result of processContent() call. That’s because the Vrite API serves its content in a JSON format. Derived from the ProseMirror library powering Vrite editor, the format allows for versatile content delivery as it can be easily adapted to various needs.

The Vrite JS SDK has built-in tools for transforming this format called Content Transformers. They allow you to easily process the JSON to a string-based format, like HTML or GFM (both of which have dedicated transformers built into the SDK).

For DEV, using the GFM transformer would be fine in most cases. However, this transformer ignores embeds that are supported by both the Vrite editor and DEV (i.e. CodePen, CodeSandbox, and YouTube) as they aren’t supported in the GFM specification. Thus, let’s build a custom Transformer that extends the gfmTransformer to add support for these embeds:

import { JSONContent, createClient } from '@vrite/sdk/api';
import { createContentTransformer, gfmTransformer } from '@vrite/sdk/transformers';

const processContent = (content: JSONContent): string => {
  const devTransformer = createContentTransformer({
    applyInlineFormatting(type, attrs, content) {
      return gfmTransformer({
        type,
        attrs,
        content: [
          {
            type: 'text',
            marks: [{ type, attrs }],
            text: content,
          },
        ],
      });
    },
    transformNode(type, attrs, content) {
      switch (type) {
        case 'embed':
          return `\n{% embed ${attrs?.src || ''} %}\n`;
        case 'taskList':
          return '';
        default:
          return gfmTransformer({
            type,
            attrs,
            content: [
              {
                type: 'text',
                attrs: {},
                text: content,
              },
            ],
          });
      }
    },
  });

  return devTransformer(content);
};

// ...

A Content Transform goes through the JSON tree - from the lowest to the highest-level nodes - and processes every node, always passing in the resulting content string generated from the child nodes.

In the processContent() function above, we’re redirecting the processing of inline formatting options (like bold, italic, etc.) - to the gfmTransformer, as both GFM and DEV Markdown support the same formatting options. In the case of nodes (like paragraphs, images, lists, etc.) we’re “filtering out” taskLists (as DEV doesn’t support them) and handling processing for embeds, using DEV’s liquid tags and embed URL available as a node attribute — src.

Now the Worker is ready to be deployed via the Wrangler CLI:

wrangler deploy

When deployed, you should get the URL for calling the Worker in your terminal. You can now use it to create a new Webhook in Vrite:

Go to Settings → Webhooks → New Webhook (all in the side panel)

For an event select New content piece added — this will trigger every time a new content piece is created directly within the given content group (in this case Published) or dragged and dropped into it.

Now you should be able to just drag and drop your ready content piece and see it be automatically published on DEV! 🎉

Next Steps

Now, there’s a lot you can do with Vrite even right now, that I haven’t covered in this article. Here are a few examples:

Only content pieces that are newly added to the content group and getting published/updated. You might want to consider “locking” this content group so that editing these content pieces requires you to first move the article back to the Drafts or Editing column. If necessary, you can set up dedicated Webhooks for those groups, so that content pieces are automatically unpublished on DEV.
Since the introduction of Workspaces, Vrite supports Teams and real-time collaboration like in e.g. Google Docs. This elevates it from a standard CMS to an actually good editor and allows you to speed up your content delivery with no need for manual copy-pasting. So feel free to invite other collaborators to join your workspace and control their access level through roles and permissions.
With Vrite’s support for various formatting options and content blocks - you might want to limit the available features to better fit your writing style - especially when you’re working in a team. Try adjusting your Editing Experience in the settings, including mentioned options and a Prettier config for code formatting.
Finally, as Vrite is an external CMS, you can freely connect it with any other content delivery frontend (like your personal blog or other platforms) and easily cross-post your content.

Bottom Line

Now, it’s worth remembering that Vrite is still in Beta. This means that not all features are implemented yet and you are likely to encounter bugs and other issues. But that’s just part of the process and I hope you’ll bear with me as we’re evolving the technical writing landscape!

🌟 Star Vrite on GitHub — https://github.com/vriteio/vrite
🐞 Report bugs — https://github.com/vriteio/vrite/issues
🐦 Follow on Twitter for latest updates — https://twitter.com/vriteio
💬 Join Vrite Discord — https://discord.gg/yYqDWyKnqE
ℹ️ Learn more about Vrite — https://vrite.io

Separation of concerns slows you down

Arek Nawo — Fri, 27 Jan 2023 20:52:21 +0000

Separation of concerns has long been regarded as a good design principle for organizing your codebase. Personally, in my quest for writing clean code, for a long time, I tried to adhere to this principle when creating frontend web apps, while also trying to keep my development experience at a consistently high level.

Fast forward to today and it’s been a while since I decided to no longer follow this principle. I’m still focused on writing clean code, I now value more development experience and code that’s easy to scale, maintain and evolve over time. I’ve come to a difficult and controversial conclusion - separation of concerns actually slows you down.

Separation of concerns in frontend web development

As a web developer, the most common and well-known example of separation of concerns is in the UI - HTML for markup, CSS for styles, and JS for interactivity. This “golden standard” of concern separation was heavily shaken by modern JS toolings such as UI frameworks and compilers.

React through JSX brought markup right into your JS code. This was and still is a big reason why many developers are against it.

const Button = (props) => {
  return (
    <button onClick={(event) => props.handleClick(event)}>
      {props.children}
    </button>
  );
};

You might argue that other frameworks, such as Vue, did the same thing, just in a different way. However, Vue’s Single File Components (SFCs) still adhere to this principle. While the code for all elements of the component resides in the same file, styling, markup, and JS logic are still separated into clear sections. It’s important to remember that separation of concerns doesn’t necessarily equal separation of code into separate files.

<script setup>
  import { ref } from 'vue'

  const content = ref('Hello World!')
</script>

<template>
  <button class="btn">{{ content }}</button>
</template>

<style>
  .button {
    background: #333;
    color: #fff;
    padding: 0.25rem 0.5rem;
    border-radius: 1rem;
  }
</style>

Now, a similar process has been happening with CSS. Thanks to CSS-in-JS and, more recently, utility CSS solutions like Tailwind CSS, the process of styling also moved into JS code. Combined with JSX, your entire component can now be a single file, where all elements of a component intertwine.

const Button = (props) => {
  return (
    <button
      className="bg-gray-200 hover:bg-gray-300 text-gray-800 px-2 py-1 rounded-lg"
      onClick={(event) => props.handleClick(event)}
    >
      {props.children}
    </button>
  );
};

Pros and cons of concern separation

Some might look at tools like JSX or Tailwind CSS and say it’s a highway to code spaghetti. They wouldn’t be wrong. Forgoing the separation of concerns completely and building giant-size components can quickly lead to code that’s hard to read and difficult to understand, building up technical debt over time.

However, just because it’s easy to do it wrong, doesn’t mean it can’t be done right. I’d say these modern tools force us to look at web development differently. To see components as complete building blocks, rather than just groups of styles, markup, and code.

When done right, I’d argue JSX and utility CSS (or CSS-in-JS) can speed up development, improve the developer experience and even lead to more organized code. You no longer have to switch “contexts” in your mind or files in your editor to go between markup or styles. You can see components in a new light, optimizing them to serve their intended purpose. But, over all else - you don’t have to name things as often.

If you’ve worked long enough as a software developer long enough you’ll know this job comes down to two things - solving problems and naming things - where the latter is the most difficult. Combining JSX with utility CSS means dramatically less naming required for CSS classes, element IDs, etc. This translates to even further increased developer productivity.

Different kind of separation of concerns

Now, forgoing “standard” separation of concerns doesn’t mean putting everything in a single file and calling it a day. I’ve already touched on why it’s the wrong path and how it can lead you to completely unmaintainable code. Instead, you should approach your code organization from a point of components and business logic.

Extract the smallest parts of your UI - especially ones that can be reused - and organize them correctly. I like to think it’s a more natural way to organize your app’s codebase. Thinking about components, different parts of your UI, their responsibilities, and functions seems more natural than artificially separating styles and markup just for the sake of it.

In addition to that, any complex logic that’s not directly related to the component (i.e. doesn’t change the state of the UI), such as API client, data processing, utility functions, etc., should also be separated whenever possible. In general, any business logic that doesn’t need to directly interact with the UI should be extracted and properly modularized.

Following these two, rather simple rules should lead you to a different kind of concern separation. One where concerns are separated based on their functions and responsibilities in relation to your codebase. One that should come to you more naturally and should improve your productivity rather than slow you down.

Conclusion

Now, the fact is, if you already have strong opinions or work with a framework that imposes a specific approach to the separation of concerns, this post likely won’t make you change your mind. However, if you obsess over clean code, and are trying many different approaches, frameworks, etc., then maybe this convinced you to try something different.

For the time being, this is how I approach web development on pretty much every project I have “architectural control” over. That’s how I worked with Solid.js and Tailwind CSS for the past 2 years. That’s how Vrite is being built. Has worked pretty well so far…

Solid.js, React, and Vue - reactivity systems compared

Arek Nawo — Thu, 26 Jan 2023 23:17:27 +0000

Working on many projects over the years, I’ve used multiple JavaScript UI frameworks. React, Vue, and now Solid.js are the frameworks that I’ve worked with and enjoyed the most.

Recently, especially when working with Solid.js, I started to notice and appreciate small details like how every framework’s reactivity system differs. It’s an important thing to keep in mind while jumping between frameworks as, even though the APIs keep getting more similar, how the framework works underneath influences performance, software architecture as well as how you think about your app in general.

The thing is understanding the framework’s reactivity system to the full extent takes time and requires deep knowledge of its architecture. That’s a lot - especially when working with multiple frameworks. That’s why I wanted to simplify these concepts and provide you a “good enough” starting point so that you both understand the major differences and have a solid entry point to exploring this topic deeper on your own…

Reactivity Systems

If you’ve only ever worked with or are laser-focused on a single framework you might not have thought about its reactivity system or rendering model too much - especially in comparison to other frameworks. How the component state is created and managed, what triggers a re-render, what parts of the UI are being updated, the inner workings of Virtual DOM, and how it all impacts performance - these are questions that you usually don’t consider when building UIs. However, sometimes it’s worth taking a step back and considering how these things impact your entire codebase.

React

In React, when using Functional Components with Hooks, your entire “component function” is what gets executed on every re-render. Take a look at this example:

import React, { useState, useEffect } from "react";

const Example = () => {
  const [count, setCount] = useState(0);
  const [someData, setSomeData] = useState(0);

  console.log("re-render");

  useEffect(() => {
    console.log(`You clicked ${count} times`);
  }, [count]);
  useEffect(() => {
    setInterval(() => {
      setSomeData(Math.random());
    }, 1000);
  }, []);

  return (
    <div>
      You clicked {count} times
      <button
        onClick={() => {
          setCount(count + 1);
          console.log(`Count not updated here yet: ${count}`);
        }}
      >
        Click me
      </button>
    </div>
  );
};

As the entire component function is re-run on every re-render, you use React Hooks to both define the state of the component (useState()), as well as to “filter out” pieces of code that should run only when specific state properties are updated (useEffect()). That’s the simplest way to think about the re-renders here - “everything runs by default, you use Hooks to filter out what should not”. That’s by design.

Few other things to note here:

Updating someData (in an interval) triggers re-renders even though the value isn’t used in the view. Now, due to the way React works, it’s required to keep the value in the component up-to-date, and, thanks to Virtual DOM, no DOM operation will be performed. However, it’s still a re-render that can be costly in complex components which you need to keep in mind. Changing someData to a ref using useRef() would be the solution here.
You can only be sure that count was updated, after a re-render, in the useEffect() callback. The console.log() right after setCount() will still display the old value. It’s not that problematic when you have other ways to access the new value or have adapted your mental model to the way React works. If you’re new to React, or you’re constantly switching between different frameworks, or work on complex components with complex event handlers - this might be an issue. If you want to both update the component as well as have the latest value available right away, you might have to combine useState() with useRef().
Things that aren’t as important but also stand out when compared to other frameworks: having to explicitly define effect dependencies (no auto-tracking) and no direct “on mount” lifecycle callback (useEffect(() => {}, []) serves as an alternative).

To battle, these and other “issues” React established “Rules of Hooks” and recommends splitting your code into smaller components. The more granular the component and its state, the lesser the potential overhead for every re-render.

Vue

Here’s an equivalent example for Vue 3 (note: even though it’s less popular in Vue, I’ve used JSX for a closer visual comparison with other frameworks):

import { defineComponent, ref, watchEffect, onMounted } from "vue";

const Example = defineComponent({
  setup() {
    const count = ref(0);
    const someData = ref(0);
    const onClick = () => {
      count.value = count.value + 1;
      console.log(`Count updated here already: ${count.value}`);
    };

    watchEffect(() => {
      console.log(`You clicked ${count.value} times`);
    });
    onMounted(() => {
      setInterval(() => {
        someData.value = Math.random();
      }, 1000);
    });

    return () => {
      console.log("re-render");

      return (
        <div>
          You clicked {count.value} times
          <button onClick={onClick}>Click me</button>
        </div>
      );
    };
  },
});

In Vue, every component has an entry setup() function. It’s here that you use Composition API to set up your component’s logic and finally, return a rendering function. You see a difference between what runs only once and what runs on every re-render right away. If you want a piece of code to run when certain state property changes, you have to use watchers to “filter them into” the update cycle, as opposed to “filtering out” in React.

On top of that, there are a few advantages to Vue’s reactivity system. First off, Vue does provide actual lifecycle hooks like onMounted(). On top of that, the setup() function itself serves as a great entry point for any logic that doesn’t require the component to be already rendered.

Secondly, thanks to reactive objects and refs based on JS proxies, Vue can automatically detect when a certain effect or re-render needs to be triggered. Thus, you don’t need to provide explicit dependencies in watchEffect() (though you can with the watch()), and setting someData ref in an interval won’t trigger a re-render, since it’s not used in the view.

Finally, when updating a ref, you can be sure that when you read the value again, it’ll already be changed. Keep in mind though that the re-render triggered by that change likely hasn’t happened yet and the UI isn’t up-to-date.

With all that and a lot of optimization on the Virtual DOM and other parts of the framework, Vue has much better performance when compared to React and arguably, a better development experience. The setup() function makes for a great starting point for your components, not having to worry as much about too many re-renders, while Composition API provides similar ergonomics as React Hooks.

Solid

To me, Solid often feels like the best of React and Vue combined. However, this line of thinking doesn’t give me a complete picture of the framework. Once you step beyond basic components, you quickly see that Solid is much different than other frameworks. When working with Solid, you’ll have to forget a lot of what you’ve learned about components and reactivity from other frameworks.

Consider the example below:

import { createSignal, createEffect, onMount } from "solid-js";

const Example = () => {
  const [count, setCount] = createSignal(0);
  const [someData, setSomeData] = createSignal(0);

  createEffect(() => {
    console.log(`You clicked ${count()} times`);
  });

  onMount(() => {
    setInterval(() => {
      setSomeData(Math.random());
    }, 1000);
  });

  return (
    <div>
      You clicked {count()} times
      <button
        onClick={() => {
          setCount(count() + 1);
          console.log(`Count updated here already: ${count()}`);
        }}
      >
        Click me
      </button>
    </div>
  );
};

In Solid.js components exist only to organize your code. That’s why a component function is run only once. It’s similar to a component function in React, with properties of the setup() function of Vue. However, once it’s run, the component vanishes, while all that’s left are the JSX elements, reactive primitives, and effects. That’s very different from e.g. Vue, where every component has an instance you can access. Still, Solid does provide some lifecycle hooks like onMount() but they’re more focused on reactive scope rather than the actual lifecycle of the component, like in Vue.

Another big part of Solid.js is its fine-grained reactivity system. While the API might seem similar to React Hooks, it’s completely different underneath. Solid’s reactivity is built on signals that are automatically tracked to appropriately trigger effects and UI updates. Thus, similarly to Vue, you don’t need to pass explicit dependencies to createEffect() (even though you can with on()), while updating data that’s not in the view (like someData), won’t trigger a UI update.

Solid is the only framework of the three not to have a Virtual DOM. Thanks to fine-grained reactivity, and compiler optimizations, Solid is able to quickly update just the right part of the UI - synchronously. This results in charts-topping performance and reassures you that your UI is always up-to-date. Thus, after calling the setCount() function you can be sure that both the value and the UI have already been updated.

Conclusion

As you can see, even though the frameworks seem similar on the outside, many underlying differences make or break performance and developer experience.

Personally, over the last few years, Solid has been my favorite. Thanks to its top performance, reactivity system, and API, it’s been a development experience like no other. That’s why it’s powering this blog, Vrite landing page, and soon - Vrite itself. If you’re interested, give Solid a look!