DEV Community: Maryam

Prose linting with Vale

Maryam — Mon, 06 Mar 2023 12:39:45 +0000

Writing is a time-consuming process. Depending on the writer and the review process, it can take a while to get content ready for publication. Whether you’re a solo writer or a big team, prose linters can help ensure a consistent tone and style.

Similar to code linters, prose linters automatically check your text for errors. Unlike grammar checkers, which highlight violations of grammatical rules, prose linters focus on how you can make your text better by addressing common usage problems like extra spaces, repeated words, excessive use of jargon, sexist language, and incorrect capitalization.

Prose linters can also provide a framework for creating and enforcing an editorial style guide. This helps with the review process as you can now focus on reviewing the content itself instead of pointing out typos and preferred usage patterns. This is particularly important when working in open-source projects like Meilisearch, where you have many contributors unfamiliar with your style guide.

What is Vale?

Vale is an open-source, highly customizable, syntax-aware prose linter. It supports documents written in many different formats such as Markdown, HTML, reStructuredText, AsciiDoc, DITA, and XML.

Vale isn’t your only option when it comes to prose linting. There are many other open-source tools available, including proselint, textlint, and alex.

At Meilisearch, we decided to go with Vale because it’s fast, easy to set up, flexible, and comes with existing rules to help you get started.

Where do I start?

Though it may seem daunting, setting up Vale can be fairly straightforward if you start small and keep things simple. In this post, we’ll go over how to use Vale in a project much like Meilisearch’s documentation.

Step 1: Style guide

The first step is to create a style guide. A style guide ensures a consistent tone and style regardless of how big your team gets. It establishes standard practices when people may have different opinions, such as whether or not to use the oxford comma.

If you don’t have an in-house style guide, you can check out Google’s or Microsoft’s to help you get started. Over time you will memorize most of the rules, but it’s possible to overlook mistakes and sometimes forget the rules altogether. We’re only human.

And that is where Vale comes in. It allows you to “codify” a style guide and checks your text against all the rules in that style guide. If it detects any issues, it displays suggestions, warnings, or errors on the console.

Step 2: Install Vale

I’m using macOS and ran brew install vale in my console to install Vale.

If you’re using a different operating system, check out Vale’s documentation for installation instructions.

Verify the installation by typing vale -v in your console. If this command returns Vale’s version number, the installation was successful.

Finally, create the following files and folders:

├── .vale.ini
│   ├──  styles
│   │       ├── Style guide
│   │       └── Vocab

Step 3: Configuring Vale - vale.ini

Create a vale.ini file at the root of your repository. This is the Vale configuration file where you define what you want Vale to do and what files to lint. Let’s start with a basic setup—you can always add to it later based on your project’s needs.

StylesPath = .vale/styles
MinAlertLevel = suggestion

Vocab = word_list

[*.md]
BasedOnStyles = Meilisearch

StylesPath is where Vale looks for your style guide (more on that in the next step). The path can be relative or absolute to the location of the vale.ini file.
MinAlertLevel specifies the minimum alert level that Vale will report. By default, it’s set to warning. The other options are error and suggestion. An error indicates you did something wrong, like using extra spaces or making a typo. A warning isn’t as severe as an error, but indicates something you should avoid, like making sure your sentences don’t become too long. A suggestion is a recommendation to do something that is usually—but not always—a good idea, like breaking your sentence into two instead of using a semicolon. If a rule is set to suggestion, you will see suggestions, warnings, and errors. If it is set to warning, Vale will only show errors and warnings, no suggestions.
Vocab: This is where you create a directory containing the accept.txt and reject.txt files. Both files accept words, phrases, and regular expressions. If your text contains words that do not exist in the dictionary (e.g. “Meilisearch”), you can add them to accept.txt and Vale won’t angrily scream at you for making a “typo”. Vale will flag all occurrences listed in reject.txt as errors. This can be useful when you want writers to avoid a specific word—if you are writing about search engines and databases, for example, using “indexation” can be confusing.
[*.md] tells Vale to only lint markdown files. If you want to lint plain text files, use [*.txt].
BasedOnStyles specifies the style guide Vale should use for linting.

You can specify other settings, including what tokens and HTML tags to ignore and what Vale should consider an individual word. You can read more about the vale. ini file in Vale’s documentation.

Step 4: Rules and the styles folder

As I mentioned earlier, you need a style guide to use Vale. You then convert this style guide into something Vale can understand: rules.

Rules use different extension points to perform specific tasks. For example, the existence extension point looks for the existence of a particular token, repetition looks for repeated tokens, spelling implements spell checking, and so on. In Vale, each rule is a YAML file. The styles folder contains the individual rules that make up the style guide.

If you don’t want to create your own style guide or need a starting point to build from, Vale comes with ready-to-use style guides that you can apply to your docs and start linting. Here are some style guides that helped us get started:

You can find more on Vale’s GitHub repository.

Let’s start with a rule for sentence length. Your style would say something like, “Ensure sentences don’t exceed 40 words”. This is what the rule looks like as a YAML file:

# Warning: Meilisearch.SentenceLength

# Counts words in a sentence and alerts if a sentence exceeds 40 words.

extends: occurrence
message: 'Shorter sentences improve readability (max 40 words).'
scope: sentence
link: https://docs.gitlab.com/ee/development/documentation/styleguide/index.html#language
level: warning
max: 40
token: \b(\w+)\b

This rule counts the words in a sentence and throws a warning if it exceeds 40 words. The scope is set to sentence: this ensures that Vale does not apply this rule to other parts of the text, like headings or tables.

The level is set to warning. Written text is complicated, and Vale will find false positives. There are no sure-fire ways of deciding when a rule should be a suggestion, warning, or an error. You will need to make decisions and learn as you go.

I suggest reviewing your rules over time to update and, in some cases, delete outdated rules. Initially, the Meilisearch docs didn’t have a rule on sentence length. When we added it, the maximum length of a sentence was 45. Now it’s 40, and we plan on bringing it down to 35.

You can also enable or disable specific rules within a style guide by adding them to vale.ini:

Meilisearch.Headings = NO
Meilisearch.Spelling = NO
Meilisearch.Semicolons = NO

The above lines disable the Heading, Spelling, and Semicolons rules from the Meilisearch style guide.

Step 5: Run Vale

Now, when you use the following command on your console to lint your whole project:

vale .

Vale will check all your files against the rules stored in BasedOnStyles. If Vale detects any issues, it will display suggestions, warnings, and errors on the console.

You can also lint individual files using:

vale {file_path}

Step 6: Automate Vale checks

All the checks we’ve discussed so far are for your local files. Once you’re confident the rules work as intended, you can automate these checks using the Vale GitHub action! At the Meilisearch documentation repository, we configured it to run for every pull request. As mentioned before, Vale may find false positives. Since you don’t want a PR blocked because Vale isn’t working as it should, I recommend starting with a few rules, and slowly tweaking them to avoid failing checks.

Conclusion

That is all, folks! I hope I was able to help you get started with Vale (and a style guide). This was a quick overview to introduce you to Vale’s features. Tweaking it to your needs takes time and many, many iterations.

Once configured, Vale can automate parts of the review process and allow you to focus on the parts of a text that computers are not very good at. At least not yet.

Oh, and if you’re curious, check out our style guide on GitHub to see how we use Vale!

Navigating search results: pagination vs infinite scroll vs load more

Maryam — Thu, 15 Dec 2022 13:00:01 +0000

Meilisearch v0.30 is out and brings with it the ability to create numbered page selectors. You can read more about the new features and updates from v0.30 in our release blog post.

This article will go over the different ways of navigating search results and the pros and cons of each method.

What is pagination, and what are some common alternatives?

Let’s say you go to an ecommerce website to buy a new pair of shoes. It would be quite overwhelming if you got 700 pairs of shoes to choose from as soon as the page loads—not to mention that fetching every pair of shoes would certainly slow down the site.

To avoid this, most websites will break search results up into chunks, such as pages, and load a certain number at a time based on input from the user.

There are three main ways to display large datasets: pagination, infinite scroll, and load more. Each method has benefits and downsides that make it suited for certain use cases and content types. For example, Google’s use of numbered pagination makes it easier to find a specific result again, while Twitter, Facebook, and Instagram use infinite scroll to keep users glued to their screens. Google uses load more on mobile devices to display search results.

Changing from one type of pagination to another can completely alter user experience. Imagine waking up to a paginated version of Twitter, where you have to continuously click a “Next” button to view more posts in your feed (thanks, Elon). This would completely change the way you interact with the site!

In making the choice of how to display search results, you decide how much content to show users at one time, what action they have to take to view more, and how convenient it is to find a specific result again.

Pagination

Pagination works by breaking search results down into separate pages. There are two main types of pagination interfaces: numbered ones, where you find a list of numbered pages at the bottom of each page, and unnumbered ones, where you navigate from page to page using the “Previous” and “Next” buttons.

Pagination is one of the most popular ways to display search results and is particularly common on ecommerce websites. Pagination breaks down content, making it easier to process. Since each page is distinct, users can easily return to items they liked or share them with others. This helps users to search for something specific instead of browsing through a never-ending list of items.

The biggest downside to pagination is that most users don’t look beyond the first few pages and hence never get to look at more results. When was the last time you went to the second page of Google’s search results?

Since the results are replaced with every new page, users need to switch between pages to compare results. In addition, pagination isn’t very mobile-friendly. Mobile users prefer load more or infinite scroll as it’s more intuitive. It also requires additional actions from the user: clicking the “Next” button and waiting for the page to load. This may also give the impression that pagination is slow.

Numbered page selection

Page numbers add clarity to the search process and guide users to what they’re searching for. Numbered page selection offers numerous advantages, such as:

Users can be shown the total number of pages of results, allowing them to estimate how long they will need to find a particular item. (“I have 25 pages worth of shoes to sift through!”)
With numbered pages, it’s clear that some pages are more relevant than others. (“I didn’t find those shoes on the first five pages, so this website probably doesn’t sell them.”)
Users can remember the positioning of an item, and numbered pages allow them to return to it faster. (“I liked the pink shoes on page 3.”)
Users can generally stop in the middle of a session and continue from where they left off. (“I stopped at page 5 yesterday.”)

“Previous” and “Next” buttons

In numberless pagination, users navigate via “Next” and “Previous” buttons instead of a list of numbered pages. This alternative to numbered pagination is often used for mobile displays where there isn’t enough space to display page numbers. Since there are only two buttons, it offers less precision, as users can’t jump to specific pages. However, it offers better performance as it isn’t necessary to calculate the total number of search results.

Infinite scroll

Infinite scroll has become increasingly popular over the past decade due to the rise in smartphones. It loads more content seamlessly as the user scrolls, reducing friction. It’s also more intuitive than pagination; you just keep scrolling to view more content! Additionally, since all the content is loaded to a single page, it may seem faster than traditional pagination.

The biggest downside to infinite scroll is poor usability. The page length increases infinitely, making it difficult to find previously viewed results again, nor can users bookmark or share a particular set of items on the list. Page performance slows down as more content is loaded, making the page “heavier”. Meanwhile, the site footer may be inaccessible since infinite scroll pushes it endlessly down as more content is loaded. Since Googlebot cannot emulate scrolling, infinite scroll also has a negative impact on SEO.

However, the greatest impact by far is on user behavior. At some point, users of infinite scroll will stop paying attention to individual results and begin to focus more on scrolling, leading to decreased engagement. You can read more about this in Baymard Institute's study on design patterns for loading products on desktops and mobile devices.

Based on the above, infinite scroll may not be the best solution for ecommerce websites. How do you find the pink shoes you liked 15-ish scrolls ago?

There is one use case where infinite scroll cannot be beat. For entertainment websites whose main purpose is to keep the users with no particular aim on the page for as long as possible, infinite scroll all but guarantees a longer time on site. This is why infinite scroll is closely linked with social media and other business models that rely on advertising or data harvesting.

Load more

Like infinite scroll, load more is a mobile-friendly option that works by extending one page infinitely. However, unlike infinite scroll, load more allows users to choose whether they want to view more results by clicking a “Load more” button at the bottom of the page. This critical difference gives users more control, making it easier, for example, to access the footer or keep track of a specific result. Studies show that it also increases the amount of attention users give to each individual item.

Load more shares many downsides with infinite scroll, such as poor page performance. Like infinite scroll, it’s not ideal for SEO as Googlebot cannot click the “Load more” button. This means it cannot crawl or index all of your content.

So, what do I use?

The short answer: it depends on the kind of product experience you want.

Pagination

Your users are looking for highly specific results
SEO is very important to you

Infinite scroll

You’re building a social media site
Your users want to explore a large amount of content

Load more

You want to hide your content behind a paywall
You want to build an ethical mobile experience
You want to gain some of the benefits of both pagination and infinite scroll

Or you could go hybrid! Consider Ikea: they use a load more button but also display the total number of search results and a progress bar, enabling users to see how many results they’ve viewed so far.

While we have covered the three main ways to display search results today, they are far from set in stone. At the end of the day, your goal is to help your users find the content they need. If you understand which key goals you want to accomplish with your search navigation interface and which pitfalls you want to avoid, you may be able to design something even better than what’s been done before. Good luck, and let us know what you create with Meilisearch!