DEV Community

Cover image for Regex: From beginner to BEST on your team in one blog 👑
Luke Garrigan
Luke Garrigan

Posted on

Regex: From beginner to BEST on your team in one blog 👑

Recently I've been doing quite a lot of work using Regex and to be honest, I was pretty shite at it. It's one of those things that has been on my to-learn for years, but I've never really got round to it, because - like most people - I google a regex problem and see how other people have solved it rather than doing it myself.

What is regex

So if you don't know what regex is, don't worry this blog should be suitable for devs at all levels! Regex is short for Regular Expression which is a sequence of characters that specifies a search pattern. We use it reasonably frequently in programming to search for patterns in strings either to pull the information out or to make changes to it.

Let's get started

In this blog I recommend you open up https://regexr.com/ which is an online tool to evaluate regex, allowing you to put in test strings to see if your regex is working. Follow along with me making sure to use your own test strings to really solidify your understanding.

Slashes

The first thing you'll see is the slashes, one at the start and one at the end, they simply indicate when a regex starts and when a regex ends.

Flags

After the end slash you may have noticed there is usually a letter at the end. These are called flags, there are 6 flags:

  • g - global search, meaning it'll match all occurrences, this is the most common and will be used for the rest of the blog
  • i - ignore case, meaning case-insensitive
  • m - multiline mode, which will make more sense when we cover the ^ and $ below.

u, y, s are the others but won't be covered in this blog as I I've never really seen them used so I wouldn't consider them a learning priority. If you are interested, you can learn more here.

Exact matching

To start off with let's begin with exact character matching, let's say we want to match on the word Panic, all we have to do is write the thing.

/Panic/g
Enter fullscreen mode Exit fullscreen mode

Which is probably what you want most of the time, and if it is, you likely don't need regex. But, anything a little more involved will require quantifiers, which we'll get onto.

Quantifiers

The +

The plus character + means match one or more of the preceding token. So in our example below, match the letter l one or more times.

/l+/g
Enter fullscreen mode Exit fullscreen mode

This image has an empty alt attribute; its file name is image-2.png

The ?

The question mark ? is the optional symbol, it means anything that is before it is optional. So in our example, we are searching for i and f if it exists, but if f doesn't exist don't worry yourself, we'll take the i. So the optional-ness is just the first token that precedes the ?, which is f.

/if?/g
Enter fullscreen mode Exit fullscreen mode

The *

The * is basically a combination of the + and the ?. It matches 0 or more of the preceding token. So it's optional and if there are multiple of the preceding token we'll take that too. So in our example, we will match on anything that has an e and zero or many ls.

/el*/g
Enter fullscreen mode Exit fullscreen mode

This image has an empty alt attribute; its file name is image-4.png

The .

The dot matches any characters except line breaks. In the example below, we are looking for any character that has a space after it.

/. /g 
Enter fullscreen mode Exit fullscreen mode

This image has an empty alt attribute; its file name is image-5.png

The \w

So you might look at this and think oh god this is why regexes are so complicated, what's the bloody slash for? Well, if we just put in 'w' then it'd search for the 'w' letter as seen above, so this allows us to search just for a word's character. So any alphanumeric or underscore.

Our example matches on any word character then the letter s:

/\ws/g
Enter fullscreen mode Exit fullscreen mode

This image has an empty alt attribute; its file name is image-6.png

The \s

The \s quantifier is very simple, it just matches on any whitespace character (tabs, spaces, line breaks):

/\s/g
Enter fullscreen mode Exit fullscreen mode

This image has an empty alt attribute; its file name is image-7.png

The \S

Matches anything that isn't whitespace:
This image has an empty alt attribute; its file name is image-8.png
This is true for other quantifiers too, so when they're capitalised it means the opposite. So for \d that matches on digits there is \D that matches on anything that isn't a digit, and \W matches on anything that isn't a word character.

The |

The pipe means or, so search for this or that. So if you want to search for some literal text let's say "happy" but we also want to grab all the occurrences of the word "day" then we could just put those either side of a pipe and we'll match on both!

/happy|day/g
Enter fullscreen mode Exit fullscreen mode

This image has an empty alt attribute; its file name is image-9.png

The ^

This symbol denotes the start of the string, or, if the multiline tag is enabled then it denotes the start of the line.

Example without the multiline tag:

/^the/g
Enter fullscreen mode Exit fullscreen mode

This image has an empty alt attribute; its file name is image-10.png
Notice how it doesn't grab the "the" before Galaxy as it is not at the start of the string, and also how it doesn't grab the "the" on the second line, because that's also not the start of the string. However, if we enable multiline:

/^the/gm
Enter fullscreen mode Exit fullscreen mode

This image has an empty alt attribute; its file name is image-11.png
Then a match is made at the start of both lines.

The $

This denotes the end of the string, so with multiline it'd mean the end of the line, in the example below I'm not using multiline:

/Thursday$/g
Enter fullscreen mode Exit fullscreen mode

This image has an empty alt attribute; its file name is image-12.png
Notice how it doesn't match on the other Thursday as that's not at the end of the string, if you got rid of the $ then it certainly would match!

Length matching

Specifying a length in regex is pretty easy, all you have to do is choose your desired token and then follow it with some curly braces and pop a number in it. Below means give us any word characters that are 8 in length:

/\w{8}/g
Enter fullscreen mode Exit fullscreen mode

This image has an empty alt attribute; its file name is image-13.png
You can also specify a min and max range:

/\w{4,6}/g
Enter fullscreen mode Exit fullscreen mode

This image has an empty alt attribute; its file name is image-14.png
So any word character that is at least 4 characters long and at most 6.

Character sets

Character sets allow you to plonk stuff in square brackets which basically says match on anything that is within these square brackets. So the following means match any lowercase vowels:

/[aeiou]/g
Enter fullscreen mode Exit fullscreen mode

This image has an empty alt attribute; its file name is image-15.png
And this means match any vowels that are followed by the letter r:

/[aeiou]r/g
Enter fullscreen mode Exit fullscreen mode

This image has an empty alt attribute; its file name is image-16.pngYou can also add ranges in character sets, the following means match on any lowercase characters in the alphabet:

/[a-z]/g
Enter fullscreen mode Exit fullscreen mode

This image has an empty alt attribute; its file name is image-17.png
We want to grab the capital letters too, let's add that range as well.

/[a-zA-Z]/g
Enter fullscreen mode Exit fullscreen mode

This image has an empty alt attribute; its file name is image-18.png
Another really common range you might see in regexes is [0-9]

Groups

Putting parentheses around part of your regex allows you to group it. This basically means that you can apply a quantifier to the entire group rather than just the preceding token.
The following means to find words that contain "the" with an upper or lowercase t.

/(t|T)he/g
Enter fullscreen mode Exit fullscreen mode

This image has an empty alt attribute; its file name is image-19.png
Or the following example, which is looking for the letter p twice in the group and then an e after, but then notice the question mark after the group which denotes the preceding token is optional, so everything within the group is optional. Meaning e is chosen in cases where there's no preceding pp.

/(pp)?e/g
Enter fullscreen mode Exit fullscreen mode

This image has an empty alt attribute; its file name is image-20.png

Referencing Groups

In some scenarios, you may want to split your regex up into groups so that you can later reference them individually. Let's take the following example:

/(\d)\. (.+)/g
Enter fullscreen mode Exit fullscreen mode

This regex, first of all, looks for a number which is our first group, then a . (notice we had to escape the . else it would match on any character as shown previously). The second group (.+) is looking for any character except line breaks 1 or more.
This image has an empty alt attribute; its file name is image-21.png
Using the replace tools on https://regexr.com/ you can easily reference your groups. To reference the first group type in $1:
This image has an empty alt attribute; its file name is image-22.png
And then to reference the second group type in $2
This image has an empty alt attribute; its file name is image-23.png

Non-capturing groups

Let's say you want to capture the name of the book but don't really care about the number. So when you reference the group by using $1 it'll just return you the name. To do this we can create non-capturing groups, so within your brackets you can just plonk a ?: at the start to say, I don't want to include this in the result.

/(?:\d)\. (.+)/g
Enter fullscreen mode Exit fullscreen mode

This image has an empty alt attribute; its file name is image-25.png
It looks the same right? But not when outputting the groups:
This image has an empty alt attribute; its file name is image-26.png
Notice how $2 is now black and is treated as though you want to append $2 as a string on the end because it's not defined.

Lookaheads/ lookbehinds

Okay, so you've learned pretty much everything you'll need to know about regex in 95% of scenarios, let's learn about lookaheads and lookbehinds.

Positive Lookbehind ?<=

Matches a group before an expression without including it in the result.
Carrying on from the previous example let's take the same input and use positive lookbehind to find all occurrences of the letter h that are preceded by the letter t.

/(?<=t)h/g
Enter fullscreen mode Exit fullscreen mode

This image has an empty alt attribute; its file name is image-27.png
So positive lookbehind is great for looking for something that precedes a pattern but you don't really care about having that pattern in the result.

Negative Lookbehind ?<!

Specifies a group that cannot match before the main expression, so this is basically the inverse of positive lookbehind. All we do is swap the = for a !
Let's do that for the example we used in positive lookbehind:

/(?<!t)h/g
Enter fullscreen mode Exit fullscreen mode

This image has an empty alt attribute; its file name is image-28.png
So this finds all occurrences of the letter h that are not preceded by the letter t.

Positive Lookahead ?=

Matches a group after an expression without including it in the result. So exactly the same as positive lookbehind but from the other end of the string.
In the example below I'm looking for any character that has an a following it:

/.(?=a)/g
Enter fullscreen mode Exit fullscreen mode

This image has an empty alt attribute; its file name is image-29.png

Negative Lookahead

Is just the inverse of positive lookahead. Specifies a group that cannot match after the main expression.
So the following looks for any character that is not followed by the letter a:

/.(?!a)/g
Enter fullscreen mode Exit fullscreen mode

This image has an empty alt attribute; its file name is image-30.png

Let's tackle a problem together

A real common regex problem is phone numbers. Copy the following text into RegExr.

L. Garrigan: 248-555-1234  
J. Bezos: 232 123 2312  
E. Musk: (313) 555-1234  
B. Gates: (810)555-1234
Enter fullscreen mode Exit fullscreen mode

Goal: get the numbers for each person and output in the format xxx-xxx-xxxx

Let's pick the problem apart. Let's search for numbers that are in the correct format, so L.Garrigan's number is in the correct format, let's grab that.

/\d{3}-\d{3}-\d{4}/g
Enter fullscreen mode Exit fullscreen mode

So we're looking for 3 digits followed by a - then another 3 digits followed by a - then 4 digits
This image has an empty alt attribute; its file name is image-31.png
Next let's try grab the phone number with spaces rather than dashes.

/\d{3}-?\s?\d{3}-?\s?\d{4}/g
Enter fullscreen mode Exit fullscreen mode

So I've made the - optional and also added an optional space denoted by \s?
This image has an empty alt attribute; its file name is image-32.png
I'm pretty sure this can be improved though using character sets rather than two optionals:

/\d{3}[- ]\d{3}[- ]\d{4}/g
Enter fullscreen mode Exit fullscreen mode

I decided to go for [- ] which means match either - or a space, I chose the space instead because \s also matches on tabs and line breaks which I don't need.
Let's go for the third row now with the first numbers wrapped in brackets:

/\(?\d{3}\)?[-\s]\d{3}[- ]\d{4}/g
Enter fullscreen mode Exit fullscreen mode

I've just added an optional bracket ( at the start - notice it's escaped else it'd think we were creating a group and then an optional closing bracket )? after the first 3 digits.

This image has an empty alt attribute; its file name is image-33.png
Next, let's go for Gates, who helpfully decided to not put a space after his parenthesis.

/\(?\d{3}\)?[-\s]?\d{3}[- ]\d{4}/g
Enter fullscreen mode Exit fullscreen mode

So I just make the first character set optional which allows us to grab those.
This image has an empty alt attribute; its file name is image-34.png
We need to group these so we can output all the information in the desired format. So I'm going to group the first 3 numbers, then the second 3, then the last 4.
So it is just a case of wrapping parenthesis around the digit captures:

/\(?(\d{3})\)?[-\s]?(\d{3})[- ](\d{4})/g
Enter fullscreen mode Exit fullscreen mode

And then output should look like the following:
This image has an empty alt attribute; its file name is image-35.png
Wonderful, all in that lovely format we wanted.
Note there's so many edge cases here, I'd recommend checking out this article which covers the problem in more detail.

Tackle some problems!

The only real way to get decent at anything is by practicing, so to really solidify what you've learnt in this blog have a crack at some problems on here it's a really good website that'll take you through some regex problems dependent upon your ability. It takes into consideration the quantifiers you know and don't know and designs tests accordingly.

I hope you've enjoyed this blog, be sure to check out my personal blogging site at codeheir.com for some extra free content!

Top comments (3)

Collapse
 
maryslarson1 profile image
maryslarson1

Regex is often used in web scraping and the best thing it's relatively easy to learn, so why not to give it a try? Thanks for the useful article! I want to automate my extraction for achieveessays.com/ data.

Collapse
 
deven96 profile image
Diretnan Domnan

Found this very helpful. Thanks!

Collapse
 
lukegarrigan profile image
Luke Garrigan

Glad it helped!