DEV Community

Cover image for NER With Transformers and spaCy (Python)

NER With Transformers and spaCy (Python)

James Briggs
418 bio is a teapot
・1 min read

Named entity recognition (NER) consists of extracting 'entities' from text - what we mean by that is given the sentence:

"Apple reached an all-time high stock price of 143 dollars this January."

We might want to extract the key pieces of information - or 'entities' - and categorize each of those entities. Like so:

  • Apple  : Organization
  • 143 dollars :  Monetary Value
  • this January :  Date

For us humans, this is easy. But how can we teach a machine to distinguish between a granny smith apple and the Apple we trade on NASDAQ?

(No, we can't rely on the 'A' being capitalized…)

This is where NER comes in - using NER, we can extract keywords like apple and identify that it is, in fact, an organization - not a fruit.

The go-to library for NER is spaCy, which is incredible. But what if we added transformers to spaCy? Even better - we'll cover exactly that in this video.

Discussion (0)

Forem Open with the Forem app