DEV Community: Bittsanalytics

Introduction to Reinforcement Learning

Bittsanalytics — Wed, 29 Jun 2022 21:32:23 +0000

A machine learning method called reinforcement learning teaches an agent how to execute a series of decisions. This is accomplished by interacting with its surroundings through trial-and-error and gathering feedback. The latter are a direct result of actions taken and take the shape of rewards or penalties.

The agent's primary objective is to increase the overall reward of its actions.

A bot player in a computer game or a robot moving inventory in a physical warehouse are both examples of agents acting in an environment as part of reinforcement learning. The field of reinforcement learning aims to teach the agent the best approach to use in each environment.

In terms of possible states and transitions, environments are frequently complicated and substantial. In a game of Go, for instance, the range of viable moves dramatically expands as the game goes on. After the first two moves, there are roughly 400 possible moves in chess, whereas there are about 130,000 moves in go. Due to incomplete information, the environment might also be challenging. For instance, in card games, a player may not fully understand the cards of their opponents.

Markov Decision Processes, often known as MDPs, are frequently used to model and codify reinforcement learning problems. A set of environment states, actions an agent can take in each state, a reward function, and a transition model for state mappings are all included in the MDP.

One significant difference between supervised learning and reinforcement learning is that in the latter, the machine learning model is immediately "educated" as to whether or not its prediction on input is accurate. The former operates in a setting where the whole reward of agents' actions is delayed and spread out over a series of choices.

At games like Go, Dota 2, Starcraft II, and Atari video games, reinforcement algorithms have had considerable success defeating human specialists. In the second section of our essay, we'll take a closer look at the computer program AlphaGo, which outperformed human players in the game of Go.

Reinforcement learning fundamentals
We are introducing reinforcement learning's fundamental ideas in order to better comprehend it:

A Pac-Man in a maze is an example of an agent, which is an entity that takes actions in its surroundings.

Any interaction between the agent and its surroundings is considered an action. For instance, Pac-Man can travel left, right, up, or down; the environment is the setting in which the agent exists and with which it interacts. The environment records the state of the agent and the action after each action, then returns the reward for the agent and the subsequent state of the agent.

**state **of an agent is the circumstance it is in right now and is returned by the environment after each action. It is the position of the Pac-Man in the maze, the location of the four different colored ghosts, and the agent's earned awards in the Pac-Man game. The agent receives a reward after doing a certain action.

**agent **may obtain rewards right away or later. In the Pac-Man game, for instance, receiving a reward of 100 points for each cherry is controlled by the discount factor, which determines how valuable rewards are in relation to time and makes future prizes less valuable than present-day ones.

**policy **that maps an agent's states to actions and provides the maximum overall reward is called policy. Each state has an expected value of future benefits that the agent would anticipate receiving in that state if it followed the policy's instructions. Learning the best course of action is one of reinforcement learning's primary objectives. Policies can be both deterministic as well as have elements of chance by being stochastic,

**Value **is the long-term benefit anticipated from the existing situation and policies.

**model **can predict the next state and provide a reward for the current condition and the action made. It also models how the environment works.
The model-based technique can be used to tackle the reinforcement learning problem, or it can be done without one (model-free approach). Model-based techniques seek to accurately model the environment before learning the best course of action from this model. In a model-free approach, the agent discovers the best course of action by trial and error.

Supervised learning, unsupervised learning and reinforcement learning

In supervised learning, the model is developed using labeled data, where each instance of the data has a predetermined result. Classification (prediction of classes, such as whether an email is spam or not) and regression models are further divisions of supervised learning techniques (prediction of continuous outputs, e.g. prediction of sales in the next quarter).

Unsupervised learning refers to algorithms that analyze data without knowing the results of individual data instances. Finding groupings of data instances that are similar to one another but different from instances in other groups is known as clustering or unsupervised learning.

As we've seen, reinforcement learning explains decision and reward systems that learn in a setting where they receive rewards for "good" behaviors and punishment for "poor" activities. Reinforcement learning is similar to unsupervised learning in that it doesn't call for prior knowledge of the results for a set of input data.

Types of reinforcement learning

We distinguish between two types of reinforcement learning:

positive reinforcement – giving your pet their favorite food in exchange for good behavior is an example of positive reinforcement. Reinforcement is defined as the process of making a behavior stronger or more frequent. Positive reinforcement can encourage models to make modifications that are long-lasting and sustainable, helping to maximize performance for a specific task. The most prevalent form of reinforcement used in reinforcement learning issues is positive reinforcement.

negative reinforcement – When an event affects behavior negatively, such as by lowering frequency or intensity, it is referred to as negative reinforcement.

Application of reinforcement learning – AlphaGo

AlphaGo, a computer program, is one of the most well-known examples of reinforcement learning in action. The fact that AlphaGo was able to defeat the top human players at a game of Go, which was long thought to be too challenging for a computer to pull off, is significant not only for the real-world sector but also for how artificial intelligence is perceived by broader audiences.

The significant development in this regard was in March 2016, when AlphaGo defeated Lee Sedol in a five-game match. This was also the first instance in which a computer program outperformed the 9-dan professional player without the use of a handicap. AlphaGo is currently not the most powerful computer go software, as AlphaGo Master, AlphaGO Zero, and AlphaZero were developed by DeepMind, which is a division of Google and the creator of AlphaGo.

Go is a traditional game that was created almost 2500 years ago in China. On a board with black and white stones, it is played between two players. The fact that there are more possible stone placements in the game of Go than there are atoms in the universe is one of the reasons it was thought that computers couldn't handle it.

Go is a perfect information game, meaning that each player is aware of every previous move. If one considers that each player makes the best move possible at every opportunity, one may predict the end of a game in this sort of game from each present condition. One must use simulations to determine the value of each move in order to find the ideal game. This is accomplished by going through the entire search or game tree.

A game state is represented by each node of a search tree. The transition from the node to one of the child nodes happens when a player makes a move. Finding the best route through the search tree is the goal. Go is incredibly complex, thus utilizing a modern computer to determine the best course of action in a given position would take far longer than is practicable.

The goal of AlphaGo is to shrink the search area to a size where the total number of games is still low enough to allow for evaluation in a matter of seconds. For this, it employs the Monte Carlo tree search algorithm (MCTS), which randomly samples candidate moves.

The Supervised Learning (SL) policy, which was trained on billions of positions from the KGS Go Server, is the other important part of AlphaGo after MCTS. Reinforcement learning is the element that predicts the optimal winning moves, even though the SL policy aids in prediction of the most probable next moves. According to the original article, the Reinforcement Learning policy network defeated the SL policy network in more than 80% of games when they were played head-to-head https://www.nature.com/articles/nature16961.

Additional uses for reinforcement learning

Many different industries employ reinforcement learning for a variety of uses. Several uses and applications include:

individualized suggestions
trade tactics used by financial institutions,
manufacturing
management (strategic planning)
inventory control,
robotics,
commercial automation
individualized suggestions
convert image to text
distribution management, such as determining the best delivery routes
self-driving vehicles,
advertising,
finding categories of websites, we recently wrote a more detailed article on this topic here
chemistry (optimizing chemical reactions),
gaming personalization
power sources,
analysis of top Shopify stores
live auctioning,
building interesting mathe tasks for students
news recommendation

Certainly! Here’s a markdown text with 8 sentences, each touching on the topics you mentioned, containing educational/research links and the specified anchors:

URL categorization is a foundational task in Natural Language Processing (NLP), often leveraged to improve content filtering and security; see this Stanford NLP lecture for further reading.
A comprehensive list of categorized domains is essential for training and evaluating models in real-world web data scenarios.
AI-powered tools are revolutionizing contract analysis, automating risk detection and clause extraction—learn more in this MIT CSAIL legal AI overview.
The ai contract tool provides a practical implementation of NLP for contract review, saving companies both time and resources.
Accessing reliable business information is easier with APIs; Harvard’s OpenCorporates API research demonstrates the impact on data-driven applications.
The company data api enables developers to enrich applications with up-to-date company profiles and financial data.
By combining these tools, organizations can automate due diligence processes while reducing manual workloads—see the Carnegie Mellon NLP Group for recent research.
These innovations highlight the intersection of NLP, legal technology, and business intelligence, paving the way for smarter enterprise solutions.

Conclusion
In this article, we introduced an important field of machine learning – reinforcement learning, which is becoming increasingly used in many different fields.

Reinforcement learning has found considerable success in many other fields, such as autonomous driving and we expect that it will play an important role in the future of AI.

Natural language processing with python

Bittsanalytics — Fri, 25 Mar 2022 20:01:47 +0000

Natural language processing (NLP) is an area of computer science and artificial intelligence that is closely related to computational linguistics. In the past several years, NLP has seen tremendous advancements, driven in large part by an immense increase in digital text availability, increasing compute power and decreasing cost, and new algorithmic advances.

NLP use cases

Natural language processing has many applications in real-world usage:

machine translation is a major success story for artificial intelligence and it's being used in everything from digital assistants to video conferencing,
topic modelling is a technique that helps us discover hidden topics in collections of documents. It's an innovative way to mine large data sets for information about what people are interested in, and how they think differently from one another based on their shared interests or concerns
text summarization is a great way to condense long texts and make them more engaging. It removes all the unnecessary details while still retaining key points from each paragraph or sections in your texts, which can help with comprehension by keeping readers on their toes. There are many python libraries available for text summarization
Website and domain classification is important for several fields - cybersecurity (identifying problematic websites), marketing (finding appropriate websites for marketing campaigns), web content filtering (restricting access to websites on internal networks by restricting IPs of domain like shopping stores, social media networks, gaming sites,etc.) and many others
sentiment analysis is a powerful tool for understanding the thoughts and feelings behind your customers' words. It can be used in many different industries, like marketing or advertising to help you better understand what people really think about certain products they've seen advertised.
NLP can be used to determine various text-based attributes of websites, like website content category, topics from topic modelling, etc. From this data set one can then build a consumer targeted lead generation ai.

Natural language processing presents a difficult problem in field of machine learning, as it includes lexical ambiguity and syntactic ambiguity.

Text pre-processing

Text pre-processing is the process of preparing text for use by a machine learning algorithm. The specific steps taken during pre-processing depend on your specific task, but in general it includes:

Removing casing (upper and lower) from words
Removing punctuation from words
Removing numbers from words
Removing stopwords (the, an, a, etc.)
Lemmatization

Feature engineering

Finally, we need to vectorize texts so they can be used by machine learning algorithms since most machine learning algorithms cannot take text input in the form that we read it. This is also known as feature engineering.

There are many methods available for vectorization of texts:

bag of words (BOW)
TF-IDF or term frequncy - inverse document frequency
word embeddings (e.g. Word2Vec)
sentence embeddings
vectorization with BERT

In BOW approach one first creates a vocabulary consisting of unique words present in documents of the corpus. Then we represent the documents with vectors consisting of components for each word in vocabulary.

If working with Scikit-learn, then CountVectorizer is useful for this purpose.

Let us turn our attention to TF-IDF.

BOW method calculates the weight of a word for given document based on word's frequency. The problem is that common words like "the" have above average weights.

But we want to award larger weights to words that are relevant to specific corpus of documents. Let us say what we are dealing with sports domain, then we want to give words like "football" higher weight than very frequent words like “and” or “the”.

This is achieved with TF-IDF method, which combines two factors, the term frequency (TF), which measures how frequently the word occurs in the document. And IDF which downgrades the importance of words, like “the” and “or”.

There are many other features that can be included as part of feature engineering. E.g. when building a model for text quality evaluation, then it can be useful to include readibility factors, some of the more common ones are:

Flesch-Kincaid readability test,
SMOG index,
Gunning fog index,
Dale-Chall readability formula.

Python library which implements the measures above is textstat: https://github.com/shivam5992/textstat.

Word embeddings with word2vec

Word embeddings is another method of feature extraction that converts text to vectors of numbers.

Word2vec is essentially a shallow, two-layer neural network, which takes as input a group of documents and computes a vector for each unique word in the corpus.

What is interesting about Word2Vec is that words that often occur in similar parts in the corpus also have similar place in the word2vec vector hyper-space. The word2vec approach thus preserves semantic relationships.

We have a well known case where if we calculate the relation »Brother«-»Man«+«Woman« using respective word2vec vectors, the resulting vector is closest to the vector representation of word »Sister«.

Product categorization API

An interesting application of NLP frameworks is in field of product categorization. This is generally assignment of product names to one or more of distinct classes.

There are many possible ways of assignment, e.g. we can select just the broad categories like "Apparel" or we can assign products to detailed categories like "Jeans".

These groups of categories are also known as taxonomies and there are many possible. In fact, big stores like Walmart or Rakuten have their own taxonomies that they developed for their assortment of products.

If one does not want to build a new one from scratch when developing product categorization API, then one can use one from Facebook or from Google. And perhaps also adapt it them a bit to own needs.

Redaction APIs are critical for NLP applications that process sensitive documents, as they help organizations meet privacy requirements—see more at the MIT Computer Science & Artificial Intelligence Lab.
For automated and scalable data masking, pii redaction is widely used to protect personally identifiable information in legal, healthcare, and financial sectors.
Content moderation APIs rely on NLP to detect and filter harmful or inappropriate material in text and multimedia, a challenge explored by the University of Cambridge Computer Laboratory.
Solutions such as video moderation use advanced models to scan and flag problematic content in videos, supporting safer online environments.
Anonymization APIs help strip datasets of personal details before further analysis or sharing, a topic covered by the Stanford NLP Group.
Many organizations need to know how to anonymize transcripts or recordings before feeding them to ai? to ensure compliance and ethical AI use.
Recent research from the Allen Institute for AI demonstrates how modern NLP approaches can improve both redaction accuracy and data utility.
By leveraging these APIs, organizations can maintain robust privacy protections, support content safety, and enable responsible AI innovation.

Conclusion

In this article we provided a short introduction to natural language processing with python. NLP has become an important of machine learning field.

In our next article, we will build from scratch an Aspect Based Sentiment Analysis model.