TWEET ANALYSYS ALGORITHM

Ulhak — Thu, 28 Apr 2022 07:24:27 +0000

The algorithm takes as input a sample space of ‘n’ predefined number of tweets. It also takes the highest trending ‘x’ number of trends. The output of running the algorithm is the tweets of the sample space ranked with a deceasing description index.

The algorithm also uses two dictionaries. The first dictionary contains the list of the words, which have less significance to the content description and are more grammatical tools, namely articles, prepositions and conjunctions. The second dictionary consists of all common nouns, adjectives, adverbs, verbs and their derivatives. The former will be called ‘filter’ and the latter ‘cnfilter’ hereon.

The used sample space is placed in a file, separated by an end of tweet character, like ‘%%’. Once the tweets are acquired, the frequency of every word that is used in the file containing the tweet sample space is found. This would exclude the ‘#’ tags and the ‘@’ tags. The URLs in the tweets are also ignored while finding the frequencies. Hence the list of words and their corresponding frequencies is prepared and stored.

It is now to check for association of the highest trending tweet with the other high trending tweets. Tweets about the same event, or person, hold useful content and can be assumed to contain more relevant data. The tweets with a high trending hashtag along with the highest trending hashtag for the second time are used to collect the frequency so as to update the previously generated frequency table.

Once the frequency list is obtained we perform a rating on the words to find its weighted score. This weighted score is used to get the cumulative score of each tweet, which can be used to rank the tweets according to its content relevance.

Also proposed is a way to learn from the newer tweets about the hashtag and get more accurate tweet ranking.

Detecting Trends on Twitter

Ulhak — Wed, 27 Apr 2022 08:36:37 +0000

In today’s society most people communicate through social media every day, ranging from ordinary people to politicians and corporations. Social media forums can be utilized to reach extensive crowds around the world if used strategically.

An example of such a social media is Twitter, which is a widely used platform with a large worldwide user base of 310 million monthly active users. When a topic or an event becomes trending on Twitter, it reaches out to an international crowd of 1 billion unique people per month.

Twitter is a social media platform where users post short messages. Other users can then like and retweet their tweets, which means that they share the other person’s tweet with their followers. Users can follow each other and no mutual following is required. This permits certain users to have millions of followers, whereas other people only have a dozen, if any at all. To enable users to find tweets related to a certain subject, a short sequence of characters starting with a “#” is used. These sequence of characters are called hashtags and allow users to participate with and view all related tweets.

In 2015, the trends ranged from world wide events and tragedies to social phenomenons[21]. These trends included #FIFAWWC (FIFA World Cup) and #ParisAttacks as well as #TheDress1. When a hashtag becomes trending on Twitter it happens fast, the Paris attacks is an example of how a previously nonexistent hashtag’s activity suddenly goes through the roof. However, a group of hashtags can behave differently and still all be classified as trends.

Finding and detecting trends on Twitter is not only useful for companies or people that wish to pick up on the latest trends, but also to Twitter themselves as a company. Twitter makes approximately 85% of its revenue based on selling advertisements. Twitter has developed an algorithm that finds trending hashtags, both global and local trends. Then it is possible for Twitter to strategically use the trending hashtags to place its advertisements such that it reaches out to a large crowd.

Previous work has been done in the area of trend detection in Twitter’s feed. In 2012, Nikolov et al. developed an algorithm that found trends in the Twitter data feed, before Twitter’s algorithms was able to do so. The research team was able to do this by experimenting with different machine learning methods and arrive at an algorithm that used the frequencies of hashtags to predict an upcoming trend.

Machine learning is a relatively new area within computer science that uses data-driven learning to train programs without literally programming them to do so. Methods within machine learning are used for image recognition and selfdriving cars among others[13]. In the context of detecting trends, machine learning algorithms are trained to look at existing Twitter data and then be able to find trends in new data.

Within machine learning, there are two categories of methods. The first one is supervised learning that trains the algorithm by providing a training set of input data and the expected outcome. The second one is unsupervised learning that
takes in a training set of input data, but no data of the expected outcome. This makes unsupervised learning different, as the algorithm has to find a structure and patterns in the given data without knowing what the output is supposed to be.

Unsupervised learning methods can also be helpful in other contexts, such as unsupervised pre-training. Supervised training methods aim to find a global minimum (fitting the problem with the smallest error), but the global minimum
is not guaranteed to be reached every time. Unsupervised pre training can help initialize the supervised method such that when it begins, it is more prone to reach the global minimum.

DEV Community: Ulhak

TWEET ANALYSYS ALGORITHM

Detecting Trends on Twitter