DEV Community: Steven Mathew

Alibaba releases new Qwen-2-VL which can analyze 20mins long videos

Steven Mathew — Fri, 13 Sep 2024 14:04:09 +0000

Alibaba has recently unveiled Qwen-2-VL, an advanced AI model designed for analyzing long-format videos, particularly those exceeding 20 minutes. This model represents a significant advancement in multimodal AI, as it can process both video and audio content, offering more precise insights from complex visual and auditory data. Unlike previous models that focused on short clips, Qwen-2-VL can comprehend intricate narratives and patterns over extended durations.

One of the standout features of Alibaba AI Model Qwen-2-VL is its ability to understand contextual information from long videos. By analyzing content in real time, the model can identify important moments, summarize key points, and generate rich interpretations. This makes it ideal for applications in education, media production, and entertainment, where long-format videos are common, and detailed analysis is crucial.

Qwen-2-VL also excels in bridging the gap between text, images, and video. Its multimodal capabilities mean it can answer questions based on video content and create summaries that incorporate both visual and textual elements. This could revolutionize how video-based information is processed, enabling faster insights in sectors like marketing, content creation, and e-learning.

By releasing Qwen-2-VL, Alibaba demonstrates its commitment to advancing AI technology, focusing on models that provide greater utility in real-world applications. This AI model could pave the way for more efficient content analysis, offering deeper insights from videos in ways that were previously difficult for AI to achieve.

Benefits of Alibaba AI Model Qwen-2-VL:

Long Video Analysis: Unlike previous AI models that struggle with longer content, Qwen-2-VL can analyze videos exceeding 20 minutes, providing more in-depth analysis and understanding of complex sequences.

Multimodal Processing: It can handle both video and audio content simultaneously, offering enhanced insights compared to single-modality models.

Real-time Analysis: Qwen-2-VL processes content as it plays, making it highly effective for live video summarization and analysis.

Comparison to Existing Models:

Long-Content Capability: Most existing AI models, like OpenAI’s GPT-4 and Google’s PaLM, are excellent at handling text but struggle with extended video content. Qwen-2-VL fills this gap by focusing on video understanding, particularly long-format videos.

Contextual Understanding: While some models are optimized for short clips or image-based tasks (like OpenAI’s CLIP), Qwen-2-VL is more robust in comprehending intricate and evolving narratives in longer videos.

Integrated Multimodal Performance: Unlike older models that either focused on text, video, or audio separately, Qwen-2-VL integrates these modalities, making it more versatile for real-world use cases like educational videos, media, and entertainment analysis.

Sarcasm Detection AI Model (97% Accuracy) Trained With Reddit Comments - Training & Testing

Steven Mathew — Sun, 07 Jul 2024 04:20:56 +0000

Now we are going to split the data to train and test the data to check the accuracy.

df = pd.read_csv('labeled_reddit_comments.csv')

This line reads the previously saved CSV file (labeled_reddit_comments.csv) containing cleaned Reddit comments and their corresponding labels into a Pandas DataFrame (df).

Splitting Data into Training and Testing Sets

X_train, X_test, y_train, y_test = train_test_split(df['cleaned_comment'], df['label'], test_size=0.2, random_state=42)

Here, we split the data into two parts:
X_train and y_train: These variables contain 80% of the data (df['cleaned_comment'] and df['label']) which will be used for training the model.

X_test and y_test: These variables contain the remaining 20% of the data, which will be used to evaluate how well the trained model performs on new, unseen data.

Creating a Pipeline with a Random Forest Classifier

pipeline = Pipeline([
    ('tfidf', TfidfVectorizer()),
    ('clf', RandomForestClassifier(random_state=42))
])

This sets up a pipeline (pipeline) that sequentially applies two steps to the data:
Step 1 ('tfidf', TfidfVectorizer()): Converts the text data (X_train and X_test) into numerical TF-IDF (Term Frequency-Inverse Document Frequency) vectors.

Step 2 ('clf', RandomForestClassifier(random_state=42)): Trains a Random Forest classifier on the TF-IDF vectors. The random_state=42 ensures reproducibility of results.

Defining Hyperparameters for Tuning

param_grid = {
    'tfidf__max_features': [10000, 20000, None],
    'clf__n_estimators': [50, 100],
    'clf__max_depth': [None, 10],
    'clf__min_samples_split': [2, 5],
    'clf__min_samples_leaf': [1, 2]
}

This dictionary (param_grid) specifies different hyperparameter values to explore during the grid search process:
'tfidf_max_features': Limits the number of features generated by TfidfVectorizer.
'clfn_estimators', 'clfmax_depth', 'clfmin_samples_split', 'clf_min_samples_leaf': Parameters that control the behavior of the Random Forest classifier.

Performing GridSearchCV for Hyperparameter Tuning

grid_search = GridSearchCV(pipeline, param_grid, cv=5, scoring='accuracy', verbose=1, error_score='raise')
grid_search.fit(X_train, y_train)

Here, GridSearchCV is used to search for the best combination of hyperparameters (param_grid) for the pipeline (pipeline). It:
Divides the data into 5 folds (cv=5) for cross-validation.

Uses accuracy (scoring='accuracy') as the metric to evaluate the performance of each combination of hyperparameters.
Prints detailed messages (verbose=1) during the search process and raises errors (error_score='raise') if an error occurs.

Evaluating the Best Model

best_model = grid_search.best_estimator_
y_pred = best_model.predict(X_test)

# Print evaluation metrics
print(f"Accuracy: {accuracy_score(y_test, y_pred)}")
print(classification_report(y_test, y_pred))

After finding the best set of hyperparameters (best_model), the code evaluates this model's performance on the test data (X_test) that was set aside earlier (y_test).

It:
Predicts labels (y_pred) for the test data.
Calculates and prints the accuracy score (accuracy_score) of the predictions compared to the actual labels (y_test).

Prints a detailed classification report (classification_report) showing precision, recall, F1-score, and support for each class (sarcasm and non-sarcasm).

After training and Testing I got an accuracy of 97%

Testing with sample text

Checking on the top 5 comments on a post on Reddit

GITHUB: https://github.com/stevie1mat/Sarcasm-Detection-With-Reddit-Comments

Author: Steven Mathew

Sarcasm Detection AI Model (97% Accuracy) Trained With Reddit Comments - Cleaning and Saving The Data

Steven Mathew — Sun, 07 Jul 2024 04:11:05 +0000

Now we will clean the data and save the data for training and testing in the next part.

def clean_comment(text):
    text = re.sub(r'http\S+', '', text)  # Remove any web URLs in the text
    text = re.sub(r'/u/\w+', '', text)  # Remove mentions of Reddit users (like /u/username)
    text = re.sub(r'r/\w+', '', text)  # Remove mentions of subreddits (like r/subreddit)
    text = re.sub(r'\n', ' ', text)  # Replace new line characters with spaces
    text = re.sub(r'[^A-Za-z0-9\s]', '', text)  # Remove any characters that are not letters, numbers, or spaces
    return text.lower()  # Convert the cleaned text to lowercase

This function takes in a piece of text (text) and cleans it up by removing web URLs, mentions of Reddit users and subreddits, new line characters, and any characters that are not letters, numbers, or spaces. Finally, it converts the cleaned text to lowercase.

# Load data from a CSV file into a DataFrame
df = pd.read_csv('reddit_comments.csv')

# Apply the cleaning function to each comment and create a new column for cleaned comments
df['cleaned_comment'] = df['comment'].apply(clean_comment)

Here, we load data from a CSV file (reddit_comments.csv) into a table-like structure called a DataFrame. Then, for each comment in the 'comment' column of this DataFrame, we use the clean_comment function we defined earlier to clean up the text. The cleaned versions of the comments are stored in a new column named 'cleaned_comment'.


# Manually assign labels to the comments
labels = [0, 1] * (len(df) // 2)  # Create a list of labels alternating between 0 and 1
if len(labels) < len(df):
    labels.append(0)  # Add one more label to match the number of comments

df['label'] = labels  # Assign the labels to a new column named 'label' in the DataFrame

In this part, we assign labels to each comment to indicate whether it's sarcastic or not. For demonstration purposes, we alternate between labels 0 (for non-sarcastic) and 1 (for sarcastic). We make sure that each comment gets a corresponding label. These labels are stored in a new column named 'label' in the DataFrame.

# Remove rows where the cleaned comment is empty or NaN (missing)
df = df.dropna(subset=['cleaned_comment'])  # Remove rows where 'cleaned_comment' is NaN
df = df[df['cleaned_comment'].str.strip() != '']  # Remove rows where 'cleaned_comment' is empty or only whitespace

# Save the cleaned and labeled data to a new CSV file
df.to_csv('labeled_reddit_comments.csv', index=False)  # Save DataFrame to CSV without including the index

Finally, we clean up the data further by removing any rows where the cleaned comment is empty or missing (NaN). We also remove rows where the cleaned comment consists only of whitespace.

After cleaning and filtering, we save the cleaned and labeled data (including the 'cleaned_comment' and 'label' columns) to a new CSV file named labeled_reddit_comments.csv.

Note:
The index=False parameter ensures that the CSV file does not include an extra column for row numbers.

Read the Part 3 - Sarcasm Detection From Reddit Comments : Training & Testing

GITHUB: https://github.com/stevie1mat/Sarcasm-Detection-With-Reddit-Comments

Author: Steven Mathew

Sarcasm Detection AI Model (97% Accuracy) Trained With Reddit Comments - Part 1

Steven Mathew — Sun, 07 Jul 2024 04:04:00 +0000

I have trained a Sarcasm Detection AI model using Reddit comments. This is how you can do it too.

Requirements:
Google Colab
Reddit API Credentials
Lots of time
Coffee

First we will import the necessary libraries.

import asyncio  # For asynchronous programming in Python.
import asyncpraw  # Python Reddit API Wrapper for asynchronous Reddit API interactions.
import pandas as pd  # Data manipulation and analysis tool.
import nest_asyncio  # Necessary for allowing nested asyncio run loops.
import re  # Regular expressions for pattern matching and text manipulation.
from sklearn.model_selection import train_test_split  # Splits data into training and testing sets.
from sklearn.feature_extraction.text import TfidfVectorizer  # Converts text data into TF-IDF feature vectors.
from sklearn.ensemble import RandomForestClassifier  # Random Forest classifier for machine learning.
from sklearn.metrics import accuracy_score, classification_report  # Metrics for evaluating model performance.
from imblearn.over_sampling import SMOTE  # Oversampling technique for handling class imbalance.
from sklearn.pipeline import Pipeline  # Constructs a pipeline of transformations and estimators.
from sklearn.model_selection import GridSearchCV  # Performs grid search over specified parameter values.

Connecting to Reddit API Get your API credentials from https://www.reddit.com/prefs/apps

`client_id = 'your_client_id'
client_secret = 'your_client_secret'
user_agent = 'MyRedditApp/0.1 by your_username'

reddit = praw.Reddit(client_id=client_id,
                     client_secret=client_secret,
                     user_agent=user_agent)`

This code sets up authentication credentials (client_id, client_secret, user_agent) to create a Reddit API connection using praw. The Reddit object initializes a connection to Reddit's API, allowing the Python script to interact with Reddit, retrieve data, and perform various actions programmatically on the platform.

Initialization and Setup

`nest_asyncio.apply()`

This line ensures that asyncio can be used in a nested manner, which is necessary when using asynchronous operations in environments that already have an event loop running.

Asynchronous Function Definition

`async def collect_reddit_comments(subreddit_name, keyword, limit=1000):
    reddit = asyncpraw.Reddit(
        client_id=client_id,
        client_secret=client_secret,
        user_agent=user_agent
    )`

Defines an asynchronous function collect_reddit_comments to retrieve comments from Reddit. It initializes a Reddit instance using asyncpraw, passing in credentials (client_id, client_secret, user_agent) for API authentication.

Fetching Subreddit and Comments

`subreddit = await reddit.subreddit(subreddit_name)
comments = []
count = 0
after = None`

Asynchronously fetches the subreddit object based on subreddit_name. Initializes an empty list comments to store comment data, and sets counters (count) and pagination marker (after) for comment retrieval.

Looping Through Submissions and Comments

`while len(comments) < limit:
    try:
        async for submission in subreddit.search(keyword, limit=None, params={'after': after}):
            await submission.load()
            submission.comment_limit = 0
            submission.comments.replace_more(limit=0)`

Explanation: Enters a loop to fetch submissions matching keyword within the specified subreddit. Asynchronously loads submission details and retrieves all comments for each submission, handling cases where more comments are nested (replace_more).

Collecting and Storing Comments

           ` for comment in submission.comments.list():
                if isinstance(comment, asyncpraw.models.Comment):
                    author_name = comment.author.name if comment.author else '[deleted]'
                    comments.append([comment.body, author_name, comment.created_utc])
                    count += 1

                    if count >= limit:
                        break

            after = submission.id  # Sets the 'after' parameter for pagination

            if count >= limit:
                break`

Iterates through each comment in the submission, checking if it's a valid comment. Collects comment details such as body, author name, and creation time (created_utc). Controls the loop with count and limit to ensure the specified number of comments (limit) is collected.

Handling API Exceptions

    `except asyncpraw.exceptions.APIException as e:
        print(f"API exception occurred: {e}")
        wait_time = 60  # Wait for 1 minute before retrying
        print(f"Waiting for {wait_time} seconds before retrying...")
        await asyncio.sleep(wait_time)`

Catches and handles API exceptions that may occur during Reddit API interactions. Prints the exception message, waits for a minute (wait_time) before retrying, and then resumes fetching comments.

Returning Results

`return comments[:limit]`  # Returns up to 'limit' number of comments

Returns a list of collected comments, limited by the specified limit, ensuring only the required number of comments are returned.

Main Function to Execute Collection

async def main():
    comments = await collect_reddit_comments('sarcasm', 'sarcastic', limit=5000)  # Adjust limit as needed
    df = pd.DataFrame(comments, columns=['comment', 'author', 'created_utc'])
    df.to_csv('reddit_comments.csv', index=False)
    print(f"Total comments collected: {len(df)}")
    print(df.head())

Defines an asynchronous main function to orchestrate the comment collection process. Calls collect_reddit_comments with parameters subreddit_name='sarcasm', keyword='sarcastic', and limit=5000 (can be adjusted). Converts collected comments into a Pandas DataFrame (df), stores it as a CSV file (reddit_comments.csv), and prints summary information about the collected data.

Running the Main Function

`await main()`

Executes the main function asynchronously, initiating the process of collecting Reddit comments, processing them into a DataFrame, saving them to a CSV file, and providing feedback on the number of comments collected and a preview of the data.

Read the Part 2 - Sarcasm Detection From Reddit Comments : Cleaning & Saving The Data

GITHUB: https://github.com/stevie1mat/Sarcasm-Detection-With-Reddit-Comments

Author: Steven Mathew

Flutter Youtube List With PHP & MySql

Steven Mathew — Fri, 21 Oct 2022 17:20:01 +0000

While working on a project, I came across the need to create a dynamic Youtube list of videos using the youtube_player_flutter package.

So here is how I went about developing the code.

To start with, I create a new flutter project. Then I opened up my PHPMyAdmin to start creating tables for storing the dynamic data that will be shown to the user.

Create A SQL Table
CREATE TABLE `videosapp` ( `youtubeid` varchar(100) NOT NULL ) ENGINE=InnoDB DEFAULT CHARSET=latin1; ALTER TABLE videosapp ADD PRIMARY KEY (youtubeid); COMMIT;

Create the PHP file to provide the JSON data from the PHPMyAdmin database.

Connect to the phpmyadmin.

static $DB_SERVER=””;

static $DB_NAME=””;

static $USERNAME=””;

static $PASSWORD=””;

Fetch the data and convert it into JSON format.

while($row=$result->fetch_array()) {

array_push($spacecrafts, array(“youtubeid”=>$row[‘youtubeid’])); }

print(json_encode(array_reverse($spacecrafts)));

The final step is to parse the value in the Flutter application.

Use a ListView Builder to create a widget that will create a list from the Youtube Id’s retrieved from the PHP file in JSON Format.

ListView.builder( itemCount: widget.spacecrafts.length, itemBuilder: (context, int currentIndex) {

return createViewItem(widget.spacecrafts[currentIndex], context); },

Create a List from the Youtube controller by providing the Youtube Id’s from the database to the Youtube Player Controller intialVideoId.

final List<YoutubePlayerController> _controllers = [spacecraft.youtubeid] .map<YoutubePlayerController>( (videoId) =>

YoutubePlayerController(

initialVideoId: videoId,

flags: YoutubePlayerFlags( autoPlay: false, ), ), ) .toList();

Create a Future Builder and retrieve the snapshot data and show it onto the screen. You can also use Firebase or any other backend service to display the list of videos by using StreamBuilder.

You can find the Full Source Code of the project here.

stevie1mat / Flutter-Youtube-List-With-PHP-MySql

Flutter Youtube List From PHP - MySQL (using phpmyadmin)

Flutter Youtube List From PHP - MySQL (using phpmyadmin)

Flutter youtube list view using PHP and MySQL by converting it into json and the parsing it in the Flutter app. You can also use json directly if You don't want to use PHP. (All of these work remotely so You can edit and change it.)

Packages Used

Update to the lastest version so that You don't face any errors.

Steps For Setting This Up:

Create A SQL Table


      CREATE TABLE `videosapp` (
  `youtubeid` varchar(100) NOT NULL
) ENGINE=InnoDB DEFAULT CHARSET=latin1;

ALTER TABLE videosapp ADD PRIMARY KEY (youtubeid); COMMIT;

Edit the videoapp.php file with Your credentials & table name and upload it to Your server.

Finally change the link to php file in the list.dart file. Sample link has been provided Please don't misuse it in any way.

Screen Shots

Buy Me A Coffee

View on GitHub

Author: Steven Mathew