DEV Community: Sujan Koirala

PawRover:A Community-Driven Solution for Pet Care

Sujan Koirala — Mon, 02 Mar 2026 05:23:46 +0000

This is a submission for the DEV Weekend Challenge: Community

The Community

Pet owners face a universal challenge: finding trustworthy, affordable care for their furry family members when they travel. Traditional boarding facilities can be expensive and stressful for pets, while finding reliable sitters through word-of-mouth is hit-or-miss. I built Paw Rover for the global community of pet lovers—people who understand that pets aren't just animals, they're family.

This platform serves two interconnected groups: pet owners who need peace of mind while traveling, and pet enthusiasts who want to experience the joy of caring for animals while exploring new places. It's a community built on trust, shared love for animals, and the belief that pet care should be accessible, transparent, and rewarding for everyone involved.

What I Built

Paw Rover is a platform that connects pet owners with trusted sitters through a home exchange model. Instead of expensive boarding fees, pet owners can find caring individuals who'll stay in their homes and look after their pets. In return, sitters get free accommodation in unique locations while enjoying the company of adorable animals.

Key features include:

Find a Pet Sitter: Search for reliable sitters in your area or travel destination
Explore Homes to Sit: Browse pet-friendly homes available for sitting opportunities
Interactive Onboarding: Guided tour for new users using Shepherd.js

The platform emphasizes transparency and trust through its rating system, ensuring both pet owners and sitters can connect with confidence.

Demo

🎥 Live Demo

📸 Screenshots

Home Page with Interactive Guide by Rabbit:

Choose your pets:

Explore perfect pet sitters:

Explore comfy homes to sit

Code

Github Link for PawRover

How I Built It

Paw Rover is built with modern web technologies focused on performance and user experience:

React: Powers the interactive UI and component-based architecture
Vite: Provides lightning-fast development and optimized production builds
Tailwind CSS: Enables responsive, beautiful styling with utility-first approach
Shepherd.js: Creates an intuitive guided tour for first-time users

The tech stack was chosen to deliver a smooth, responsive experience that works seamlessly across devices—because pet owners and sitters need to connect on the go.

Impact

Paw Rover transforms pet care from a transactional service into a community-driven experience. By eliminating the high costs of traditional boarding and creating meaningful connections between pet lovers, the platform makes travel more accessible for pet owners while giving animal enthusiasts opportunities to explore new places.

Every successful match strengthens the network, creating a ripple effect of positive experiences that benefit pets, owners, and sitters alike.

Built with ❤️ for pet lovers everywhere

LessonSync - Interactive Video Learning Platform

Sujan Koirala — Mon, 05 Jan 2026 03:12:01 +0000

🎥 LessonSync - Interactive Video Learning Platform

This is a submission for the DEV's Worldwide Show and Tell Challenge Presented by Mux

What I Built

LessonSync is a revolutionary video-based learning platform that transforms passive video watching into interactive, collaborative experiences. Unlike traditional video platforms where viewers can only watch and move on, LessonSync allows learners to ask timestamped questions, record video responses, and navigate content through an intelligent interactive timeline.

Key Features:

🎯 Timestamped Q&A - Click anywhere on the video timeline to ask questions at specific moments
📹 Video Responses - Record video answers directly in the browser using built-in camera integration
🗺️ Interactive Timeline - Visual timeline with question markers for easy navigation and content discovery
👥 User Engagement - Upvoting system, user avatars, and real-time collaboration
📊 Analytics Dashboard - Track video engagement, views, and learning patterns
🔍 Public Discovery - Explore educational content from other creators

My Pitch Video

Demo

🔗 GitHub Repository: https://github.com/Sujan-Koirala021/lessonsync

The Story Behind It

The idea for LessonSync came from a frustrating personal experience during online learning. I was watching educational videos and constantly had questions about specific moments, but there was no way to ask them contextually. I'd pause, write notes, lose track of timestamps, and eventually give up on getting answers.

I realized that video learning was fundamentally broken - it was a one-way street. Students watch passively, questions pile up unanswered, and engagement dies. What if we could make every moment in a video a potential conversation starter?

The "Aha!" Moment: What if clicking on a video timeline could do more than just seek? What if it could start a conversation?

This led to LessonSync's core innovation: timestamped interactions. Every question is tied to a specific moment, creating a rich, contextual learning experience where knowledge builds upon itself.

Why It Matters:

📚 For Students: Get answers to specific questions without losing context
👨‍🏫 For Educators: Understand exactly where students struggle and engage directly
🌍 For Communities: Transform isolated learning into collaborative knowledge building

Technical Highlights

LessonSync is built as a modern full-stack application with a focus on real-time interactivity and seamless video experiences.

Architecture

Frontend: React 19 + Vite for lightning-fast development and modern UI
Backend: Node.js + Express with JWT authentication
Database: SQLite for rapid development with easy deployment
Video Infrastructure: Mux for professional video processing and streaming

Key Technical Innovations

1. Interactive Timeline System

// Custom timeline component with clickable question markers
const getTimelineMarkers = () => {
  return questions.map(question => ({
    ...question,
    position: (question.timestamp / video.duration) * 100
  }));
};

2. Real-time Video Recording

Built-in browser-based video recording using MediaRecorder API
Automatic codec detection for cross-browser compatibility
Real-time camera preview with stream validation

3. Smart User Experience

Contextual Timestamps: "5m ago", "just now" instead of raw dates
Consistent User Avatars: Color-coded avatars based on username hashing
Progressive Enhancement: Works without JavaScript for basic video playback

4. Scalable Data Architecture

// Efficient question-to-video relationship mapping
const videoQuestions = Array.from(questionsStore.values())
  .filter((q) => q.videoId === videoId)
  .sort((a, b) => a.timestamp - b.timestamp);

Use of Mux (Additional Prize Category Participants Only)

LessonSync leverages Mux extensively beyond basic video hosting, utilizing multiple advanced features:

1. Direct Upload Integration

// Seamless direct-to-Mux uploads without server storage
const res = await api.post("/upload", {
  title: title.trim(),
  description: description.trim(),
  is_public: true
});
const { url, videoId, muxUploadId } = res.data;

2. Webhook-Driven Processing

Real-time status updates via Mux webhooks for upload progress
Automatic asset management when videos finish processing
Error handling for failed uploads with user feedback

// Webhook handlers for different Mux events
switch (event.type) {
  case "video.upload.asset_created":
    handleUploadAssetCreated(event.data);
    break;
  case "video.asset.ready":
    handleAssetReady(event.data);
    break;
  case "video.asset.errored":
    handleAssetError(event.data);
    break;
}

3. Advanced Playback Features

Mux Player React integration for professional video experience
Programmatic seeking for timeline-based navigation
Time-based event handling for question synchronization

4. Video Recording Pipeline

Dual video streams: Main educational content + user-generated Q&A videos
Automatic transcoding for optimal playback across devices
Thumbnail generation for video previews

5. Analytics Integration

View tracking with completion percentages
Engagement metrics tied to specific video moments
Performance monitoring for video load times

My Experience Building with Mux

What I Loved:

🚀 Developer Experience: The API is intuitive and well-documented
⚡ Speed: Videos process incredibly fast, keeping users engaged
🔧 Flexibility: Direct uploads eliminate server storage concerns
📊 Reliability: Webhooks provide real-time updates without polling

Challenges Overcome:

Webhook Timing: Learned to handle async processing gracefully
Error States: Built robust fallbacks for upload failures
Cross-browser Recording: Mux's format flexibility solved codec issues

Impact on User Experience:
Mux enabled LessonSync to feel like a professional platform from day one. Users upload videos and see them processing in real-time, creating a seamless experience that would have taken months to build from scratch.

What's Next?

LessonSync represents just the beginning of interactive video learning. Future plans include:

🤖 AI-powered question suggestions based on video content
📱 Mobile app for learning on the go
🎓 Classroom integration for educators
🌐 Multi-language support for global accessibility

Built with ❤️ for the future of interactive learning

Building Bunny Brew Café: A Delightful Interactive Pet Experience with Hot Design & AI

Sujan Koirala — Sun, 07 Dec 2025 16:41:19 +0000

This is a submission for the AI Challenge for Cross-Platform Apps - Hot Design Showcase

What I Built

I created Bunny Brew Café - an interactive virtual pet experience where a charming barista bunny serves coffee in a cozy café setting. The app combines the joy of virtual pet interactions with a delightful coffee shop theme, featuring:

🐰 An animated barista bunny with 10 different GIF animations
☕ Interactive coffee menu cards with smooth hover effects
🎨 Real-time particle effects (floating coffee beans, carrot rain)
💬 Dynamic speech bubbles with personality-filled responses
📋 A chalkboard menu with daily specials and rotating quotes

What started as a simple pet playground evolved into a full café experience, and Hot Design Agent was instrumental in refining the UI interactions and animations in real-time.

Original Design Reference

The concept was inspired by:

Virtual pet games like Tamagotchi and Neko Atsume
Cozy café aesthetics with warm brown tones (#6F4E37, #D4A574)
Interactive UI patterns from modern mobile apps

Demo

🎥 Live Demo

👉 Watch here: YouTube – Live Demo

📸 Screenshots

Main Café View:

Interactive Bubble Dialogue:

Carrot Rain Celebration:

🔗 Repository

GitHub Repository

Hot Design Agent in Action

Real-Time UI Refinement

The Hot Design Agent was a game-changer for bringing the café to life. Here's how it helped me tackle complex animation challenges:

Challenge 1: Dynamic GIF Switching on Interaction

Problem: Wanted the bunny to change expressions when users click different coffee cards, but managing 10 different GIF states was complex
AI Solution: Described the desired behavior - "bunny should react differently to each coffee type"
Agent's Help: Suggested organizing GIFs by mood (happy, excited, eating) and creating a mapping system
Implementation: Agent helped me set up event handlers for each card that trigger specific animations
Result: Bunny now shows unique reactions - excited for espresso, content for latte, delighted for mocha!

Challenge 2: Speech Bubble Appearing on Card Hover

Initial Attempt: Bubble appeared instantly, felt jarring
AI Guidance: Agent recommended a pop-in animation with BackEase for a bouncy, playful feel
Real-Time Tuning: While app was running, adjusted timing from 200ms → 150ms for snappier response
Polish: Added scale transform (0.5 → 1.0) with amplitude of 0.5 for that perfect "pop"
Impact: Bubble now feels alive and responsive, enhancing the café's personality

Challenge 3: Carrot Rain Celebration Effect

Vision: When users buy coffee, celebrate with falling carrots (bunny's favorite treat!)
Complexity: 30 animated carrots with rotation, drift, and fade effects
AI Assistance: Agent helped optimize the staggered creation (100ms delays) and animation curves
Fine-Tuning: Adjusted fall duration (1500-2500ms random), rotation speed (2-5 full spins)
Result: Delightful reward animation that makes purchases feel special

Challenge 4: Performance Optimization - Too Many Animations

Problem: App became laggy with floating beans + carrot rain + bubble animations all running simultaneously
Symptoms: Frame drops, stuttering on lower-end devices
AI Diagnosis: Agent identified the particle cleanup wasn't happening efficiently
Solution: Implemented proper storyboard.Completed handlers to remove particles from Canvas
Optimization: Reduced floating beans from 15 → 10, adjusted opacity to 0.25 for lighter rendering
Result: Smooth 60fps animations even with multiple effects active

Design Process

1. Initial Setup with Hot Reload

Started with basic XAML structure:

<Border Background="#6F4E37" CornerRadius="12" Width="140" Height="180">
  <StackPanel Padding="16">
    <!-- Coffee card content -->
  </StackPanel>
</Border>

Hot Reload kept my Visual Studio and running app perfectly synchronized as I:

Adjusted color schemes in real-time
Modified spacing and margins
Tested different font sizes

2. Animation Development

GIF State Management:

private void OnEspressoHover(object sender, PointerRoutedEventArgs e)
{
    ShowCoffeeDialogue("Espresso! ☕ Bold and intense, just like you!");
    AnimateCardHover(sender as Border, true);
}

private void OnImageTapped(object sender, TappedRoutedEventArgs e)
{
    _currentGifIndex = (_currentGifIndex % TotalGifs) + 1;
    var newSource = $"ms-appx:///Assets/Pet/rabbit{_currentGifIndex}.gif";
    PetImage.Source = new BitmapImage(new Uri(newSource));
}

Speech Bubble Pop Animation:

private void ShowCoffeeDialogue(string message)
{
    SpeechText.Text = message;
    var scaleXIn = new DoubleAnimation
    {
        From = 0.5,
        To = 1.0,
        Duration = TimeSpan.FromMilliseconds(150),
        EasingFunction = new BackEase {
            EasingMode = EasingMode.EaseOut,
            Amplitude = 0.5
        }
    };
    // Bubble pops in with personality!
}

Carrot Rain Celebration:

private void CreateCarrotRain()
{
    for (int i = 0; i < 30; i++)
    {
        var delay = i * 100; // Stagger creation
        var timer = new DispatcherTimer {
            Interval = TimeSpan.FromMilliseconds(delay)
        };
        timer.Tick += (s, e) => {
            CreateFallingCarrot();
            (s as DispatcherTimer)?.Stop();
        };
        timer.Start();
    }
}

Hot Design Agent helped me:

Organize GIF switching logic for different interactions
Perfect the bubble pop timing (200ms → 150ms)
Optimize carrot animation performance with proper cleanup

3. Visual Polish with AI Assistance

Floating Coffee Beans:

Used Canvas for particle system
Hot Design Agent suggested opacity (0.25) and size ranges (30-50px)
Implemented sine wave motion for natural floating effect
Performance Fix: Reduced from 15 → 10 beans after Agent identified frame drops

Speech Bubbles:

Positioned dynamically with CompositeTransform
AI helped optimize the pop-in animation timing (150ms sweet spot)
Added personality with randomized messages array
Trigger: Appears on card hover, creating delightful feedback

Carrot Rain Particles:

30 carrots with rotation (2-5 full spins), drift, and fade
Agent helped implement proper cleanup: storyboard.Completed += (s, e) => ParticleCanvas.Children.Remove(carrot);
Random fall durations (1500-2500ms) for natural variation
Challenge: Initial implementation caused lag; Agent suggested staggered creation with 100ms delays

Color Palette Refinement:

Original: #6F4E37 (coffee brown)
Hover: #96644B (warmer brown) - suggested by Agent
Accent: #D4A574 (caramel) for buttons and highlights

4. Controls & Bindings Modified

Modified in Hot Design:

Border.Background - Color animations
StackPanel.Padding - Spacing optimization
TextBlock.FontSize - Typography hierarchy
Button.Padding - Touch target sizing
Border.CornerRadius - Rounded corners
ThemeShadow - Depth and elevation

Data Bindings:

Dynamic quote rotation from string array
Random message selection for speech bubbles
Mood-based animation selection (10 rabbit GIFs)

Design Process

Key Takeaways

🚀 Speed & Efficiency

Hot Design Agent + Hot Reload reduced iteration time by ~70%. What used to take multiple compile cycles now happens in seconds with visual feedback.

🎨 AI as a Design Partner

The Agent doesn't just execute commands - it suggests improvements:

"This scale might cause overflow, try 1.08 instead of 1.12"
"Reduce padding to create more breathing room"
"Consider elastic easing for a playful feel"

🔄 Real-Time Iteration

Being able to adjust UI while the app runs is transformative:

Test hover states without restarting
See animation timing changes immediately
Validate color choices in context

📱 Cross-Platform Confidence

Built with Uno Platform, the same XAML runs on:

Windows (WinUI 3)
Web (WebAssembly)
Mobile (iOS/Android)
macOS and Linux

Hot Design ensures consistency across all platforms.

🤖 AI-Powered vs Traditional Development

Traditional Approach:

Write XAML
Compile
Run app
Notice issue
Stop app
Modify code
Repeat

Hot Design + AI Approach:

Run app once
Describe desired change to Agent
See it happen in real-time
Iterate visually
Keep refining without stopping

Result: From idea to polished UI in a fraction of the time.

💡 Lessons Learned

Start simple, iterate visually - Basic structure first, then refine with Hot Design
Performance matters with animations - Too many simultaneous effects = lag; Agent helped identify bottlenecks
Animation timing is everything - 50ms difference between 200ms and 150ms is noticeable; real-time testing is crucial
Proper cleanup prevents memory leaks - Always remove particles from Canvas in Completed handlers
Stagger complex animations - 100ms delays between carrot creations made the effect smooth instead of janky
GIF state management needs structure - Organizing 10 animations by mood/action makes code maintainable
Cross-platform from day one - Hot Design makes it easy to validate on multiple targets

Technical Stack

Framework: Uno Platform 5.x
Language: C# + XAML
IDE: Visual Studio Code, Kiro
Tools: Hot Design, Hot Reload, Hot Design Agent (Preview)
Animations: Storyboards with custom easing functions

Conclusion

Building Bunny Brew Café with Hot Design Agent showed me the future of UI development. The combination of visual design tools and AI assistance creates a workflow that's both faster and more creative. Instead of fighting with code, I could focus on crafting delightful interactions and polishing every detail.

The barista bunny is ready to serve! ☕🐰

Thank you for checking out Bunny Brew Café! I hope it makes you smile as much as it made me smile building it. 😊

How Apache Spark Transforms Big Data Analytics

Sujan Koirala — Fri, 18 Jul 2025 11:01:15 +0000

Background

With the increase in internet usage, there is tremendous increase in the quantity of data from various sources including social media posts, images, audio, video, texts, etc. Due to these large data from variety of sources, the traditional data processing tools become incapable of processing.

So the concept of big data emerged, where big data is defined as extremely large and complex datasets that are difficult to process using the traditional data processing tools. Big data is generally characterized by 3Vs:

Volume (data size varies from terabytes to petabytes)
Velocity (high speed with which data is processed)
Variety (data with different format, source and structure)

Introduction to Apache Spark

Apache Spark is an open-source distributed data processing engine for large data. Started in 2009 as a research project at the University of Colombia, the main objective of the project was to speed up the processing jobs in Hadoop systems.

It supports batch processing, stream processing, graph processing and machine learning, with support for programming languages like Python, Java, Scala, R, and SQL.

Why Spark?

Spark is used because of its fast data processing, advanced API support, scalability and ease of use, which are explained below:

1. Scalability

Apache Spark can handle data ranging from a single machine to thousands of servers in a cluster. It distributes data and computation across many nodes, allowing you to process very large datasets (big data) efficiently.

You can easily add more nodes to scale up processing power.

2. Real Time

Spark supports real-time data processing with its Spark Streaming module.

It processes live data streams (e.g., from sensors, logs, social media) with low latency.

3. Fault Tolerance

Spark’s data abstraction, RDD (Resilient Distributed Dataset), is designed to handle failures. Each RDD keeps track of how it was derived from other datasets (lineage).

If a node fails, Spark can recompute lost partitions automatically without needing to restart the entire job.

4. Ease of Use

Spark provides high-level APIs that simplify writing distributed applications.

It supports multiple programming languages (Scala, Python, Java, R) and comes with built-in libraries (Spark SQL, MLlib, GraphX) for common tasks, reducing the amount of code needed.

5. Speed

Spark performs data processing in-memory, reducing time spent on disk I/O. This makes Spark much faster (up to 100x) than traditional systems like Hadoop MapReduce for many workloads. It also optimizes execution plans to run tasks efficiently in parallel.

Why Spark Outperforms Hadoop MapReduce

Apache Hadoop, a traditional distributed data processing tool, writes the intermediate result to disk after each step resulting in slow performance, making it inefficient for iterative jobs.

However, Spark makes use of in-memory processing, i.e., the intermediate results are stored in RAM so that only the final result could be written into the disk, making it 100 times faster than Hadoop.

Besides, Apache Spark's fault tolerance, based on its resilient distributed datasets (RDDs), allows it to quickly recover from node failures by recomputing only lost data partitions — enhancing efficiency.

Moreover, Spark supports real-time data processing through its streaming capabilities, whereas Hadoop is mainly limited to batch jobs.

Spark Ecosystem

Figure: Overview of the Apache Spark Ecosystem and its Core Components

(Source: Data Flair)

The Apache Spark ecosystem consists of powerful tools and libraries built around the core Spark engine to support various big data processing tasks. Here are the key components:

1. Spark Core

This is the foundation of the Spark ecosystem, where all other Spark components are built on top of it.

It handles:

Task Scheduling
Memory Management
Fault Recovery
Interactions with Storage Systems

2. Spark SQL

It enables querying structured data using SQL or DataFrame APIs. It supports various data sources like JSON, Hive tables, and Parquet files.

3. Spark Streaming

It allows processing of real-time data streams from sources such as Kafka, Flume, or TCP sockets and processes it in near real-time. This is crucial for applications like fraud detection, live monitoring, and alert systems.

4. MLlib (Machine Learning Library)

It is a scalable machine learning library that offers algorithms for:

Classification
Regression
Clustering
Collaborative Filtering
and more.

5. GraphX

It is a library for graph processing and analytics.

It enables computation on graphs and graph-parallel computations.

Resilient Distributed Datasets(RDD)

RDD is the fundamental data structure of spark and is the Spark's main programming abstraction. It is immutable and fault tolerant collection of elements or objects that can be processed in parallel across the cluster of machines.

RDDs are called "resilient" because they track data lineage information so that lost data can be rebuilt if there is a failure, making RDDs highly fault-tolerant.

Once created, a RDD cannot be changed. Transformations are applied to RDD to create new RDD leaving the original one unchanged.

RDD Operations

There are two types of operations that can be performed on RDD:

Figure: Transformations and Actions

1. Transformations
Transformations are the operations that create new RDD from existing one.
As mentioned earlier, RDD are immutable but, we can create new RDD by applying the transformation. It takes RDD as input and produces one or more RDD as output. Transformations are lazy in nature i.e., they get execute when we call an action. They are not executed immediately. Two most basic type of transformations is a map(), filter().
Transformations can be Narrow Transformation or Wide Transformation.

A. Narrow Transformation

This type of transformation doesn't require data to be shuffled across different partitions. Each input partition produces one output partition and there is no need of communication between the nodes in the cluster for the operations.
It is more efficient as there is less network overhead.

Figure: Narrow Transformation

(Source: Data Flair)

B. Wide Transformation

This type of transformation require shuffle of data between different partitions. Operation depends on data from the multiple partitions and needs to be redistributed across the cluster.
It has higher latency and less efficient execution due to disk I/O and network involved in shuffling.

Figure: Narrow Transformation
(Source: Data Flair)

2. Actions
Actions trigger the actual computation of RDD. When the action is triggered after the result, new RDD is not formed like transformation. Thus, Actions are Spark RDD operations that give non-RDD values. Results are returned to driver or write data to external storage systems.

Job, Stage, and Task

Job

A job is the sequence of transformations on data. It can be thought of as the total work that your Spark application needs to perform, broken down into a series of steps.

Stage

When a job is divided, it results in the creation of multiple stages. A stage is a sequence of transformations executed in a single pass (i.e., without shuffling of data).

Each stage comprises tasks that all perform the same computation.

Task

A task is the unit of execution that runs on a single machine. It is associated with the number of partitions.

Lazy Evaluation

Lazy Evaluation is a powerful feature in Apache Spark that allows Spark to optimize the execution of data processing pipelines.

It means that Spark delays the execution of operations until it absolutely needs to perform them (i.e., until an action is called).

How Lazy Evaluation Works

In Spark, transformations like map(), filter(), and flatMap() are lazy. They don’t immediately execute; rather, they are recorded as transformations in a DAG (Directed Acyclic Graph).
Spark builds up a series of transformations on the data, but doesn't compute anything until it encounters an action like collect(), count(), or save().
When an action is called, Spark looks at the entire DAG of transformations, optimizes the execution plan, and then runs the tasks in the most efficient order.

Spark Architecture

1. Driver Program
It is the central co-ordinator of the spark application. It contains the Spark Context, which is the entry point to the Spark's execution environment.

What it does?

Job Scheduling
Task Distribution
Fault Tolerance

2. Cluster Manager
It is responsive for allocating resources across the cluster and decides which worker node your application gets to use. Examples are YARN, Mesos, etc.

3. Executors
These are the worker processes that run on the worker nodes. It is responsible for executing the tasks assigned to it by driver. It then reports the result back to the driver.

4. Worker Nodes
Worker nodes are the machines in the cluster where executors run.

Execution Flow of Apache Spark

1. App Submission

When a user submits an application, a driver program is launched.

Driver communicates with the cluster manager to request resources.

2. Job Creation and DAG Creation

Driver converts user code into jobs, which are divided into stages and further into tasks.

Driver creates a logical DAG representing the sequence of stages and tasks.

3. Stage Division and Task Scheduling

DAG Scheduler divides the DAG into stages, each with multiple tasks.

Task Scheduler assigns tasks to executors based on available resources and data.

4. Task Execution on Worker Nodes

Executors run tasks on worker nodes, process data, and return results to the driver.

Driver aggregates and presents the result to the user.

Speculative Execution in Spark

In a large spark job, tasks are distributed across many nodes in a cluster. Sometimes, a few tasks may runs significantly slower than others due to the issues like hardware degradation, network congestion, data skew, resource contention.
So what speculative execution in spark does is that it launches duplicate copies of slow-running tasks on different nodes under the assumption that these new nodes may complete faster than the older one. Then, whichever task finishes first wins, and the result of others is discarded.

Conclusion

In conclusion, Apache Spark is a powerful, open-source distributed data processing engine that has revolutionized big data analytics. Its ability to handle vast amounts of data in real-time, coupled with its fault tolerance, scalability, and speed, makes it an ideal choice for various industries dealing with massive datasets.

PawPal: Automate Your Dog’s Daily Routine with One Prompt Using Runner H

Sujan Koirala — Mon, 07 Jul 2025 06:40:31 +0000

This is a submission for the Runner H "AI Agent Prompting" Challenge

What I Built

PawPal is a Runner H-powered AI agent that manages the entire daily care routine for your pet — without needing a single line of code. It automatically plans when to feed and walk your dog, taking into account local weather and your calendar availability, and updates your pet sitter via email when you're unavailable.

Instead of manually juggling schedules, emails, and logs, PawPal handles everything from one single prompt — making pet care consistent, reliable, and intelligent.

Demo

With single prompt, Runner H starts the workflow.

Result presented by Runner H

Create logs in Google Docs

Schedules updated in Google Calendar

Email sent to the caretaker automatically

How I Used Runner H

I connected Runner H with these tools:

Google Calendar – To schedule feeding and walking sessions for my dog Jessica.
Gmail – To notify my pet sitter when I’m unavailable.
Google Docs – To log the complete pet care schedule and updates.

Then, I ran this single prompt to automate the entire workflow:

“Take care of my dog Jessica today. Check the local weather and schedule walking times accordingly in my Google Calendar (preferably when it’s not too hot or raining). Also, add feeding times at 8 AM and 6 PM to the calendar. If I have overlapping events and can't manage Jessica, email my dog-sitter at xyz@gmail.com with today's full pet care schedule. Finally, log the entire care routine (feeding + walking times, weather note, and sitter update status) in the Google Doc connected to my account.”

Use Case & Impact

PawPal is perfect for:

Busy pet owners with packed calendars.
People who travel and want their sitter informed without daily effort.
Elderly owners who need automated reminders and logs.

Impact:

Removes the mental load of remembering pet care tasks.
Prevents missed walks or meals.
Provides backup coverage (via email) automatically.
Keeps all actions transparently logged in a Doc.

It’s like having a smart assistant for your dog, operating quietly in the background every day.

Social Love

// Detect dark theme var iframe = document.getElementById('tweet-1942113446906015841-693'); if (document.body.className.includes('dark-theme')) { iframe.src = "https://platform.twitter.com/embed/Tweet.html?id=1942113446906015841&theme=dark" }

MovieMonk: An AI-Powered Movie Recommendation App Using MindsDB Knowledge Base + Agent

Sujan Koirala — Mon, 30 Jun 2025 16:47:25 +0000

🎯 Why I Built MovieMonk

I’m a movie maniac. But scrolling endlessly through streaming platforms or generic recommendation lists? Not fun.

That’s why I built MovieMonk, an AI-powered movie discovery app using MindsDB that lets you ask natural language questions like:

Recommend a thrilling action time travel movie with high ratings from the last 5 years.

🧠 What is MindsDB?

MindsDB is an AI data platform that lets you query data with natural language and build smart agents on top of structured and unstructured sources.

In June 2025, MindsDB introduced Knowledge Bases — a way to semantically index and query documents, similar to vector search — but SQL-native and way more developer-friendly.

🤖 Why MindsDB?

When building an intelligent app, most of the time you get stuck juggling:

🔢 Embedding models
🧠 Vector databases
🔍 Search logic
📦 Data pipelines
🤯 Prompt engineering

That’s a lot of moving parts for one simple goal: query your data intelligently.

With MindsDB, all of this is unified in one SQL-like interface.

What I Did?

Here's a quick overview of what I built:

Prepared a movie dataset with metadata like Title, Genre, Overview, Release Date, Ratings, etc.
Created a MindsDB Project and uploaded the movie data into a table using the Python SDK.
Built a Knowledge Base (KB) from that table using Title, Genre, and Overview for semantic search, and added metadata columns like Vote_Average, Popularity, and Release_Date.
Created a Movie Agent using CREATE AGENT with a custom prompt so it talks like a friendly movie expert.
Evaluated the Knowledge Base using EVALUATE KNOWLEDGE_BASE to check the quality of the results.

🧱 Challenges

Here’s a list of "Problems Faced" based on your context:

❌ Encountered error with version='llm_relevancy' during evaluation, so had to switch to version='doc_id'.
🧪 Needed to create a custom test dataset manually for evaluation purposes.
⚙️ Faced initial confusion while setting up and managing Jobs.
📚 Had no prior experience using a knowledge base, so had to learn its structure and usage (though MindsDB simplified the process).

🎬 Use Cases

MovieMonk is just the beginning — here are other ways you could use MindsDB KBs + Agents:

Streaming platform assistant: Recommend shows/movies based on user taste
Smart filtering bot: Let users find media with filters like genre, language, and popularity
Media search tool: Let analysts or journalists search through a movie database semantically
Personal movie planner: Build a chatbot that plans your weekend watchlist
Trivia or quiz bots: Use movie data to power engaging Q&A experiences

🧠 Final Thoughts

Building with MindsDB was a refreshing experience.

It simplified complex tasks like vector search, agent creation, and prompt engineering — all within a single platform. I didn’t have to worry about separate vector databases, embeddings, or model integration.

With just Python, SQL, and a few lines of config, I built a fully functional AI-powered movie app.

MovieMonk proves how easy it is to turn a plain CSV file into a smart, conversational experience.

If you're thinking about building anything with your own data — whether it's books, reports, research papers, or customer feedback — MindsDB’s Knowledge Base + Agent combo is a powerful toolkit worth exploring.

Make your Day with Daytona

Sujan Koirala — Fri, 10 Jan 2025 11:38:25 +0000

The Challenge: A Tale of Technical Debt 😓

Just a few days ago, I participated in Online Hackathon to put myself into test. Like many developers, I was excited to dive straight into coding and bring my ideas to life. However, reality hit hard when I realized that setting up my development environment was eating into my precious hackathon time. The hackathon project required a complex stack:-

React frontend with specific Node.js version requirements
Python backend with multiple dependencies
Multiple API integrations.

With only 48 hours on the clock, I spent the first 3 hours just wrestling with environment setup. Different team members were running different operating systems, and we kept running into version conflicts and dependency issues. It was frustrating to watch valuable hackathon time slip away on setup tasks instead of building features.

The breaking point came when one team member couldn't get the development environment running at all, forcing them to spend hours debugging instead of contributing to the project. We needed a better way to ensure everyone could start coding quickly and have identical development environments regardless of their local setup.

Enter Daytona: The Game-Changer ✨

After struggling with the setup for weeks, I discovered Daytona through a colleague's recommendation. The promise of standardized development environments that "just work" seemed too good to be true. Spoiler alert: it wasn't.

Daytona is an open-source Development Environment Manager (DEM) licensed under the Apache License 2.0. Daytona streamlines the entire workflow by automating instance provisioning, configuration interpretation and application, prebuild setup, secure VPN connection establishment, seamless integration with your local or Web IDE, and assigning a fully qualified domain name to the development environment for effortless sharing and collaboration.

This allows developers to dive straight into what truly matters—their code.

daytonaio / daytona

The Open Source Dev Environment Manager.

The Open Source Development Environment Manager

Set up a development environment on any infrastructure, with a single command

Documentation · Report Bug · Request Feature · Join Our Slack · Twitter

Features

Single Command: Activate a fully configured development environment with a single command.
Runs everywhere: spin up your development environment on any machine — whether it's local, remote, cloud-based, physical server, or a VM & any architecture x86 or ARM.
Configuration File Support: Initially support for dev container, ability to expand to DevFile, Nix & Flox (Contributions welcome here!).
Prebuilds System: Drastically improve environment setup times (Contributions welcome here!).
IDE Support : Seamlessly supports VS Code & JetBrains locally, ready to use without configuration. Includes a built-in Web IDE for added convenience.
Git Provider Integration: GitHub, GitLab, Bitbucket, Bitbucket Server, Gitea, Gitness, Azure DevOps, AWS CodeCommit, Gogs & Gitee can be connected, allowing…

View on GitHub

Setting Up Sample React Project using Daytona

I will walk you through setting up a daytona project for ReactJS app.

The Tech Stack I Used 🛠️

My project involved:

React
Tailwind
Express JS
Mistral and Llama(Groq)

Installation and Initial Setup 🗺️

First, I installed Daytona using the following official documentation. Make sure you have Docker Desktop pre-installed.

Installation | Daytona

Learn how to install Daytona on Linux, macOS, and Windows.

daytona.io

Next, I created a devcontainer.json file inside the .devcontainer folder. Here's what our .devcontainer.json looked like:

{
    "name": "React Dev Container",
    "image": "mcr.microsoft.com/devcontainers/javascript-node:18",
    "features": {
      "python": "3.10"
    },
    "forwardPorts": [
      5173
    ],
    "postCreateCommand": "npm install"
  }

You may make use of Devcontainer.ai help generate the initial file and then change it as per your requirement. Finally, I pushed .devcontainer folder to my GitHub repo where rest of my project is.

Creating a workspace

Now you can create workspace with just single command.

daytona create https://github.com/daytonaio/sample-react-job-assistant

Now you are ready to go 🎉 !!

The Benefits I Saw 🌟

Consistent Environments: No more "works on my machine" syndrome. Every team member had identical setups.
Rapid Onboarding: New team members could start contributing within minutes instead of days.
Resource Efficiency: Daytona's workspace management meant we weren't wasting local resources on multiple development environments.
Version Control: Our development environment configuration was version-controlled alongside our code.

Tips for Success 💡

Start Small: Begin with a basic configuration and iterate.
Document Everything: Keep a changelog of your Daytona setup.
Use Pre-builds: They significantly reduce workspace creation time.
Leverage VS Code Remote: It works beautifully with Daytona.

The Impact 📈

After implementing Daytona:

Development environment setup time decreased from 4+ hours to 15 minutes
Onboarding new developers now takes 1 hour instead of 2-3 days
Zero environment-related issues

Community and Support 🤝

The Daytona community has been incredibly helpful. Don't hesitate to Join their Slack channel

Conclusion 🎉

Daytona transformed our development workflow from a constant struggle with environment issues to a smooth, efficient process. For teams dealing with complex setups or legacy applications, it's a game-changer.

P.S. If you're struggling with similar challenges, feel free to reach out or check out my project's public documentation. Together, we can make development workflows better for everyone! 🚀

References

Source Code to my project

Building Personalized Assistance of Ex-Convicts: Ex-cons Thrive

Sujan Koirala — Mon, 30 Sep 2024 11:53:07 +0000

Ex-Cons Thrive is a project designed to help formerly incarcerated individuals reintegrate into society by providing job search assistance, legal support, mental health services, and more. This project draws inspiration from the hardships faced by ex-convicts during reintegration, including the stigma, lack of opportunities, and legal barriers that often limit their options.

Key Features

Key Features of Ex-Cons Thrive

Personalized Job Search for Ex-Convicts
- Users can input their job experience, skills, legal restrictions, interests, and rehabilitation programs to receive tailored job recommendations.
- AI-generated job listings with detailed descriptions, helping users find opportunities that suit their background.
AI-Powered Legal Support
- Provides categorized legal guidance on common issues faced by ex-convicts.
- Uses AI models like Llama and Mistral to offer contextual responses to predefined legal questions, making it easier to understand complex legal processes.
Mental Health Support Chatbot
- Emotional support chatbot powered by Groq API for two AI models to help users manage stress, mental health, and other emotional challenges they may face during reintegration.
- Provides personalized suggestions and coping strategies based on user inputs.

How Ex-Con Thrive Leverages Multiple AI Models

Llama3-8b-8192

This model is designed for high-speed token processing and excellent performance. It excels in performing tasks requiring faster response times, making it ideal for providing real-time job recommendations based on user input (experience, skills, legal restrictions, etc.).

Mixtral-8x7b-32768

This model is slower compared to Llama3, processing ~80.7 tokens/second. However, providing more creative, contextually rich, and nuanced answers is its strength, which makes it ideal for legal support and mental health features.

Performance Comparison

Model	Speed (tokens/sec)	Best Use Case	Strengths
Llama3-8b-8192	212.7	Job Recommendations	Fast, efficient, ideal for generating multiple results.
Mixtral-8x7b-32768	80.7	Legal Support, Mental Health	Deep, context-aware, ideal for nuanced, complex responses.

Challenges Faced While Working with APIs

While developing the Ex-Cons Thrive application, I faced several challenges while integrating and utilizing multiple AI APIs. Here are some of the major issues I faced:

API Response Formatting:

One of the major challenges was ensuring that the responses from the APIs were properly formatted and structured. The responses from the Groq Cloud API, particularly when generating job suggestions, sometimes returned unexpected formats. I spent almost an hour for maintaining suitable format.
Rate Limiting:

Managing API rate limits was another hurdle. During peak testing times, I experienced instances where requests to the Groq API were throttled, leading to delayed responses or failed requests.
Integration Complexity:

Integrating two different AI models into a cohesive user experience was complex.

Though these challenges were difficult, I found it very rewarding to overcome them. Each of these obstacles not only enhanced my problem-solving skills but also deepened my understanding of how to effectively work with AI APIs.

Conclusion

Ex-Cons Thrive aims to benefit ex-convicts by providing personalized job assistance and support through the use of advanced AI models. The combination of Llama3-8b-8192 and Mixtral-8x7b-32768, facilitated by Groq Cloud, ensures that users receive tailored job suggestions based on their unique experiences, skills, and needs and mental as well as legal support.

By addressing the specific challenges faced by ex-convicts, this project not only aims to improve employment opportunities but also seeks to develop a supportive community.