DEV Community: Gitroom

The project I built with the Twitter API, GPT4 & CopilotKit…🚀🧑‍💻

Nevo David — Thu, 08 Aug 2024 10:58:53 +0000

TL;DR

Last week I built an application that automates my social media posting using AI. I will show you how I did it.

We will cover how to:

add Twitter authentication to a Next.js application,
create a calendar-like interface from scratch,
integrate AI assistants into software applications with CopilotKit,
create action-specific AI copilots to handle various tasks within the application, and
build a post generator and scheduling application.

This project is a great way to learn how to build AI-powered apps and master social media APIs, but don't use it to be mean 😈

CopilotKit: The framework for building in-app AI copilots

CopilotKit is an open-source AI copilot platform. We make it easy to integrate powerful AI into your React apps.

Build:

ChatBot: Context-aware in-app chatbots that can take actions in-app 💬
CopilotTextArea: AI-powered textFields with context-aware autocomplete & insertions 📝
Co-Agents: In-app AI agents that can interact with your app & users 🤖

Star CopilotKit ⭐️

Prerequisites

To fully understand this tutorial, you need to have a basic understanding of React or Next.js.

We'll also make use of the following:

CopilotKit - an open-source copilot framework for building custom AI chatbots, in-app AI agents, and text areas.
Redis - an in-memory database for storing the post schedule.
BullMQ - a Node.js library that manages and processes jobs in a queue.
Node Cron - a Node.js library that schedule and runs tasks (jobs) at specific intervals.
Headless UI - for creating accessible UI components for the application.
X Client ID and Secret - for authenticating users and creating posts on their behalf.
OpenAI API Key - to enable us to perform various tasks using the GPT models.

Project Set up and Package Installation

First, create a Next.js application by running the code snippet below in your terminal:

npx create-next-app social-media-scheduler

Select your preferred configuration settings. For this tutorial, we'll be using TypeScript and Next.js App Router.

Next, install the project dependencies:

npm install @headlessui/react lodash bullmq ioredis node-cron

Finally, install the required CopilotKit packages. These packages enable us to use AI auto-completion within the application, allow the AI copilot to retrieve data from the React state, and make decisions within the application.

npm install @copilotkit/react-ui @copilotkit/react-textarea @copilotkit/react-core @copilotkit/backend

Congratulations! You're now ready to build the application.

Building the Posts Scheduler App with Next.js

In this section, you'll learn how to create the user interface for the scheduling application. The application is divided into two pages: the Login page and the Dashboard page, where users can create and schedule posts.

The Login page authenticates users using their X (Twitter) profile, while the Dashboard page allows users to create, delete, and schedule posts.

The Login Page

The Login page represents the application's home page. Users need to sign in with their Twitter account to access the dashboard.

To implement this, update the page.tsx file to display a sign-in button as shown below:

import Link from "next/link";
import { getTwitterOauthUrl } from "@/app/util";

export default function Home() {
    return (
        <main className='w-full min-h-screen flex flex-col items-center justify-center p-8'>
            <h2 className='font-semibold text-2xl mb-4'>Your AI Post Scheduler</h2>
            <Link
                href={getTwitterOauthUrl()}
                className='bg-black py-3 px-6 hover:bg-gray-700 text-gray-50 rounded-lg'
            >
                Sign in with Twitter
            </Link>
        </main>
    );
}

The code snippet above displays a Sign in with Twitter button that redirects users to Twitter Oauth2 page. You'll learn how to setup the Twitter authentication shortly.

The Dashboard Page

Before we proceed, create a types.d.ts file at the root of the Next.js project. This file will contain the type declarations for the variables within the application.

interface DelSelectedCell {
    content?: string;
    day_id?: number;
    day?: string;
    time_id?: number;
    time?: string;
    published?: boolean;
    minutes?: number;
}
interface SelectedCell {
    day_id?: number;
    day?: string;
    time_id?: number;
    time?: string;
    minutes?: number;
}

interface Content {
    minutes?: number;
    content?: string;
    published?: boolean;
    day?: number;
}

interface AvailableScheduleItem {
    time: number;
    schedule: Content[][];
}

Create a utils file within the Next.js app folder and copy this code snippet from the GitHub repository into it. It contains the necessary functions for performing various data manipulations within the application.

Next, create a dashboard folder containing a page.tsx file within the Next.js app directory.

cd app
mkdir dashboard && cd dashboard
touch page.tsx

Copy the code snippet below into the dashboard/page.tsx file. It renders an App component that accepts the application's schedule as props and displays them in a table:

"use client";
import _ from "lodash";
import { useState } from "react";
import App from "@/app/components/App";
import { availableSchedule } from "../util";

export default function Dashboard() {
    //👇🏻 saves a deep copy of the availableSchedule array into the React state
    const [yourSchedule, updateYourSchedule] = useState<AvailableScheduleItem[]>(
        _.cloneDeep(availableSchedule)
    );

    return (
        <App yourSchedule={yourSchedule} updateYourSchedule={updateYourSchedule} />
    );
}

Here is the data structure for the table above:

export const tableHeadings: string[] = [
    "Time",
    "Sunday",
    "Monday",
    "Tuesday",
    "Wednesday",
    "Thursday",
    "Friday",
    "Saturday",
];

export const availableSchedule: AvailableScheduleItem[] = [
    {
        time: 0,
        schedule: [[], [], [], [], [], [], []],
    },
    {
        time: 1,
        schedule: [[], [], [], [], [], [], []],
    },
    {
        time: 2,
        schedule: [[], [], [], [], [], [], []],
    },
    {
        time: 3,
        schedule: [[], [], [], [], [], [], []],
    },
    {
        time: 4,
        schedule: [[], [], [], [], [], [], []],
    },
    {
        time: 5,
        schedule: [[], [], [], [], [], [], []],
    },
    {
        time: 6,
        schedule: [[], [], [], [], [], [], []],
    },
    {
        time: 7,
        schedule: [[], [], [], [], [], [], []],
    },
    {
        time: 8,
        schedule: [[], [], [], [], [], [], []],
    },
    {
        time: 9,
        schedule: [[], [], [], [], [], [], []],
    },
    {
        time: 10,
        schedule: [[], [], [], [], [], [], []],
    },
    {
        time: 11,
        schedule: [[], [], [], [], [], [], []],
    },
    {
        time: 12,
        schedule: [[], [], [], [], [], [], []],
    },
    {
        time: 13,
        schedule: [[], [], [], [], [], [], []],
    },
    {
        time: 14,
        schedule: [[], [], [], [], [], [], []],
    },
    {
        time: 15,
        schedule: [[], [], [], [], [], [], []],
    },
    {
        time: 16,
        schedule: [[], [], [], [], [], [], []],
    },
    {
        time: 17,
        schedule: [[], [], [], [], [], [], []],
    },
    {
        time: 18,
        schedule: [[], [], [], [], [], [], []],
    },
    {
        time: 19,
        schedule: [[], [], [], [], [], [], []],
    },
    {
        time: 20,
        schedule: [[], [], [], [], [], [], []],
    },
    {
        time: 21,
        schedule: [[], [], [], [], [], [], []],
    },
    {
        time: 22,
        schedule: [[], [], [], [], [], [], []],
    },
    {
        time: 23,
        schedule: [[], [], [], [], [], [], []],
    },
];

The tableHeadings array contains the headings for the table columns, while the availableSchedule array holds a group of objects. Each object has a time property representing each hour of the day and a schedule property containing a nested array, with each element representing a day of the week.

For example, when a user sets a schedule for Wednesday at 8 AM, the application searches for the object with a time property of 8 and updates its schedule property by inserting the schedule into the nested array at the fourth index.

You can copy the remaining UI elements for the Dashboard page from its GitHub repository.

In the upcoming sections, you'll learn how to add Twitter OAuth and CopilotKit to the application.

How to add X Authentication to your Next.js application

In this section, you’ll learn how to create a X Developer project and add X authentication your Next.js applications.

Ensure you have an X account and visit the X Developers' Portal to create a new project.

Enter the project name and provide answers to the required questions to create a new project and an app.

Set up the user authentication settings to allow you read and write posts on behalf of the users.

Finally, fill the App info section accordingly.

After setting up the authentication process, save the OAuth 2.0 Client ID and secret into a .env.local file.

TWITTER_CLIENT_ID=<your_client_ID>
NEXT_PUBLIC_TWITTER_CLIENT_ID=<your_client_ID>
TWITTER_CLIENT_SECRET=<your_client_Secret>

Authenticating users via X

Create an api folder within the Next.js app folder. Inside the api folder, create a twitter directory containing a route.ts file. This will create an API endpoint (/api/twitter) that enables us to authenticate users.

cd app
mkdir api && cd api
mkdir twitter && cd twitter
touch route.ts

Copy the code snippet below into the route.ts file:

import { NextRequest, NextResponse } from "next/server";

const BasicAuthToken = Buffer.from(
    `${process.env.TWITTER_CLIENT_ID!}:${process.env.TWITTER_CLIENT_SECRET!}`,
    "utf8"
).toString("base64");

const twitterOauthTokenParams = {
    client_id: process.env.TWITTER_CLIENT_ID!,
    code_verifier: "8KxxO-RPl0bLSxX5AWwgdiFbMnry_VOKzFeIlVA7NoA",
    redirect_uri: `http://www.localhost:3000/dashboard`,
    grant_type: "authorization_code",
};

//👇🏻 gets user access token
export const fetchUserToken = async (code: string) => {
    try {
        const formatData = new URLSearchParams({
            ...twitterOauthTokenParams,
            code,
        });
        const getTokenRequest = await fetch(
            "https://api.twitter.com/2/oauth2/token",
            {
                method: "POST",
                body: formatData.toString(),
                headers: {
                    "Content-Type": "application/x-www-form-urlencoded",
                    Authorization: `Basic ${BasicAuthToken}`,
                },
            }
        );
        const getTokenResponse = await getTokenRequest.json();
        return getTokenResponse;
    } catch (err) {
        return null;
    }
};

//👇🏻gets user's data from the access token
export const fetchUserData = async (accessToken: string) => {
    try {
        const getUserRequest = await fetch("https://api.twitter.com/2/users/me", {
            headers: {
                "Content-type": "application/json",
                Authorization: `Bearer ${accessToken}`,
            },
        });
        const getUserProfile = await getUserRequest.json();
        return getUserProfile;
    } catch (err) {
        return null;
    }
};

//👉🏻 API endpoint utilizing the functions above

From the code snippet above,
- The BasicAuthToken variable contains the encoded version of your tokens.
- The twitterOauthTokenParams contains the parameters required for getting the users' access token.
- The fetchUserToken function sends a request containing a code to Twitter's endpoint and returns the user's access token.
- The fetchUserData function uses the token to retrieve the user's X profile.

Add this endpoint below the functions. It accepts a code from the frontend when a user signs in and stores the user ID, username, and access token in a file that can be accessed when executing jobs on the server.

import { writeFile } from "fs";

export async function POST(req: NextRequest) {
    const { code } = await req.json();
    try {
        //👇🏻 get access token and the entire response
        const tokenResponse = await fetchUserToken(code);
        const accessToken = await tokenResponse.access_token;
        //👇🏻 get user data
        const userDataResponse = await fetchUserData(accessToken);
        const userCredentials = { ...tokenResponse, ...userDataResponse };

        //👇🏻  merge the user's access token, id, and username into an object
        const userData = {
            accessToken: userCredentials.access_token,
            _id: userCredentials.data.id,
            username: userCredentials.data.username,
        };
        //👇🏻 store them in a JSON file (for server-use)
        writeFile("./src/user.json", JSON.stringify(userData, null, 2), (error) => {
            if (error) {
                console.log("An error has occurred ", error);
                throw error;
            }
            console.log("Data written successfully to disk");
        });
        //👇🏻 returns a successful response
        return NextResponse.json(
            {
                data: "User data stored successfully",
            },
            { status: 200 }
        );
    } catch (err) {
        return NextResponse.json({ error: err }, { status: 500 });
    }
}

Update the dashboard/page.tsx to send the code to the API endpoint after authenticating a user.

import { useSearchParams } from 'next/navigation'
const searchParams = useSearchParams()
    const code = searchParams.get('code')

    const fetchToken = useCallback(async () => { 
            const res = await fetch("/api/twitter", {
                method: "POST",
                body: JSON.stringify({ code }),
                headers: {
                    "Content-Type": "application/json",
                },
            });
            if (res.ok) {
                const data = await res.json();
                console.log(data);
            }
    }, [code]);

    useEffect(() => {
        fetchToken();
    }, [fetchToken]);

Congratulations! When users click the Sign in with Twitter button, it redirects them to the Twitter authorisation page to enable them access the application.

How to add CopilotKit to a Next.js application

In this section, you'll learn how to add CopilotKit to the application to enable users to schedule posts automatically using AI copilots and also add auto-completion when creating post contents.

Before we proceed, visit the OpenAI Developers' Platform and create a new secret key.

Create a .env.local file and copy the your newly created secret key into the file.

OPENAI_API_KEY=<YOUR_OPENAI_SECRET_KEY>
OPENAI_MODEL=gpt-4-1106-preview

Next, you need to create an API endpoint for CopilotKit. Within the Next.js app folder, create an api/copilotkit folder containing a route.ts file.

cd app
mkdir api && cd api
mkdir copilotkit && cd copilotkit
touch route.ts

Copy the code snippet below into the route.ts file. The CopilotKit backend accept users’ requests and make decisions using the OpenAI model.

import { CopilotRuntime, OpenAIAdapter } from "@copilotkit/backend";

export const runtime = "edge";

export async function POST(req: Request): Promise<Response> {
    const copilotKit = new CopilotRuntime({});
    const openaiModel = process.env["OPENAI_MODEL"];

    return copilotKit.response(req, new OpenAIAdapter({model: openaiModel}));
}

To connect the application to the backend API route, copy the code snippet below into the dashboard/page.tsx file.

"use client";
import App from "@/app/components/App";
import _ from "lodash";
import {  useState } from "react";
import { availableSchedule } from "../util";

//👇🏻 CopilotKit components
import { CopilotKit } from "@copilotkit/react-core";
import { CopilotPopup } from "@copilotkit/react-ui";

//👇🏻 CSS styles for CopilotKit components
import "@copilotkit/react-ui/styles.css";
import "@copilotkit/react-textarea/styles.css";

export default function Dashboard() {
    const [yourSchedule, updateYourSchedule] = useState<AvailableScheduleItem[]>(
        _.cloneDeep(availableSchedule)
    );

    //👉🏻 other UI states and functions

    return (
        <CopilotKit runtimeUrl='/api/copilotkit/'>
            <App
                yourSchedule={yourSchedule}
                updateYourSchedule={updateYourSchedule}
            />
            <CopilotPopup
                instructions='Help the user create and manage ad campaigns.'
                defaultOpen={true}
                labels={{
                    title: "Posts Scheduler Copilot",
                    initial:
                        "Hello there! I can help you manage your schedule. What do you want to do? You can generate posts, add, and delete scheduled posts.",
                }}
                clickOutsideToClose={false}
            ></CopilotPopup>
         </CopilotKit>
    );
}

The CopilotKit component wraps the entire application and accepts a runtimeUrl prop that contains a link to the API endpoint. The CopilotKitPopup component adds a chatbot sidebar panel to the application, enabling us to provide various instructions to CopilotKit.

How to schedule posts with CopilotKit

CopilotKit provides two hooks that enable us to handle user's request and plug into the application state: useCopilotAction and useCopilotReadable.

The useCopilotAction hook allows you to define actions to be carried out by CopilotKit. It accepts an object containing the following parameters:

name - the action's name.
description - the action's description.
parameters - an array containing the list of the required parameters.
render - the default custom function or string.
handler - the executable function that is triggered by the action.

useCopilotAction({
    name: "sayHello",
    description: "Say hello to someone.",
    parameters: [
        {
            name: "name",
            type: "string",
            description: "name of the person to say greet",
        },
    ],
    render: "Process greeting message...",
    handler: async ({ name }) => {
        alert(`Hello, ${name}!`);
    },
});

The useCopilotReadable hook provides the application state to CopilotKit.

import { useCopilotReadable } from "@copilotkit/react-core";

const myAppState = "...";
useCopilotReadable({
  description: "The current state of the app",
  value: myAppState
});

Now, let’s plug the application state into CopilotKit and create an action that helps us to schedule posts.

Within the App component, pass the schedule state into CopilotKit. You can also provide additional information (context) to enable CopilotKit make adequate and precise decisions.

//👇🏻 Application state
useCopilotReadable({
    description: "The user's Twitter post schedule",
    value: yourSchedule,
});

//👇🏻 Application context
useCopilotReadable({
    description: "Guidelines for the user's Twitter post schedule",
    value:
            "Your schedule is displayed in a table format. Each row represents an hour of the day, and each column represents a day of the week. You can add a post by clicking on an empty cell, and delete a post by clicking on a filled cell. Sunday is the first day of the week and has a day_id of 0.",
    });

Create a CopilotKit action that schedule posts based on the user’s prompts:

    useCopilotAction({
        name: "updatePostSchedule",
        description: "Update the user's Twitter post schedule",
        parameters: [
            {
                name: "update_schedule",
                type: "object",
                description: "The user's updated post schedule",
                attributes: [
                    {
                        name: "time",
                        type: "number",
                        description: "The time of the post",
                    },
                    {
                        name: "schedule",
                        type: "object[]",
                        description: "The schedule for the time",
                        attributes: [
                            {
                                name: "content",
                                type: "string",
                                description: "The content of the post",
                            },
                            {
                                name: "minutes",
                                type: "number",
                                description: "The minutes past the hour",
                            },
                            {
                                name: "published",
                                type: "boolean",
                                description: "Whether the post is published",
                            },
                            {
                                name: "day",
                                type: "number",
                                description: "The day of the week",
                            },
                        ],
                    },
                ],
            },
        ],
        handler: ({ update_schedule }) => {
            setAddEventModal(true);
            setSelectedCell({
                day_id: update_schedule.schedule[0].day + 1,
                day: tableHeadings[update_schedule.schedule[0].day + 1],
                time_id: update_schedule.time,
                time: formatTime(update_schedule.time),
            });
            setContent(update_schedule.schedule[0].content);
            setMinute(update_schedule.schedule[0].minutes);
        },
        render: "Updating schedule...",
    });

The code snippet above shows the useCopilotAction hook in action. It accepts an object containing name, description, parameters, handler, and render properties.

The name property represents the name of the action.
The description property provides a brief overview of what the function does.
The parameters array contains an update_schedule object with a time and schedule property. The schedule object includes content, minutes, published, and day attributes.
The handler function describes the action to be carried out when triggered. In the example above, the handler function opens the AddPost modal, updates its value with AI-generated inputs, and allows the user adjust the schedule accordingly.

Managing and Scheduling Posts using Redis and BullMQ

In this section, you’ll learn how to store the post schedule in a Redis database, and create a job that checks the schedule at intervals to post the content on X (Twitter).

First, you need to install Redis on your computer. If you are using MacOS and have Homebrew installed, run the code snippet in your terminal to install Redis:

brew --version
brew install redis

Once the installation process is complete, you can test your Redis server by running the following code snippet in your terminal:

redis-server

Now, you can use the Node.js Redis client within the application.

Create an /api/schedule API route on the server that accepts the entire schedule table when a user adds or deletes a scheduled post.

import { NextRequest, NextResponse } from "next/server";

export async function POST(req: NextRequest) { 
    const { schedule } = await req.json();
    try {
         console.log({schedule})

        return NextResponse.json(
            { message: "Schedule updated!", schedule },
            { status: 200 }
        );

    } catch (error) { 
        return NextResponse.json(
            { message: "Error updating schedule", error },
            { status: 500 }
        );

    }   
}

Update the API endpoint to store the entire schedule in the Redis database. Redis stores data in key/value pairs, making it super-fast for storing and retrieving data.

import { NextRequest, NextResponse } from "next/server";
import Redis from "ioredis";
const redis = new Redis();

export async function POST(req: NextRequest) { 
    const { schedule } = await req.json();
    try {
        //👇🏻 saves the schedule
        await redis.set("schedule", JSON.stringify(schedule));

        return NextResponse.json(
            { message: "Schedule updated!", schedule },
            { status: 200 }
        );
    } catch (error) { 
        return NextResponse.json(
            { message: "Error updating schedule", error },
            { status: 500 }
        );
    }
}

You can also add a GET request handler within the api/schedule/route.ts file to fetch existing scheduled posts from the Redis database and display them when a user logs into the application.

export async function GET() {
    try {
        const schedule = await redis.get("schedule");
        if (schedule) {
            return NextResponse.json(
                { message: "Schedule found", schedule: JSON.parse(schedule) },
                { status: 200 }
            );
        }
    } catch (error) {
        return NextResponse.json(
        { message: "Schedule not found" },
        { status: 500 }
    );
}
}

Finally, you need to set up a job queue that runs every minute to check for posts scheduled for the current day and post them at the appropriate time.

Create a worker.ts file within the Next.js src folder and copy the following code into the file:

import data from "./user.json";
import { Worker, Queue } from 'bullmq';
import Redis from "ioredis";

//👇🏻 initializes a job queue connected to the Redis database
const redis = new Redis({maxRetriesPerRequest: null});
const scheduleQueue = new Queue('schedule-queue', { connection: redis });

The code snippet above creates a job queue that is connected to the Redis database.

Implement a scheduleJobs function within the worker.ts file that gets the posts scheduled for the current time and adds them to the job queue.

//👇🏻 add jobs to the queue
export const scheduleJobs = async (schedule: AvailableScheduleItem[]) => {
    //👇🏻 gets current time and day
    const now = new Date();
    const currentHour = now.getHours();
    const currentMinute = now.getMinutes();
    const currentDay = now.getDay();

    //👇🏻 gets posts for the current hour
    const currentSchedule = schedule.find((item) => item.time === currentHour);
    const schedulesForTheHour = currentSchedule?.schedule[currentDay];

    //👇🏻 gets scheduled posts for the current time
    if (schedulesForTheHour && schedulesForTheHour?.length > 0) {
        const awaitingJobs = schedulesForTheHour.filter(
            (scheduleItem) =>
                scheduleItem.minutes && scheduleItem.minutes <= currentMinute
        );

        //👇🏻 add jobs to queue
        return awaitingJobs.map(async (scheduleItem) => {
            const job = await scheduleQueue.add("jobs", {
                message: scheduleItem.content
            }, {
                removeOnComplete: true,
            });
            console.log(`Job ${job.id} added to queue`);
        });
    }
};

Import the scheduleJobs function into the api/schedule endpoint and trigger the function every minute using Node Cron.

//👉🏻 api/schedule/route.ts
import cron from "node-cron";

export async function POST(req: NextRequest) { 
    const { schedule } = await req.json();
    try {
        await redis.set("schedule", JSON.stringify(schedule));
        cron.schedule('* * * * *', async() => {
            console.log('Triggering jobs...');
            await scheduleJobs(schedule);
        });

        return NextResponse.json(
            { message: "Schedule updated!", schedule },
            { status: 200 }
        );
    } catch (error) { 
        return NextResponse.json(
            { message: "Error updating schedule", error },
            { status: 500 }
        );
    }
}

Next, add a worker function within the workers.ts file that executes the jobs within the queue by sending the posts’ content to X (Twitter).

//👇🏻 processing jobs
const scheduleWorker = new Worker('schedule-queue', async (job) => {
  console.log(`Processing job ${job.id} of type ${job.name} with data: ${job.data.message}`)
    console.log("Posting content...")

    //👇🏻 post content to X
    const postTweet = await fetch("https://api.twitter.com/2/tweets", {
        method: "POST",
        headers: {
            "Content-type": "application/json",
            Authorization: `Bearer ${data.accessToken}`,
        },
        body: JSON.stringify({ text: job.data.message })
    });
    if (postTweet.ok) { 
          console.log("Content posted!")
    }
}, { connection: redis})

//👇🏻 listening for completed job
scheduleWorker.on('completed', job => {
    console.log(`${job.id} has completed!`);
});

Finally, you can execute the worker by running npm run worker after updating the scripts within the package.json file.

{
 "scripts": {
    "worker": "npx tsx --watch src/worker.ts"
  }
}

Congratulations! You’ve completed the project for this tutorial.

Conclusion

So far, you’ve learned how to authenticate users via X, store data in a Redis database, create and manage jobs with Redis and BullMQ, and integrate AI assistants into your Next.js applications using CopilotKit.

CopilotKit is an incredible tool that allows you to add AI Copilots to your products within minutes. Whether you're interested in AI chatbots and assistants or automating complex tasks, simplifies the process.

If you need to build an AI product or integrate an AI tool into your software applications, you should consider CopilotKit.

You can find the source code for this tutorial on GitHub:

https://github.com/dha-stix/ai-post-generator-and-scheduler-with-copilotkit

Thank you for reading!

I 10x’d my Coding Productivity Using this AI Tool: I wish I had known it Earlier 🤯

Nevo David — Mon, 22 Jul 2024 14:48:39 +0000

I usually work on multiple projects simultaneously, and managing all these projects can become very hectic and overwhelming. So, I wanted a solution to reduce my workload.

Considering the significant progress in AI in the past few years, I was specifically looking for an AI tool that can

Access my GitHub repository.
Solves any given issue autonomously.
Tests the code and pushes the fix to the repository.

So, I started my research and stumbled upon quite a few solutions. Here is my experience working with these solutions.

But before that, let’s understand what AI and SWE (Software Engineering) agents are.

What are AI Agents?

AI agents are systems powered by AI models that can autonomously perform tasks, interact with their environment, and make decisions based on their programming and the data they process.

Components of AI Agents

An AI agent has three crucial components.

Inference Engine: Inference Engines are AI models responsible for planning and decision-making.
Memory: The memory component allows the agents to manage short-term and long-term information to keep track of the workflows.
Tools: Tools enable Agents to interact with the external environment. For instance, a GitHub tool allows the agent to extract information from repositories and make necessary changes.

Characteristics of an SWE Agent

So, what makes an AI agent a SWE agent?

A SWE agent is an AI agent that mimics the qualities and characteristics of a human software developer, such as

Long-term planning and reasoning
Use standard developer tools.
Improve code quality through testing and feedback.
Debug and resolve issues autonomously.
Collaborate with human counterparts as and when needed.

Simillar to any AI agent, an SWE agent has

A robust inference engine. Preferably state-of-the-art code models.
A memory component to keep track of events.
Specific tools for writing, debugging, and running codes (Isolated code environment, Internet, GitHub, File tools, etc.)

Devin - The AI Software Developer

Devin is the first thing you will find when discussing AI agents that can code. It is an SWE agent from Cognition Labs, which started it all. It garnered a lot of attention from developers, founders, and investors and quickly became the talk of the town.

However, a few things made me turn away from it.

Closed Source: It is a closed source. So, there is no scope for customization.
OpenAI only: It exclusively utilizes OpenAI models, considering Llama 3 400b launch. I wanted something more flexible.
Not Publicly Available: It is still on the waitlist, and few people can access it.

While enterprises may prefer it, I needed something more transparent and flexible.

Open-source Alternatives

The next obvious choice was all the open-source alternatives for Devin. While they are more transparent and flexible in terms of LLM providers. They still didn’t click with me.

Most open-source solutions are

Bloated: They ship with many things I might never use.
Complex: Running these tools felt like a job.
Non-extensible: I can’t extend their capabilities using external tools and integrations.

That being said, they certainly hold their own merits.

SweKit - Simple Extensible Software Engineering Agent Framework

After extensive searching and research, I was leaning towards building it myself until I stumbled upon this framework.

SweKit is an open-source framework from Composio that helps you create simple and extensible SWE agents.

It clicked all the criteria I was looking for.

Customizable: It is simple and highly customizable. You can use the LLM providers of your choice and change the prompts to suit your requirements.
Framework Agnostic: It works with all the popular agentic frameworks, such as LangChain and LlamaIndex.
Extensible: Adding more tools, such as browsing and scrapping, can easily extend the agent's capability. You can add tools from the Composio ecosystem.
Open-source: Needless to say, the code is open-source, so you can fork and add custom changes.
Multiple Language Support: SweKit has native support for Python and Typescript. You can go with the language of your choice.

SweKit can access any public or private repository, work on provided issues, and push changes to the repository.

The SWE agent runs in Docker by default for security and isolation. This sandboxes the agent's operations, protecting against unintended consequences of arbitrary code execution. You can also run it locally, but be cautious, as it won't have Docker's protective layer.

Final Words

I have already started using this in my projects, and I cannot stress enough how much time and energy it has saved me.

I spoke with the team building Composio, and I can confidently say this will be a game-changer.

I cannot star them enough, so do me a favour, visit the repository, use the product and give them a star.

show them support by starring the repo ⭐

I am suitable for the job but unsure why I can't get it! 😕

Nevo David — Mon, 20 May 2024 10:41:17 +0000

In 2020, I had just closed my previous startup and got tired of my part-time job at a project company. I was (and still am) a digital nomad and looked for remote jobs.

It was pre-COVID, and remote work was not as vital as today. Too many developers (primarily freelancers) looked for jobs and a few positions.

I couldn't find a job back there, and I couldn't even get an interview. With my skill set, I knew I would fit anywhere (I already had more than 7 years as a developer.)

Doing my own stuff

I started another startup called Linvo, and it went pretty well. I got customers and made money - but it didn't work in the end.

Back then, I was working in a co-working every day (9 am to 12 am) and signed unemployment to get money from the government 🙈

I was in the open space of the coworking space, so I had the option to personally connect with different founders—not to get a job but to talk about literally everything.

But as the conversation continued, people learned about my skills.

Hanging out in coworking spaces

I want to be humbled, but I was offered to work at almost any startup that worked in the open space, fully remote, in almost any term I wanted. So, I started to work at Novu for precisely 2 years.

Now, it's evident that it is easier to get a job when you connect with people personally (this is why startups go to events to sell their products). But I wanted to dig deeper into how I went from 0 to 100.

I made a previous article about contributing to open-source will 10x your chances of landing a new job

I stand by it. But there is more.

Everything is about risk in life - the more trust you have, the higher the chances somebody will hire you / use your services / buy software from you.

But trust is hard.

How can you get trust when you - write code?

What I say vs. What I am

Whenever I looked for a freelancer (for example, a frontend developer), I couldn't determine whether:

They have an excellent personal fit
They can actually write a good code
They can think out of the box.

Because all I could extract from them was a few words talking about themselves.

As an entrepreneur, I have shifted my job from a developer to developer growth (growth hacking) as I figured this is essential for a business to grow.

In my second year in Novu, I decided that I needed to build my portfolio of knowledge so people could learn anything possible about me - to gain their trust and show them my personality.

So, I created my Gitroom newsletter and YouTube channel so people can see my marketing skills, but also my Gitroom open-source so they can see my development skills.

And I was very consistent (showed up every day)—currently, the newsletter is on a 46-week streak (almost 1 year).

And share valuable information - this is where it's essential:

I don't teach simple code stuff like algorithms
I don't teach CSS and HTML

I focus on mainly seniority-based to get their trust.
Today, I have too many customers - I turned down a lot of jobs until I learned how to scale or get some missing skills.

Do this

Since most people reading the article are purely developers (not marketers/growth hackers), I would suggest a few things.

Contribute to open source — the best way to show off your skills. Nothing compares to an open-source contribution to a large codebase, which is most of the proof people need.
Amplify your contribution - Not everybody knows about your contribution; talk about it on every possible social media (X, Linkedin, Youtube, Daily.dev) to make people aware of what you have done.
PRO TIP: Tag the company to which you have contributed your code; they might re-post you.
Think about your audience - I see far too many "influencers" that focus on junior developers with things like (10 CSS tricks and 10 HTML tags you must know) and while this will tremendously grow your audience size because there are a lot more junior developers in the world than seniors. You will build the wrong audience. You will mark yourself with many open-source founders as a junior. It might be wrong even for a dev-rel position, as you create an audience that will never pay money.

Gitroom

I am going to inject some self-promotion that can help you :)

Gitroom is an open-source tool for social media scheduling.
Contributing to Gitroom can achieve the two points I discussed.

Contribute to a large code-base
Amplify your contribution by scheduling social media posts.

But there is also number three, that I think you will like.
Gitroom is the biggest community for open-source founders; you will literally hang out with them, and they are constantly looking for remote employees worldwide.

That's a great way to achieve some internal trust.

After you have contributed, join our Discord channel.
All the open-source founders are there.

Please Star Gitroom if you can

Sugar Coating

This article is not going to sugar-coat it for you.
If you want to get hired, you need to be good.

This is not a quick scheme to "trick" people into hiring you.
It's only a tool to amplify your knowledge and skills.

Don't expect to be hired if you can't contribute to a large code base.

Here is a conversation I had in my previous article:

If you are not good enough yet, work on your skills.
There are no magic tricks when it comes to the actual job.

🚀 The easiest way tech influencers and content writers can make money 💰

Nevo David — Tue, 14 May 2024 13:41:37 +0000

Over the last two years, I have spoken to more than 100 founders. They have all faced the same problem - they couldn't find good content writers and influencers.

On the flip side, I have encountered many content writers and influencers having a hard time:

Getting work
Managing their orders
Keep themselves safe by earning money

That's where Gitroom shines

TL;DR: it's free. You can register and earn money.

Why is it different?

Today, Gitroom is one of the most prominent open-source communities for open-source founders. Many of them are funded — their problem is not spending money but finding the right people to spend their money on.

Next week, we will have a big launch for the new writers and influencers for the community - so you want to be there.

Gitroom is really similar to Fiverr.

You list yourself on the platform. Founders contact you for promotion.
You send them an offer per post.
They pay for the offer, and their money will be held by Gitroom.
You start to create posts from inside the system (Yes, Gitroom is a scheduling tool, also)
They approve your post
Once it is published, Gitroom will automatically send you the funds.

The trick about Gitroom is that the posting goes from inside of Gitroom, so you are 100% protected.

How to start

Head over to the Gitroom platform and register.

Then, complete the onboarding; it's super essential that:

You add your name, bio, and profile picture.
Connect your Social media platform so you can offer them later.

Go into the marketplace as a seller, connect your bank account, and add your expertise.

I am done. What's now?

PLEASE MAKE SURE YOU ADD YOUR BANK DETAILS AND BUSINESS INFORMATION OR YOU WON'T SHOW UP IN THE LIST

Just wait for an email in your inbox about new offers.
Also, Gitroom is fully open-sourced.
So feel free to check the code and see if everything is okay.

You can also suggest new features / contribute some code :)

At the moment, you don't need to apply to be in the platform, but if I start to get a major amount of spam, I will change it.

Let me know what you think about the system ❤️

My social media scheduling tool is now also available at Postiz.

Contributing to open-source will 10x your chances to land a new job 🚀

Nevo David — Tue, 30 Apr 2024 11:06:16 +0000

I have been trying to get some programmers to work for me on Upwork lately. Every person who applied raised so many questions about whether they could actually handle my project.

I haven't hired any yet.

Adding a JOB to Upwork

I put a very simple job description with a few questions to answer:

and started receiving many unhelpful applications. I would not be surprised if many of them were created with ChatGPT.

And then got so many applications with unresolved questions:

How can I know they can handle my project?
How do I know they don't lie?
How can I know if they can handle it with a high-quality contribution?
How do I know they know how to communicate with me properly?

So many unanswered questions put a big risk in hiring any of the candidates. If they are not good, I can:

Lose time
Lose money
Lost my time (because I need to teach them)

I would need to interview and test every candidate (big headache)

Open source shines

Now, let's assume somebody contributed to my open-source repository.

Now I know:

Their level
Their ability to handle big projects
How they communicate

I can tell you that in Novu, many contributors have shifted their roles from contributors to employees.

Mistakes most developers do

I got an application from one person over GitHub. This is their GitHub profile:

Instead of making many contributions to different open-source repositories, they tried creating and applying their own demos.

That's a big mistake.

Every developer with 1+ years of experience can start something from 0; it's much easier to be in a controlled environment.

Actually, contributing to a large code base (and getting merged) is a different level.

Here is a message I got from the HackSquad discord:

"But beginners can't contribute to their gigantic code bases"

This is how you reduce the risk when applying for a job.

SO, INSTEAD OF CREATING 100 DEMO PROJECTS, FOCUS ON 1-2 CONTRIBUTIONS FOR A BIGGER CODEBASE PROJECT.

Be strategic with your contribution

Before you contribute to any big codebase project, check:

Is it a company?
Are they hiring?
Do they have an open position for what I am looking for?
Can I show this contribution to a similar company or in Upwork?

Project you should contribute to

All the listed projects are fully remote companies that hire everywhere.

1. Gitroom

I don't have a huge budget, but I am trying to offload some of the work. This is a good project if you want to:

Start with a mid-size project
Work with the biggest technologies out there, such as NextJS and NestJS
Be exposed to more open-source companies (because Gitroom) that support many open-source companies.

I have also opened a bunch of cool issues you can start with!

2. CopilotKit

Today, there is a new concept of "AI Engineer." It's mostly for people who are interested in building stuff with AI without building the AI themselves. Copilotkit is building chatbots that wrap existing LLMs. This is a good project if you want to:

Work with LLMs
Work with NextJS and, in the future, more frameworks
Do some cool AI shit :)

3. Winglang

While many people go to learn computer science, they work as developers and not engineers to program full-stack software; you don't have to learn CS (I barely finished high school). If you want to pursue a career as an engineer, This is a good project if you want to:

Build a transpiler that takes code and converts it to JS at the end
Write sophisticated algorithms
Build a deep infrastructure

4. LLMWare

I asked Santiago what the biggest trend in AI is today. He said that RAG and LLMs are the biggest hypes or mostly the future.

LLMWare are killers with everything connected to RAG. This is a good project if you want to:

Play with RAG
Play and fine-tune models
Be a Python professional

5. Novu

I worked for Novu, so I can tell you it's a good company :)
Their codebase is pretty big (3 years of coding). This is a good project if you want to:

They have already hired many community members
Easy codebase full-stack javascript
Huge community (it's just fun)

6. Taipy

I love Taipy. They build a unique Python app builder; this is a cool project because it's not as easy as full-stack JavaScript applications. This is a good project if you want to:

Learn Python
Showoff with a kick-ass contribution for a large codebase
Be a part of a big community.

If you are an open-source company also hiring, add your repository in the comment :)

Also, check out Postiz - the open-source social media scheduling tool.

I fine-tuned my model on a new programming language. You can do it too! 🚀

Nevo David — Thu, 25 Apr 2024 12:31:00 +0000

I have been using OpenAI ChatGPT-4 for a while now.
I don't have a lot of bad stuff to say about it.
But sometimes, it's not enough.

In Winglang, we wanted to use OpenAI and ChatGPT-4 to answer people's questions based on our documentation.

Your options are:

Use OpenAI assistant or any other vector-based database with (RAG). It worked nicely since Wing looked like JS, but there were still many mistakes.
Passing the entire documentation into the context window is super expensive.

Soon enough, we realized that was not going to work.
It's time to host our own LLM.

Your LLM dataset

Before we train our model, we need to create data that will be used to train the model. In our case, the Winglang documentation. I will do something pretty simple.

Extract all the URLs from the sitemap, set a GET request, and collect the content.
Parse it; we want to convert all the HTML into readable content.
Run it with ChatGPT 4 to convert the content into a CSV as the dataset.

It should be something like this:

Once you finish, save the CSV with one column named text and add the question and the answer. We will use it later. It should look something like this:

text
<s>[INST]How to define a variable in Winglang[/INST] let a = 'Hello';</s>
<s>[INST]How to create a new lambda[/INST] bring cloud; let func = new cloud.Function(inflight () => { log('Hello from the cloud!'); });</s>

Save it on your computer in a new folder called data.

Autotrain, your model

My computer is pretty weak, so I have decided to go into a smaller model - 7b parameters: mistralai/Mistral-7B-v0.1

There are millions of ways to train a model. We will use Huggingface Autotrain. We will use their CLI without running any Python code 🚀

When you use Autotrain from Huggingface, you can train it on your computer (my approach here) or train it on their servers (pay money) and train larger models.

I have no GPU with my old Macbook Pro M1 2021. thank you, Apple 🍎.

Let's install autotrain.

pip install -U autotrain-advanced
autotrain setup > setup_logs.txt

Then, all we need to do is run the autotrain command:

autotrain llm \
--train \
--model "mistralai/Mistral-7B-Instruct-v0.2" \
--project-name "autotrain-wing" \
--data-path data/ \
--text-column text \
--lr "0.0002" \
--batch-size "1" \
--epochs "3" \
--block-size "1024" \
--warmup-ratio "0.1" \
--lora-r "16" \
--lora-alpha "32" \
--lora-dropout "0.05" \
--weight-decay "0.01" \
--gradient-accumulation "4" \
--quantization "int4" \
--mixed-precision "fp16" \
--peft

Once finished you will have a new directory called "autotrain-wing" with the new fine-tuned model 🚀

Playing with the model

To play with the model, start by running:

pip install transformers torch

Once completed, create a new Python file named invoke.py with the following code:

from transformers import pipeline

# Path to your local model directory
model_path = "./autotrain-wing"

# Load the model and tokenizer from the local directory
classifier = pipeline("text-classification", model=model_path, tokenizer=model_path)

# Example text to classify
text = "Example text to classify"
result = classifier(text)
print(result)

And then you can run it by running the CLI command:

python invoke.py

And you are done 🚀

Keep on working on your LLMs

I am still learning about LLMs.
One thing I realized is that it's not so easy to track changes with your models.

You can't really use it with Git because a model can reach a very large size > 100 GB; it doesn't make much sense - git doesn't handle it nicely.

A better way to do this is with a tool called KitOps.

I think it will soon be a standard in the world of LLM, so make sure you star this library so you can use it later.

Download the latest KitOps release and install it.
Go to the model folder and run the command to pack your LLM:

kit pack .

You can also push it to Docker hub by running

kit pack . -t [your registry address]/[your repository name]/mymodelkit:latest

💡 To learn how to use DockerHub check this

⭐️ Star KitOps so you can find it again later ⭐️

I started a new YouTube channel mostly about open-source marketing :)

(Like how to get Stars, Forks and Client)

If that's something that interests you, feel free to subscribe to it here:
https://www.youtube.com/@nevo-david?sub_confirmation=1

React Crazy: This AI library transformed my app to the next level

Nevo David — Thu, 11 Apr 2024 13:08:04 +0000

I have been building tons of applications.
My latest one is Gitroom. It helps you to:

Schedule your posts on multiple social media platforms and cross-post them to publishing platforms such as DEV, Medium, and Hashnode.
Track your GitHub repository metrics
Monitor GitHub trendings

(it's open-sourced 🙏🏻)
That's nice and all, but here is the real kicker.

Add AI to your application

I see tons of AI libraries every day, and most of them are really cool, but I ask myself, will they help me in real life?

Yes, babyagi might be the next big thing. But honestly, I don't see anything I can do with it.

So what can you do with AI, and what do people usually do?

I think in today's ChatGPT world and local LLMs, the use case is pretty simple:

Auto-complete content - perfect for Gitroom to let you generate additional information or use something like GitHub copilot.
Automation - You pass content to the LLM, which returns a JSON with operations you can perform.
Function calling - Mostly with ChatGPT, you pass different functions to the LLM, and they decide which one to run.
Miscellaneous - Classification, translation, etc.

I think "function calling" is not leveraged enough because adding it in a way that makes sense is complex.

Help your users with everything

I admit I love to build stuff, but I sometimes skip developing a good UX, which causes people to leave the system because they don't know what to do.

I think inherently, if you build systems for developers, you can get away with it - because they are more sophisticated.

But that is mostly not the case.
You should have:

Onboardings
Tutorials
YouTube videos (inside your system, yes yes)

This is where AI can save you without writing too much code.

Add a chatbot that can do everything for you

So, one of the things more and more companies offer is a chatbot that can interact with your data and perform actions for the user.

For example, I can tell the chatbot I want to schedule a new post for Gitroom at 7:00 p.m. US time.

And it will do it for me.

The problem with most of the solutions I found is that they are:

Not open-source
It is not an infrastructure product but a complete product that is mainly for product managers.

CopilotKit can help you

So, I found this incredible open-source library licensed under MIT - which is fantastic.

It's just a chatbot sitting on your right sidebar.

You feed it with:

Your current context information.
Your data resources, such as your database, dropbox, Google Drive, etc.
Different functions in the system can be called.

Once you have finished doing that, it's just like magic.
Does somebody want to schedule a post?

It will trigger the schedulePost function.
Of course, it uses the ChatGPT "function calling" in the background, but it simplifies everything.

I think it fits any app. You can use it with big LLMs like ChatGPT / Gemini, but also with smaller ones like LLAMA2 and Mistral.

Check out CopilotKit

I started a new YouTube channel mostly about Tech, open-source, and marketing :)

If that's something that interests you, feel free to subscribe to it here:
https://www.youtube.com/@nevo-david?sub_confirmation=1

I sold my startup because of bugs: I wish I had this serverless repository!

Nevo David — Thu, 04 Apr 2024 11:00:03 +0000

In 2021, I built a startup called Linvo, a LinkedIn automation tool. It generated good money but also had a high burn rate.

I can tell you that I lost most of my money as a young entrepreneur (entirely bootstrapped) 😆

The problem with my system is that it kept on failing; I am not the best programmer in the world, but let me elaborate.

Puppeteer sucks your memory

Linvo went into a few iterations during its lifetime. I used the tech that I know: React and NestJS.

Basically, any automated cloud scraping goes into making tasks and then solving them.

It looks something like this:

Cron runs every 10 minutes, looks for new stuff to do, and sends it to the queue.
Workers take tasks from the queue and process them.
Save everything to the DB.

This is a very naive solution that was running on multiple EC2 (AWS) and Redis.

In theory, it should work - Yes, you would need to scale the workers horizontally, but that was not my problem.

I started to experience something else - memory leaks.

As you probably know, Puppeteer - a chromium automated browser, is heavy (same as Chrome on your computer), and I was running lots of them - they sucked both my memory - causing the servers to stop working and took the internet bandwidth.

Needed a solution

Refactoring sounds like - yeah, let's do that, but my Intercom support was filled with support tickets (I already had many users), and I knew I had to create some crazy refactoring.

I had an idea for a solution - let's use AWS lambda.

Lambda start
Do the task
Die (most of the time)

It doesn't suck your internet.
It will die and not cause a memory leak.

It's a bit more expensive because Lambdas were not meant to run indefinitely—that would remove the whole concept, but I was desperate. (Well, the good thing is that you don't need to think about scale.)

Here is the new infrastructure:

Event bridge cron running every 1 minute (that's the minimum)
Lambda that creates six events for the queue (making it like the cron runs every 10 seconds)
SQS queue (bye bye, Redis)
Lambdas shooting out with auto-scaling (yay?)

Vendor Locking

As you probably understand from the context - now I am all in with AWS, but I didn't mind - only how I would test it on my computer.

Back in the day, I used Sam. It's almost the same as Serverless to create the entire environment quickly.

And yes, they do bring some local testing. They can take your lambdas and turn them into endpoints! Yay!

But I was using Event Bridge, SQS, and Lambdas. That was just not possible.

I have encountered localstack. It was supposed to help me build an entire AWS stack on my computer, but many things were not supported back then, such as Lambda layers. And I was overwhelmed in using it.

So I started to create my "own" infrastructure where I deployed one "test" SQS on AWS and ran local code with it (a function that calls a function), and it was - bad.

I did not work well in production; I felt like the word was squeezing up on me back in the day.

I just wanted to get rid of this startup; I even offered it to people for free, taking the customers and rid myself of that problem.

Long story short, I managed to sell the system for a low price, but lately, I discovered something interesting.

Winglang - Typescript like dialect for the cloud

I guess my pain was real, and even though I would be happier to find this back in the day - I am glad it's alive today.

Winglang helps you to build the same cloud infrastructure as I used, but the nice thing is:

It doesn't require you to use any configuration files. You basically define the entire infrastructure with code (you can check their entire preflight and inflight documentation)
You can run it locally - thank you, god, 🙏🏻🙏🏻🙏🏻🙏🏻 with all the infrastructure included.
You can deploy it to any cloud provider, and you don't have to use AWS. They can literally generate any Terraform or cloud formation for easy deployment.

I talked to the founder of Winglang, Elad (builder of the famous CDK), and he said something that made me twitch.

You can literally work on your projects when you are on an airplane and write tests for it 🤯

I can't give them enough stars myself - but do me a favor and star their repository and of course, use it for your next project.

https://github.com/winglang/wing

Final words

I am angry at Winglang. I wish they had done it sooner. Now they show up. When it's all about AI?

I started a new YouTube channel mostly about Tech, open-source, and marketing :)

If that's something that interests you, feel free to subscribe to it here:
https://www.youtube.com/@nevo-david?sub_confirmation=1

My new open-source repository to schedule all your content!

Nevo David — Tue, 26 Mar 2024 08:23:14 +0000

Hi there, community!
A few years ago, I shared my story of closing my previous startup and making it open-source.

Since then, I have worked for Novu, an open-source notification infrastructure, and I have actually learned a lot about the power of an open-source community.

I have written 74 posts on DEV that generated 1,182,911 views 🤗
Any by that making Novu one of the most popular open-source repositories (31k stars)

I started to talk about it

I created a newsletter called Gitroom (formerly GitHub20k) and wrote about everything I have done during the year, from posting actual-value tutorials on DEV to promoting the repository using influencers.

I have talked to more than 30 open-source maintainers per month! Helping them get visibility in their repository.

But there is one nothing I have never solved.
My tactics are very stressful. It was not randomly writing articles.
But I was writing them at the right time and promoting them on different Reddit and social media channels.

The problem is that your launch is stuck before the article is published—you can't time anything, so you have to do it during the week.

It becomes tough if other people are involved in the launch; I used to miss much of it. So, I decided to create the Gitroom Platform.

The Gitroom Platform

The Gitroom platform is the best tool for scheduling your open-source launches. It's 100% open-source! Check and star it here.
https://github.com/gitroomhq/gitroom

Tech Stack

NextJS (tailwind)
NestJS

Analytics

Starting with the basics, I wanted to know:

When is the most performant day (regarding the number of stars), and then cross-reference it to the launch week I have done.
Know when I am trending over GitHub (get a notification)
Know when the GitHub feed will be refreshed to schedule my content accordingly.

Launches Page

To launch, you need:

To have all the channels used in open-source from Social Media such as Reddit to publishing articles such as DEV.to
See the timeline for the next refresh so I can plan it accordingly.
Add comments and collaborate with my team before the launch happens.

Schedule Page

This is where the magic happens:

You can cross-post to multiple channels at the same time (say DEV, Hashnode, and Medium)
You can post to more complex channels, such as Reddit, where you must define flair and subreddit.
You can edit everything globally and then switch to a specific channel.
See the preview of how it will look in the existing channel.
Use links from previous scheduled posts before they are posted! You can also use it as a canonical link!

The community's future!

This is the first version of Gitroom, mainly for planning your launch. Here is what you should expect:

Exchange or buy posts from other community members and influencers! YES! Influencers will create posts for you, and they will appear on the timeline. What is the best thing? They will be paid only once the post is published (Good incentives)
Be part of the OSS friends, an API that lists all the open-source companies in the system - Great for SEO.
Add testimonials and “used by” with other open-source friends!

And Also:

Import from other platforms such as Notion—I personally write all my blog posts on Notion, but every time I copy them, I need to upload all the pictures manually.
More channels to schedule are available, such as Discord and Mastodon.
Automatic cross-posting (if you don’t want to use Gitroom scheduler)
Media generators, like the number of stars you have.

Help me out! 🤗
This is a new project, 100% open-source (first of his kind!)
Any star you can give me will help me grow it even more ❤️

Star it here ❤️

You can now find my schedule tool on Postiz.

Mirrord trick to get on hackernews

Nevo David — Wed, 10 Jan 2024 15:22:54 +0000

I had the pleasure of talking to Eyal @ CTO at Metalbear and the maintainer of Mirrord. I got some crazy insights.

You can watch the full video here:

Getting on Hackernews five times

The team at Mirrord had a secret to get into Hackernews.

They knew that if they published their post on Hackernews, the chances they would get in were rare (They have tried multiple times and failed.)

So they would go to Reddit and promote their content there, wait for a bit, and let somebody else with a higher Karma post their article on Hackernews.

But you have to understand that their content was super technical. It was not about building another “todo” app or how to use Kubernetes.

Here is an example of their post.

You can see that the person who posted them is the Founder/CEO of Pipedream with 130,687 karma.

They have managed to nail it like that multiple times.

Want a free trick? Copy their article to your tool and release it on various subreddits.

The Kubernetes trick that blew my mind

Honestly, I didn’t even know it was possible. They went into the Kubernetes blog and contributed a blog post (Their blog is open-source). They have made a comparison between their tool and other tools. This article still keeps sending them traffic daily - mind-blowing 🤯

Try it out, visit their website repository blog, and contribute your blog. Maybe you can even contribute to an existing one.

Running Google Ads

Honestly, every founder I talk to is usually unhappy with ads, but we all do it (at least try.) Mirrord ran ads mostly on their competitor’s names as it’s usually the cheapest and most targeted. I find ads very problematic because most developers have an ad blocker, so measuring the conversion is hard.

I recently ran my ads on Reddit and couldn’t use the pixel properly.

You want a platform that has a “conversion” API. Unfortunately, Reddit doesn’t have it yet - Twitter does.

And even with that, it’s hard to get it.

Developers tend not to click on ads and then search for the brand on Google (we have all done it)

I don’t recommend people run ads in general, only when you are in a scaling mode (you have maximized all the other channels)

I invite you to register for my newsletter.

This newsletter is good for you if:

You are considering open-sourcing your product (or building a new one).
You are considering opening a by-product and open-source it (to reflect on your main product).
You are in tech and want growth without the stars / without GitHub trending.

It’s a 100% free newsletter (and always will be). Feel free to register at:

https://gitroom.com

The Discord Plan for open-source in 2024

Nevo David — Wed, 03 Jan 2024 13:18:22 +0000

Hi everyone!

I wish you all a fantastic New Year 🥳

Mine is going to be stressful and fun. I hope yours is too!

Discord is a chat messaging platform you can run to interact with developers for your products. It can be used for marketing, support, or to build a strong community for your product.

During my time at Novu, we always tried to make the Discord server “active.”

This is something that I have experienced with many founders.

They want their Discord to be more than just a “support” channel.

Most founders aim to make the Discord channel active just from people engaging with each other, but how do you do it?

Here is a list of things you can try that worked well for me in Gitroom and HackSquad Discord servers.

1. Make people come back for more

I want to start with the simplest example - Midjourney.

Their main product lives on Discord; if you want to generate images, you must do it with the Discord bot. Whenever people come to create pictures, they are exposed to other people and interact with them. That was a genius move.

In Gitroom, I implemented a mechanism for people to launch their content. Since most people launch their content every week - they have to come back every week and, at the same time, interact with other people.

That’s why you usually see tons of activity in Gitroom every Monday.

2. Make a “non-product” Discord

When your Discord server is something like “Novu,” visitors of your Discord have the concept of “This is the Novu product. Let’s talk about that,” while this is what most products do, you don’t have to.

Our Discord server could be as well as “Notification Infrastructure.” Now, we have broadened our audience and given them a place to learn everything about notifications.

HackSquad was super active because it was a developer-based server where people could talk about anything.

3. Join with intent

Lately, I have experienced that people that I give a pitch over our Discord server during my calls are a lot more active than people who join on their own.

I think that’s the main secret in building an active community - and while you can’t talk to everybody, only a few are already pretty good because they will be active and involve other people.

Other options you can do are:

Give more information during the Discord “onboarding” wizard so people can understand what they can get by being active.
Show some content or a YouTube video next to the discord link so people can learn about the value of the server.
Tag people who join in different activities and ask for their opinions.

Do you like this blog post so far?

Make sure you register to the newsletter to read the next one

* Add your email, a video of getting the first 1,000 stars will be sent to your email

Or invite your friend to learn How to get GitHub stars

4. Retain

Bringing people to your Discord is a part of the “Acquisition” process, but making them stay is part of the “Retention” process.

In HackSquad, bringing people back to Discord makes them more active.

For example:

Instead of making a YouTube Live of a webinar or a Zoom Live, I would make a Discord Live and then send people back to Discord through the newsletter.
I run repeatable events that go beyond “How to use Novu.” I would actually do more generic stuff to attract more people.
I run Discord giveaways, making developers come back.
I constantly write on Social Media about the benefits of being a part of Discord.

5. Appoint moderators

The first thing I have done with Gitroom is find the first active and talented people on Discord and make them mods. That’s one of my best moves to make the community 10x more alive.

Once the community grows a bit, I intend to bring more mods.

In HackSquad, I appointed ten moderators in 1 month - Best Decision Ever.

I am so happy to have such talented people as Nathan and Saurabh, who help me grow the community and put it all together.

Mods automatically become more active when you give them “power”

6. Create an onboarding funnel

When somebody joins your Discord, that will be the time that they will be the most active and motivated.

You should use this time!

Ask them some questions on Discord and include them in some conversations to give some opinions.

Also, DM them. While this is not connected to being active on Discord - it’s an excellent time to understand your community members and extract some information from them:

Why did you join?
What are you building?
Who told you about us?
Why do you want to get from this community?

Try even to go for a 10-minute call if you can.

Here is the thing: you are not active 24/7, but if you have mods from around the world - they can help you!

Just make sure not all the mods DM the same person at the same time - that’s not nice 🙂

And last thing 🤣

Create a lot of bugs - that brings people in (I am kidding, of course.)

I invite you to register for my newsletter.

This newsletter is good for you if:

You are considering open-sourcing your product (or building a new one).
You are considering opening a by-product and open-source it (to reflect on your main product).
You are in tech and want growth without the stars / without GitHub trending.

It’s a 100% free newsletter (and always will be). Feel free to register at:

https://gitroom.com

How keepHQ got their first 2,000 stars!

Nevo David — Tue, 05 Dec 2023 11:51:23 +0000

I enjoyed talking to Tal, CEO and co-founder at Keep.

It started as a CLI tool and, over time, became an alert aggregation tool.

Today, they have almost 3,000 stars. You can watch the full video here:

Starting out

They have made a very basic alerting CLI tool and posted it on Hackernews - “Show HN:” They went on a flight, and when they landed, they saw 600 stars.

Hackernews is an interesting website. It’s super ugly, gets a lot of traction, and is very hard to get in.

I have seen and launched many products on Hackernews. While getting in for an article on Hackernews is hard, the “Show HN” one is much easier.

It’s usually a secret weapon since you can maybe do it once a year - it’s better to tie in with more channels to get a better chance to trend on GitHub.

They created more tools, such as gnip, and while this tool is not bringing a lot of traffic today, it did on its launch date.

You see, for every side project you build, you can launch it in Hackernews (Show HN) and Product Hunt.

My suggestion? Aim to release a new project once a month.

HACK: Lately, some random person posted the Novu organization page on Hackernews. I do not know how it got accepted, but it brought us 400 stars.

Check the post here

Product Hunt with the community

While Hackernews is super friendly with the open-source community (many early adopters), Product Hunt feels a bit more like indie-hacking / non developers’ tools.

The goal is to get as many upvotes as possible and reach the top of the list.

You should usually incorporate Product Hunt with other channels.

If you are starting and want stars - put your GitHub as your “Visit” URL.

If you are a more prominent company and want to get more leads and book meetings, add your website URL.

For the first launch, I would usually aim for the GitHub repository.

Keep had a very small community but still managed to take 1st of the day.

The best founders don’t let luck guide them. Here is what they have done:

They posted on their social media about their launch (like most people)
They try to contact as many people as possible to help them out.
They have built a secret weapon tool! Slackline

Slack-freakin’-line

Tal showed me this tool a while ago, and I have used it ever since.

When you are a small startup, you must do the impossible to get from 0 to 1. Use all the possible options that you have.

Slackline is just for that. You can take any Slack group and DM every possible person in the channel.

There are a few options for it:

You can use it on your Slack channel to drive feedback for your product or get help from community members to do different things.
Ask people for help on Product Hunt.

And while number 2 sounds a bit intrusive, it works - they got 1st of the day and the fame and glory 🎩

Using bounties

You can put bounties on issues with Algora, to tell you that this will grow your community - probably not, but it will give you more credibility.

If you are starting, Algora can help you get more contributors (in the contributors list) and make you look more credible.

However, remember that contributors who come for money usually will not do the same things for free (not always).

Star Keep on GitHub

I invite you to register for my newsletter.

This newsletter is good for you if:

You are considering open-sourcing your product (or building a new one).
You are considering opening a by-product and open-source it (to reflect on your main product).
You are in tech and want growth without the stars / without GitHub trending.

It’s a 100% free newsletter (and always will be). Feel free to register at:

https://gitroom.com