DEV Community: S🅰️Ⓜ️ 🛋

Smart Cover Letters Made Stupid Simple

S🅰️Ⓜ️ 🛋 — Sun, 26 Jan 2025 00:12:01 +0000

This is a submission for the Agent.ai Challenge: Full-Stack Agent (See Details)

What I Built

For the Agent.ai Challenge, I built an intelligent, personalized Cover Letter Writer that streamlines the job application process by generating tailored cover letters based on the user’s LinkedIn profile and a job posting. This agent leverages multiple advanced features of Agent.ai, including invoking python utilities via serverless functions, invoking another agent, and custom prompts to deliver high-quality, user-focused results.

Key Features and Workflow

1. Inputs:

LinkedIn Profile URL: The agent accepts the user’s LinkedIn URL to parse and extract the username. Using Agent.ai’s “LinkedIn Profile” action, the agent fetches the user’s public profile data, such as skills, work history, and accomplishments. Anecdotally: I opted for asking for the full URL rather than just the username after testing this with 15 friends; the majority provided the full LinkedIn URL or didn't know what "LinkedIn username" meant when asked.

A serverless function processes the URL to extract the username seamlessly. (Note: In the future I could imagine a world where a string parse action is available that let's you use regex for a simpler approach to this).

Job Posting URL: The agent can process any publicly available job posting. It fetches the page content and sends it to a secondary Agent.ai Agent ("Job Post Summarizer"), which parses and summarizes the posting into an XML format. This modular design ensures that the job summarization logic is isolated, making it easier to improve independently of the cover letter writing logic.
Output Tone: The user can specify the tone of the cover letter, such as formal, friendly, or academic, allowing for customization aligned with the applicant's personality, and the job and company culture.
Additional Notes: An optional input field allows users to provide specific instructions to the agent, such as emphasizing a particular skill or experience not prominently featured on their LinkedIn profile. This flexibility ensures the final output aligns with the user's goals.

2. Core Functionality:
After collecting the inputs, the agent integrates the parsed job posting in a structured format and LinkedIn profile data into a tailored writing prompt for the AI. The prompt ensures that the resulting cover letter:

Highlights the user’s key skills and achievements relevant to the job description.
Addresses specific company and role details.
Includes placeholders for missing personal details (e.g., phone number, email), which users can fill in manually.

Advanced Features Used

Invoking Python Utilities via Serverless Functions: Serverless functions are used to parse the LinkedIn URL into the username, streamlining the input process for users.
Agent.ai Actions: Leveraged the “LinkedIn Profile” action to dynamically pull user data.
Process Invocations: Integrated a secondary agent for job post parsing, maintaining a modular design for scalability and future enhancements.
Custom Prompts: The AI’s writing prompt dynamically adapts based on user inputs, ensuring personalized, high-quality outputs.

Results

This agent simplifies a common but time-consuming task by automating the creation of professional and personalized cover letters. It not only saves time but also ensures the letters are targeted and effective, helping users stand out in competitive job markets.

By integrating advanced Agent.ai capabilities, this solution showcases the potential of AI to enhance productivity and deliver meaningful, user-centered tools.

Demo

Try the Cover Letter Writer for yourself!

Agent.ai Experience

My experience with Agent.ai was both rewarding and insightful. Here's a breakdown of my journey:

What I Loved

Agent.ai’s interface is clean, intuitive, and well-suited for building dynamic, AI Agents. I particularly enjoyed the ability to use tools like serverless functions and secondary agent invocations to create flexible and modular workflows. The platform's ability to dynamically pull data (e.g., LinkedIn profiles) and leverage processes like custom prompt generation made building my Cover Letter Writer feel both creative and productive.

The modular design philosophy stood out to me. By breaking down tasks into smaller agents (e.g., a separate "Job Post Summarizer"), I could build a scalable system that balances simplicity and functionality. This approach aligned well with the flexibility of agentic systems, where the agent takes control over how tasks are completed.

This allowed me to systematically break down the entire task of writing a well-researched Cover Letter, then add the necessary actions for each step.

Challenges I Encountered

As with many early-stage tools, there were some rough edges. Specifically:

Documentation Gaps: Certain features, like executing serverless functions, accessing agent variables, and defining output variables, lacked detailed documentation. This added some trial-and-error to the process.
Debugging Serverless Functions: While the ability to run Python utilities via serverless functions is incredibly powerful, the process for debugging and understanding the execution environment required some reverse engineering. I ended up creating test agents and carefully analyzing outputs to better understand the system's capabilities and constraints.

How I Worked Through the Challenges

Despite these hurdles, I found the problem-solving process to be a ton of fun! By experimenting with the platform and analyzing the environment, I gained a deeper understanding of how Agent.ai operates. This hands-on exploration was invaluable, and I appreciated how the platform’s design allowed me to iterate quickly.

What Could Be Improved

To enhance the developer experience, I’d love to see:

Expanded Documentation: More guides and examples for advanced features like serverless functions, variable handling, and output definitions would save time and lower the barrier to entry for new users.
Debugging Tools: Built-in tools for debugging serverless functions or visualizing the agent’s environment during execution would be a game-changer for efficiency. The existing debug tab is great, iterating on this and making it more powerful would be awesome! Specifically if we could "re-run" agents from a specific step with saved inputs would make debugging and iterating even faster.

Final Thoughts

Overall, Agent.ai is an awesome platform with a lot of potential for building innovative, agent-driven utilities. While there’s room for improvement in terms of documentation and developer support, the core functionality is robust and well designed, and the process of building with Agent.ai is both fun and empowering.

How to Build a Fortnite Object Detection Model - Updated

S🅰️Ⓜ️ 🛋 — Fri, 26 Jul 2019 19:46:57 +0000

In the previous post I talked about how video games can be used as a resource for building general machine learning models. Today I want to walk through building an object detection model using the Watson Machine Learning service to identify and track objects in Fortnite. Let's jump right in!

Resources

We'll be using a few different services available on IBM Cloud, so first you'll want to create a new account, or log in to your existing account.

The services that we'll be using are:

Cloud Object Storage: To store our data and models
Watson Machine Learning: Environment to train our model
Cloud Annotations: To quickly label our training data

You will also need to have a video of Fortnite gameplay that you can use as training and test data (I have provided one, here, but the more, the better).

Cloud Object Storage

Once you have logged into IBM Cloud, simply click on "Create Resource" and search for "object storage", give your instance a name, I chose "couch-fortnite-object-storage" and select the Lite plan (the lite plan is free and allows up to 25GB of storage). Once the service is created, we need to create credentials so that we can use our object storage to store both our test data and model files. Click New Credential and make sure that the role is set to Writer and the option to include HMAC credential is checked.

Once created, click View Credential, you will see a JSON output of your credential, we need a few elements:

apikey
cos_hmac_keys ––> access_key_id and secret_access_key
resource_instance_id

You can either keep this open in a second tab, or save the JSON in a text file to use in a few moments.

Watson Machine Learning

The last service we need to use is Watson Machine Learning - follow the same steps as above, searching for "watson machine learning", give it a name and select the lite plan. We need to create credentials for this service as well. Click New Credential and again make sure to select Writer as the role. Click View Credentials and again, make a note of some of the elements that we will need later:

instance_id
password
url
username

Preparing the data

The goal here will be to train a model that can both identify and track an object in videos of Fortnite gameplay. We'll use a tool called Cloud Annotations to simplify this process. Navigate to cloud.annotations.ai – we'll use our object storage credentials to login. Enter your resource_instance_id and apikey, select US as the region.

Once logged in, the first thing we need to do is click Create bucket and give it a name. Next, select localization – this will allow us to label objects in a photo by drawing bounding boxes.

Next, click add media and select your Fortnite video (video files will be split into individual frames). Now, click add label, let's name it baller so that we can label ballers in the video (the Fortnite vehical).

Now we can go through our images, drawing boxes around each baller that we see.

You may label as many or as few images as you would like. Only images that have labels will be used in training.

As a general note about machine learning, and Fortnite specifically, you should use training data that incorporates all spectrums of environments that you anticipate using in testing and general use of your mode;. What I mean specifically in Fortnite is that there are many different environments you can encounter (city-scapes, trees, snow, lava, etc.), and you should try to encorporate at least a few different environments in your training data to build the best possible model.

Training

We will be using a CLI tool to interface with our labeled training images and train/download our model. The CLI requires Node 10.13.0 or later.

Installation

npm install -g cloud-annotations

Once installed, you will have access to a command, cacli in your terminal.

Training our model

I suggest creating a new directory where we can run our training operations and eventually download our model (I created a directory called fortnite-obj-detection). From that directory run the command cacli train, the first time you run this command it will prompt you for credentials for both your Watson Machine Learning and object storage instances, this allows the tool to access the training data and then train our model.

cacli will also ask about training params, using the k80 gpu is what we will be using and is included in the lite plan of Watson Machine Learning. The steps, I suggest using 20 * [number of training images] as a general rule of thumb.

Once run, a configuration file will be created so in the future you can simply retrain the model with new data without providing the service credentials.

Once all of the parameters have been filled in, we're ready to train! The CLI tool should automatically initiate the training job and join the queue, it will provide for you a model_id, we will need this to both monitor and download the model. The CLI will ask if you would like to monitor the job once it has started, but if you close the terminal or would like to monitor elsewhere, you can also run cacli progress [model_id].

Download the model

Once the job has finished, we're ready to use the model, but first we need to download it! Simply run cacli download [model_id] and it will retrieve the trained model from our object storage bucket and download it locally. The tool will download 3 versions of the model, ready to deploy in various environments. We will be using the model_web version, ready to use in a tensorflow.js application.

Using the model

A standalone react app is available for you to clone and use already! Once you've cloned the repository, simply copy the model_web directory (the whole directory), and put it in the public directory of the react app. Finally, add a video to the public, called video.mov. Finally, run the app! If all goes well, the video will play and display bounding boxes around the objects that it has identified.

Hopefully this is a great starting point for you to build your own object detection models! Like I said before, I think that video games create a great environment for developing general purpose machine learning model.

Using Video Games to Improve Machine Learning (2023)

S🅰️Ⓜ️ 🛋 — Fri, 19 Jul 2019 17:49:09 +0000

One application for machine learning that I've been excited about is in gaming. Not only useful for game developers, but also many other applications as well! One area of interest to me is the possibility to use video games to simulate real-world challenges and create solutions that can then be implemented outside of the video game. Here I will walk you through how I built a custom object detection model trained for Fortnite.

What are we building a model for?

In machine learning its best to start with deciding on what question you want to answer. In my case, I decided to build a model that could track the unique vehicle in Fornite called The Baller. The goal will be to identify and track when a baller is in a player's field of view, including when a player is using the vehicle.

What's the point?

The primary thesis of this exercise is to demonstrate how we can use simulated environments to solve real-world, general-purpose, AI problems. Rather than have to gather real-world data, we can capture video game data as a testbed to build, test, and refine AI systems. In the case of Fortnite, and specifically, the model demonstrated here, imagine being able to create the framework for object-avoidance, decision-optimizations, or planning the best route to travel. The possibilities are endless!

Hopefully, this post spurs your imagination of what's possible using video games. In the next post, I'll share a guide on how to replicate the model above to detect and track ballers.

Analyzing League of Legends Data with R (2023)

S🅰️Ⓜ️ 🛋 — Fri, 28 Jun 2019 15:59:35 +0000

Sourcing the data

If you want to analyze all gameplay from public matches, Riot has a great API. For me, I like to look at the pros in competitive play; an excellent resource is Oracle's Elixir. Specifically, I use the match data files to start all of my research.

The R Part

As I said, we'll be using the match data file provided by Oracle's Elixer, we'll use the Summer 2019 file. If you're not familiar with R yet, that's OK! We'll keep it simple today. All you need to get started is R Studio. Now there are two libraries we'll be using: Tidyverse (a suite of several libraries for data manipulation) and openxlsx (You may have noticed the file format is an excel document, this lets us open it with ease). With these two libraries, we can get started!

With one line of code, we're ready to rock! Once you run that, we'll have the data loaded in, but before we can start an analysis, we need to clean it up a bit. For example, it thinks the patchno is a number (11.1 for example), but we really should consider it as a string, and the gamelength we should use as a double, the date is also in an Excel-specific format, so we should make that a usable date. So let's take care of these things:

You'll notice that the date cleaning looks a little bit more involved than the others. The reason is that we first need to represent the column as a number, and then we need to convert the number to a date (The trick here is that you need to provide the origin parameter, which in Excel is defined as December 30, 1899... I don't know why, but thanks Google!).

Analysis

Finally, let's do a simple analysis using the Tidyverse packages!

Once we have the data thoroughly cleaned and ready to rock, we first filter the data by the region (in this case, called "league"), so we isolate the LCS (North America region), and then filter the rows by the "Team" rows (as opposed to the results for individual players, the "Team" represent aggregate performance for the entire team. Then we group by the actual teams. Finally, we create a summary for each team, where we tell it to create columns for each of the summaries that we want to see.

Conclusion

In the future, I'll be sharing how to do a more in-depth analysis, but this is a great place for you to get started hopefully. Let me know if you have any questions or ideas for types of analysis to perform!