How to Seed Your Rails API with an External API

As part of Phase 4 of Flatiron School's Software Engineering bootcamp, we are tasked with building a full-stack application, including standing up an API with Rails to communicate with a React client application.

Up until this point, most of our test data has been using either Faker to generate seed data, or by creating our seed file from scratch.

But, as you can imagine, both of those strategies have their drawbacks.

With Faker, your quality of test data is limited by the ability of the stock Faker data, or by customizing your own Faker data.

With scratch-baking your seed data, it can get tedious and your tests are further skewed by the fact that you are creating the test data you are testing.

But what if I told you that you could use an External API to seed your seed data?

There are a few other services, but I used RapidAPI and specifically RAWG API which has a ton of APIs to use.

I promise it's not that hard, and it'll be fun!

Assumptions

1) You have a RapidAPI and RAWG API account
2) You are using Ruby on Rails and are using at least version 2.7.4
3) You are familiar with Ruby on Rails, Ruby Gems,and migrating and seeding a Rails API.
4) You have the HTTP/REST concepts.

Additionally, I used the RAWG API, so I will show the steps I used for that API; however, other APIs may have different instructions and steps. I would highly recommend reading the documentation for the specific API you are using as they may recommend a different process.

Step 1: Get a API key and API Endpoint

First, I registered for a free account with RapidAPI and then RAWG. This allowed me to get an API key, which is necessary for accessing their API.

(If you don't know what an API key, I would recommend reading this article for a brief overview)

On RAWG, I was directed to their API homepage, which presented a big fat "GET API KEY" button.

You might need to enter in some basic information about why you are using the API and provide a URL (I provided our GitHub repo).

After you finishing any of these admin tasks, you should be presented with the following page (I blacked out my personal key and info for obvious reasons):

Copy the value in the "API Key" field save that somewhere safe. Do not share this with anyone. This is the key your application will use to request data from your external API.

Step 2: Get your API Endpoint

Next, you'll need to get your API endpoint. This is where your mileage may vary from RAWG, but RAWG gives you their endpoint straight up on the RAWGAPI homepage:

Save that endpoint as you'll be making a GET request to that URL.

Step 3: Install Gems

You'll need to install the following gems:

1) gem rest-client
2) gem dotenv-rails

Step 4: Create ENV and Update .gitignore file

There's some quick steps we need to take in order to let our code tap into the API.

1) Create an ENV file

You'll need to create a .env file and create a variable and assign your API key to this variable:

//.env file
API_KEY = <paste your api key here>

2) Update your .gitignore file

You'll need to add the following code to your .gitignore file. This file tells GitHub which files to "ignore" when tracking and pushing files to a repo. As you can imagine, we don't want to broadcast our API key to the interwebs, so this will give us a little security when we push our code.

//.gitignore file
.env

Once you've done these two steps, we are now ready to update our seeds file!

Step 5: Create your Seed File!

In your seeds file, drop the following code into your file:

1 require 'rest-client'
2
3 puts "Getting Games Data"
4   def api_key
5        ENV["API_KEY"]
6    end
7
8    def games_dataset
9        api_data = { key: api_key }
10        games = RestClient.get("https://api.rawg.io/api/games?key=#{api_data[:key]}")
11        games_array = JSON.parse(games)["results"]
12        games_array.each do |g|
13            Game.create(
14                title: g["name"],
15                genre: g["genres"][0]["name"],
16                platform: g["platforms"][0]["platform"]["name"],
17                image: g["background_image"]
18            )
19        end
20
21
22    end
23 games_dataset() 
24 puts "Seeding Games Data"

Let me break this code down line by line.

- LINE 1: I am requiring the rest-client gem. In a nutshell, this gem gives us access to simple HTTP/REST actions, such as GET, which let's us grab data from the RAWG API.

- LINE 3: Next, I've added a simple puts to the console to tell us what's going on (always a good practice to have!).

- LINE 4-6: Next, I have a method (api_key) that is grabbing my API_KEY variable from my ENV file.

- LINE 8: Then, the big one, my games_dataset method .... Here's what is going on top down:

- LINE 9: I am creating a variable and setting it to an object with key as a key and my api key as it's value.

- LINE 10: I am setting a variable and setting it equal to a method, RestClient.get, which is a method given to me by the rest-client gem that is a get to the API endpoint.

- LINE 11: I am making another variable and using the method JSON.parse(). This Ruby method turns a JSON array into a Ruby array.

- LINE 12 - 19: I am then using a .each method to iterate through the data and using the .create method to seed my data into the appropriate attributes that I need into my Games model.

- LINE 23: I am then calling my method, games_dataset(), so that this method runs when I run rails db:seed, and then also puts-ing to the console so I know when my data is finished seeding.

I would highly recommend throwing in byebugs everywhere during this process, especially to make sure you are pinging the API and getting data successfully in return as well when iterating through the JSON data for seeding your model.

Keep testing by running rails db:seed to ensure you data is seeding your db.

Step 6: Throw Your Hands Up and Yell "I am Invincible!"

...Because you've just seeded a database with another API!

No more Lorem Ipsum's and stock cat pictures from Faker, and no more copy-pasta seeding data.

Bonus Step: When You Deploy To Heroku

When I was deploying to Heroku, I ran into an issue when trying to seed my data in Heroku (heroku run rails db:seed)

I was getting the error RestClient::Unauthorized: 401 Unauthorized.

After some troubleshooting (and help from one of our awesome FI instructors), the issue was that Heroku was trying to seed data from RAWG but wasn't using my API_KEY.

My API_KEY was stored in my .env file which was being ignore in my .gitignore file.

This caused an issue because Heroku takes your latest push to your main branch in GitHub.

Since the .gitignore file is ignored when pushing to GitHub, Heroku was not picking up my API_KEY to use when pinging RAWG.

FIX:

When you first deploy to Heroku, you'll need to configure Heroku to set an environment variable for your API_KEY, similar to what the .env file was doing.

The steps in this wonderful guide on Heroku should help you out!

---

Conclusion

While this is obviously specific to RapidAPI and specifically RAWG API, the overall concept of this should remain the same for any external API you might use.

To recap, we are doing the following:

1. Using an API key and Endpoint to fetch data from an external API
2. Iterating over that data and throwing those values into the attributes of our model.

While the endpoint and key may differ by the different API you might use, this is the basic idea of what we are doing.

There are obviously other ways to do this depending on the languages you are using, but the core concept of "hey, get me data!" remains the same.

I hope this tutorial was helpful, and, from these seeds, may you grow a strong tree database.