DEV Community: Anur

Steam Backlog Helper: Tests and Minor improvements

Anur — Tue, 19 Aug 2025 10:02:52 +0000

I haven't been extremely consistent with this last blog post - I apologize, work has been intense! In this post I will be going through a couple Unit Tests (SOME because there is a LOT of them, far too many to go through them all), and discussing some minor features I've implemented since the last time.

Unit Tests

Games Service

The Games Service is responsible for a lot of the core features of Steam Backlog Helper:

Store user's games
Filter and accordingly assign games to the metadata worker
Load games from the database (no metadata fetching)
Get user's games with metadata
Get user's games' genres
Get recommended games (could and should probably move this to another module)
Search games

First and foremost, I hardcoded some mock values that I will be using to test the service

const testSteamId = '123456789';
const mockGames = [
        {
          appid: 1,
          name: 'Game A',
          genres: ['Action', 'RPG'],
          iscompleted: false,
          expanded: false,
          rating: 4.5
        },
        {
          appid: 2,
          name: 'Game B',
          genres: ['Action'],
          iscompleted: false,
          expanded: false,
          rating: 3
        },
        {
          appid: 3,
          name: 'Game C',
          genres: ['Puzzle'],
          iscompleted: true,
          expanded: false,
          rating: 5
        }
];

Then, I mocked and imported all the necessary providers:

describe('GamesServiceService', () => {
  let service: GamesServiceService;
  let ownedRepo: jest.Mocked<Repository<OwnedGame>>;
  let usersRepo: jest.Mocked<Repository<User>>;
  let gameMetadata: jest.Mocked<Repository<GameMetadata>>;
  let metadataQueue: MetadataQueue;

  beforeEach(async () => {
    const module: TestingModule = await Test.createTestingModule({
      providers: [
        GamesServiceService,
        {
          provide: getRepositoryToken(OwnedGame),
          useValue: {
            save: jest.fn(),
            create: jest.fn(),
            insert: jest.fn(),
            find: jest.fn(),
            findOne: jest.fn(),
            update: jest.fn(),
            upsert: jest.fn(),
            createQueryBuilder: jest.fn()
          }
        },
        {
          provide: getRepositoryToken(User),
          useValue: {
            save: jest.fn(),
            create: jest.fn(),
            insert: jest.fn(),
            find: jest.fn(),
            findOne: jest.fn(),
            update: jest.fn()
          }
        },
        {
          provide: getRepositoryToken(GameMetadata),
          useValue: {
            save: jest.fn(),
            create: jest.fn(),
            insert: jest.fn(),
            find: jest.fn(),
            findOne: jest.fn(),
            update: jest.fn(),
            findBy: jest.fn()
          }
        },
        {
          provide: HttpService,
          useValue: {
            get: jest.fn(),
            post: jest.fn()
          }
        },
        {
          provide: MetadataQueue,
          useValue: {
            addFetchJob: jest.fn()
          }
        },
        {
          provide: ConfigService,
          useValue: {
            get: jest.fn()
          }
        }
      ],
    }).compile();

    service = module.get<GamesServiceService>(GamesServiceService);
    ownedRepo = module.get(getRepositoryToken(OwnedGame));
    gameMetadata = module.get(getRepositoryToken(GameMetadata));
    usersRepo = module.get(getRepositoryToken(User));
    metadataQueue = module.get(MetadataQueue);
  });

After the setup is complete, we can move on to the actual tests.

Should throw error when user has no games

it('should throw error when user has no games', async () => {
      jest.spyOn(usersRepo, 'findOne').mockResolvedValue(null);
      jest.spyOn(usersRepo, 'create').mockReturnValue({ id: 123, steam_id: testSteamId } as User);
      jest.spyOn(usersRepo, 'save').mockResolvedValue({ id: 123, steam_id: testSteamId } as User);

      jest.spyOn(service['http'], 'get').mockReturnValue({
        toPromise: jest.fn(),
      } as any);

      jest.spyOn(require('rxjs'), 'firstValueFrom').mockResolvedValue({
        data: {
          response: {
            games: []
          }
        }
      });

      await expect(service.fetchAndStoreUserGames(testSteamId)).rejects.toThrow('User has no existing metadata');
  });

This one is pretty simple. First we mockResolvedValue for the methods the service is going to call. Simply put, we are telling the repo and service what they are going to return once they've been called. We return null when the service searches for a user to check if it actually enters this condition and then force it to create a new one - one that we mocked earlier.

Should create user if missing on fetch & store user games

This one's a bit larger, but not too complicated either. The beginning is similar to the previous test, we check if it creates a user if one does not already exist.

jest.spyOn(usersRepo, 'findOne').mockResolvedValue(null);
    jest.spyOn(usersRepo, 'create').mockReturnValue({ id: 123, steam_id: testSteamId } as User);
    jest.spyOn(usersRepo, 'save').mockResolvedValue({ id: 123, steam_id: testSteamId } as User);
    jest.spyOn(service['http'], 'get').mockReturnValue({
      toPromise: jest.fn(),
    } as any);

Next step: mock the Steam API response:

const fakeGame = {
      appid: 1,
      playtime_forever: 100,
      rtime_last_played: 1234567890,
    };

    jest.spyOn(require('rxjs'), 'firstValueFrom').mockResolvedValue({
      data: {
        response: {
          games: [fakeGame]
        }
      }
    });

We can do this because we are spying on the http.get() request, which is present in the service, as seen below:

const response = await firstValueFrom(
      this.http.get<GameResponse>(
        'https://api.steampowered.com/IPlayerService/GetOwnedGames/v0001/',
        {
          params: {
            key: this.config.get('STEAM_KEY'),
            steamid: steamId,
            include_appinfo: 1,
            format: 'json',
          },
        },
      ),
    );

After mocking a new user, and mocking a response from Steam API, we have to check the next step: adding a game to user's owned games as well as adding a job to the worker. Couple of lines of code do this for us:

jest.spyOn(ownedRepo, 'upsert').mockResolvedValue({} as any);
    jest.spyOn(gameMetadata, 'findBy').mockResolvedValue([]);
    jest.spyOn(metadataQueue, 'addFetchJob').mockResolvedValue(undefined);

Last but not least, let's call the function and check if everything lines up:

const result = await service.fetchAndStoreUserGames(testSteamId);
    expect(usersRepo.create).toHaveBeenCalledWith({ steam_id: testSteamId});
    expect(usersRepo.save).toHaveBeenCalledTimes(1);
    expect(ownedRepo.upsert).toHaveBeenCalled();
    expect(metadataQueue.addFetchJob).toHaveBeenCalledTimes(1);
    expect(result).toHaveLength(1);
    expect(result[0]).toMatchObject({
      appid: fakeGame.appid,
      playtime_minutes: fakeGame.playtime_forever,
      loadingMetadata: true,
      expanded: false,
    });

As you can see, we expect that a new user should be created with our fake steam id, saved only once | that one job should be added to the metadata queue, and result (user's games with metadata) to hold an object which matches our mocked game (+ some extra fields that it gets assigned in the service).

There are 4 more tests in this suite, and a total of 16(!) suites in total, so writing about all of them here would take ages. However, I suppose you get the gist from these few.

All controllers and services are tested in SBH.

Minor changes & new features

Editing profile
Currently, we only support editing the description, however in the near future, we will support changing the gif next to user information.

Game Status Overriding
Implement functionality to override the automatically determined completion status, in order to improve recommendation results and reduce clutter

Game Details Page
Finished the Game Details Page, where users can see pictures and videos from the selected games and some additional data (such as their progress)!

I think this nicely sums up the latest changes. I'll write soon, when I have even more features and hopefully soon once SBH is live!

Steam Backlog Helper: Frontend | Part 1

Anur — Thu, 07 Aug 2025 10:32:42 +0000

Welcome to the second post about Steam Backlog Helper! As promised previously, in this blog I will discuss the current state of the app's FE, and hopefully from this point on I will just be writing regular updates, briefly detailing the latest changes and progress.

Home

I wanted the Home Page to be inviting, motivating (to play games) but also not too much. The main job for this page is to invite users to engage with the website (through Steam log in).

Went for the pixel font for the Home page as it seems aligned with the nature of this service. The background is also a video, that loops and is muted. You will notice the sprite (hooded character) standing on the letter 'R'. Upon clicking it, it will jump! This was simple, just replace the idle animation with the jumping animation (animations' duration considered).
Sidenote: The logo in the nav is still a work in progress so please don't judge me 😅.

Moving on, after the user goes through standard steam sign up we are greeted by the callback page, which also fetches the user's games and their respective data (in the background).

Dashboard

Here is the currently main page of the service: a place where the users can see all of their games, filter or sort them and search for titles too.

Starting from top to bottom, we see the small user overview. Currently, it just houses the user's steam username, steam_id (clickable link pointing to their steam profile), small description and (currently non-functional) edit and share buttons. Next to this is a small gif, currently hardcoded to a single file. In the future, I would like to gamify this platform, thus allowing the users to unlock different gifs/animations that they could use instead of this one.

Below this is the search/filtering section.

Simple enough, there's a search bar, and genres (extracted from the users' games), which , upon clicking, sort the cards correspondingly.

Game cards

These cards hold information about specific owned games:

Genres
Categories
Playtime
Completion status
Estimated time to complete

It is important not to forget the check mark and cancel buttons on the side of the cards - they are used for manual completion status corrections.

Top Picks For You

This page presents you with top picks based on a combination of your most-played genres and user reception / ratings. There will still be more polishing of the logic behind this recommended queue in the future, for example not recommending (or timing) massively multiplayer games because technically can't 'beat' them (I have CS2 in this screenshot with 600+ hours). If you want to read more about the actual logic of this, refer to my previous post: Project Progress: Steam Backlog Helper

As you can see, none of these games are completed and they are accurate. I was actually ecstatic to see Ark: Survival Evolved as my No. 1 since it completely aligns with what I like in games: dinosaurs/diverse creatures and crafting! As a matter of fact, after writing this blog I'll be hopping right in!

This concludes the current progress for SBH, I will keep writing about my future updates and features. As always, feedback is appreciated!

Until next time :)

Project progress: Steam Backlog Helper

Anur — Wed, 06 Aug 2025 08:26:22 +0000

If you've ever had a huge backlog of potentially amazing games on Steam, look no further because I (might) have a solution for you :)

Introduction

Steam Backlog Helper is a website I made to solve an issue many people experience, myself included - the endless backlog of games on Steam you can never quite get around to finishing. Scrolling through my library, there are many games I haven't even touched, and every single one of those could be a memorable experience.

This is why I decided to start my own project to solve this issue, which is where Steam Backlog Helper comes in. It is being built completely in Angular 18, with a NestJs (TypeScript) backend.

How does it work?

Steam Backlog Helper (SBH for short) contains all of your steam library, along with game metadata and your personal stats regarding those games. This includes your favorite genres of games, your playtime for each game, time required to complete games, tags & categories of games and a recommendation page.

You initally log in with Steam from our Home Page, after which you are redirected to the dashboard/games page. Once you log in with Steam, the backend does several things:

Fetch user library data from Steam:

const response = await firstValueFrom(
      this.http.get<GameResponse>(
        'https://api.steampowered.com/IPlayerService/GetOwnedGames/v0001/',
        {
          params: {
            steamid: steamId,
            include_appinfo: 1,
            format: 'json',
          },
        },
      ),

Omitted the key (even though it's grabbed by the Config service) for security purposes. Unfortunately for me, Steam API returns a very surface-level response regarding the users' games - which makes sense, given that there is a LOT of data to process. We receive data in this format:

"game_count": 193,
        "games": [
            {
                "appid": 4000,
                "playtime_forever": 5794,
                "playtime_windows_forever": 0,
                "playtime_mac_forever": 0,
                "playtime_linux_forever": 0,
                "playtime_deck_forever": 0,
                "rtime_last_played": 1542490381,
                "playtime_disconnected": 0
            },...

As you can see, we only have playtime & appid, but for SBH we need more data: Title, Description, Genres, Categories, Completion status etc.

At this point, we move on to step 2:

Associate games with user in owned_games DB table:

for (const g of games) {

      await this.ownedRepo.upsert(
        {
          user: { id: user.id },
          appid: g.appid,
          playtime_minutes: g.playtime_forever,
          last_played: g.rtime_last_played
            ? new Date(g.rtime_last_played * 1000)
            : null,
        },
        ['user', 'appid'],
      );
    }

This is pretty self-explanatory, we just write the user's games to the owned_games database. With this data in place, we can move on to the next step.

Check and insert MetaData if required In this step, I go through all of the games and check them across another table: game_metadata. This table contains associations with games (appid) and their respective metadata which I collect.

In order to determine which games are missing metadata, I use a Set. I collect all the appids which are in game_metadata, and isolate them in this set, so that I could later compare them to all of the appids, thus effectively extracting missing metadata.

const appIds = games.map((g: Game) => g.appid);
    const existingMetadata = await this.metadataRepo.findBy({
      appid: In<Number>(appIds),
    });

    const existingAppIds = new Set(existingMetadata.map((m: GameMetadata) => m.appid));
    const missingAppIds = appIds.filter((appid: number) => !existingAppIds.has(appid));

    for (const appid of missingAppIds) {
      await this.metadataQueue.addFetchJob(appid);
    }

The last part is interesting -> await this.metadataQueue.addFetchJob(appid);. In order to keep the website stable, and also allow the user to use the website while loading, I delegate the job of scraping and finding metadata to a BullMQ worker!
This worker, once assigned a job, goes on to find and insert metadata into the correct table, in the background.

In my worker service, I have defined the following function:

async getGameDetails(appid: number){
        const obs = this.http.get<GameDetailsResponse>(`https://store.steampowered.com/api/appdetails`, {
            params: { appids: appid, l: 'english'}, headers: {
                'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 ' +
                '(KHTML, like Gecko) Chrome/115.0.0.0 Safari/537.36',
                'Accept': 'application/json'
            }
        }).pipe(
            map(response => response.data)
        );
        return firstValueFrom(obs);
    }

All it does is it checks the game against the Steam Store API (api.steampowered != store.steampowered.com/api IMPORTANT), which does have a lot of the metadata I need. The extra headers are to avoid timeouts and blocks from the Store API (which has happened several times before).

Now that we understand where and how the worker gets the metadata, let's move on to the job itself.

async job => {
                const {appid} = job.data;
                await this.sleep(2000);
                let data: GameDetailsResponse;
                try {
                    data = await this.workerService.getGameDetails(appid);
                } catch (err) {
                    console.error(`Failed to fetch Steam API for appid ${appid}`, err.message);
                    return;
                }...

You might notice the 2 second sleep/delay, this is due to the API restrictions on Steam. After acquiring metadata from Steam Store, I then find the community reported time-to-beat. Originally, I've used IGDB and their API for this, however, it had plenty of gaps and was messy to use for me. Instead, I opted for RAWG. RAWG DB has a great amount of games and the data is excellent.

The worker fetches the time-to-beat data as following:

     const hltb = await this.hltbService.getGameTime(appData.name);

It's called

hltb

as an association to 'HowLongToBeat'. Next, it saves the metadata and calculates if the user completed the game (which it saves to the owned_games repo).

try {
                    await this.metadataRepo.save({
                        appid,
                        name: appData.name,
                        genres: appData.genres?.map((g: Genre) => g.description),
                        categories: appData.categories?.map((c: Genre) => c.description),
                        tags: [],
                        last_fetched: new Date(),
                        header_image: appData.header_image,
                        description: appData.short_description,
                        hltb_main_story: hltb
                    });

                    try {
                        const game = await this.ownedGameRepo.findOne({
                            where: {
                                appid: appid
                            }
                        });

                        if(!game){
                            console.log('Could not get game from ownedgames table');
                            return;
                        }

                        game.isCompleted = this.getIsCompleted(game.playtime_minutes, hltb);
                        await this.ownedGameRepo.save(game);
                    } catch (error){
                        console.log('Error saving isCompleted: ', error);
                    }
                    console.log('Saved metadata for: ', appData.name);
                    } catch (err) {
                      console.log(`Failed to save metadata for appid ${appid}`, err);
                    }

This pretty much concludes the core of the worker. There are also fail-safes and conditions for re-fetching in the service (once a week and if certain fields are missing).

The cool thing about this setup is that once a game's metadata is loaded, it's universal, meaning new users have to wait less time after signing up if their games' metadata is already collected.

After the initial log in / sign up, the user does not have to go through steam sign up until their token (given by SBH) expires. During this period, their games do not need to be assigned to the worker or fetched etc, instead, the data is pulled from the database, resulting in very fast response times.

Once in the dashboard, the user will see all of their games, paginated and sorted with appropriate tags: Genres, Playtime status (Never played, Less than 3hr played...) etc. Given the nature of Steam, online & offline play there is room for error when assessing whether or not a user has completed a game. To combat this, within the game card, the user is given an option to correct the status, marking a game either completed or not completed (in addition to the automatically calculated expectation).

Top Picks For You / Recommendations

This functionality extracts and ranks game choices for the user. Currently, it uses only vectors and weights to make this assumption, however, in the future it is very likely going to use some form of ML.

The logic for this is actually not complex and I will go through it quickly:

Get all user's games and extract genre counts

Simple, get all games and order genres in an Object which contains the number of instances a genre has appeared.

const genreCount: Record<string, number> = {};
    games.forEach(g => {
      if (Array.isArray(g.genres)){
        g.genres.forEach((genre: string) => {
        genreCount[genre] = (genreCount[genre] || 0) + 1;
      });
    }
    });

Now we calculate the total for each genre, and create a vector - this vector normalizes genre counts by dividing with total count. This gives us 'user preference weights', and looks a bit like {RPG: 0.5, MMO: 0.2...}

const total = Object.values(genreCount).reduce((a,b) => a+b, 0);
    const userVector = Object.fromEntries(Object.entries(genreCount).map(([k,v]) => [k, v/total]));

With this data in place, we then create a gVector which assigns the value 1 to each genre present in the game. It then calculates the users' preference and assigns score:

const score = Object.keys(userVector).reduce(
        (sum, genre) => sum + (userVector[genre] || 0) * (gVector[genre] || 0),
        0
      );

Effectively, the score is the sum of the user's preference weights. This score could potentially be RPG:0.5 + Action: 0.2 = 0.7 for example.

This worked fine, and the scores worked as expected. However, there was a slight issue: Many of the games recommended, despite having the "correct" genres, were also very, very small indie games or even abandoned projects. From personal experience, I found that in order to engage a user in a game, a big community and a good reception are paramount.

So, I had to introduce a new parameter: Rating. Thankfully, steam store does provide us with ratings, so after a slight modification to my table, I was able to provide ratings to game metadata. Now my goal was the following: after assessing the genre score, add the rating score to the mix as well. However, there's a catch here too: some games have an IMMENSE amount of reviews, which completely skewed the weights. I balanced it out by using logarithms on the ratings themselves, and even new weights on the two parameters: genre rating and review rating:

return scored
    .filter(g => !g.isCompleted)
    .map(g => {
      const ratingVal = g.rating && g.rating > 0 ? Math.log(g.rating) / 5 : 0;
      return {
        ...g,
        combinedScore: (g.score * 0.6) + (ratingVal*0.4)
      }
    })
    .sort((a,b) => b.combinedScore - a.combinedScore)
    .slice(0, Number(amount) || 10);
  }

This, in turn, has produced great results but there is a lot of room for improvement. There is much more code I haven't mentioned here as I'm trying to keep this as concise as possible. In the future posts, I will dive into explaining the Frontend (what I have so far) but also future features I have planned for SBH.

I also am very welcoming to any sort of advice, feedback or even critics.
Stay tuned, until next time :)