elisabethgross for Coderbyte

Posted on Jan 21, 2020

How to build a side project that will impress future employers

#javascript #career #codenewbie

Hey everyone! Welcome to a special edition of Code Review, our weekly series of coding challenges and job related content. In this post, I’ll be talking about one of my favorite side projects, Breadwinnerss, and how you can impress future employers by flexing your coding and problem-solving skills! I’ll go over what makes for a great side project and how to talk about your projects on your resume and in interviews.

Remember, if you like this content and want more of it, subscribe to our newsletter here to receive the latest and greatest we are putting out on Dev.to and on Coderbyte.

Finding awesome project ideas

From a technical perspective, a good project idea is perhaps the least important part of any good side project. But, trust me, a compelling project that solves a real problem is way more impressive to an interviewer than another 'alarm clock app.' So how do you find awesome project ideas?

You might have your own, but since you're a developer, the odds are you know someone is good at identifying cool problems to solve. Personally, I connected with the founder of the first company I worked at Nis Frome. Entrepreneurs have a track record for problem solving and I've found that they usually have a backlog of ideas, big and small. You probably know more than a few who would love to partner.

Nis had a number of projects he wanted to work on, but one in particular required some cutting-edge tech, which made it attractive to me. Today, the project is called Breadwinnerss and it solves a problem that Nis is very passionate about: helping people in his network find jobs at companies in his network.

Every week Nis gets 5-6 requests from job seekers asking for introductions to companies he has contacts at. Before Breadwinnerss, Nis would spend time browsing career pages from companies in his network to make matches and intros. Breadwinnerss was born to help alleviate some of that manual work. We essentially built a massive web scraper that scrapes career pages and aggregates the jobs into one live feed (kind of like an rss feed… get it? Breadwinnerss?). That way, when someone asks Nis for an intro, he can just send them a single link to his Breadwinnerss feed where the user can request intros for whichever roles they're interested in across dozens of companies. Already, Nis and our other users have helped several people get really cool new jobs.

Flexing your dev skills

The main goal for this side project was to learn, so I specifically picked technologies that I hadn’t done much work with before. This is an excellent talking point in an interview. It’s an opportunity to show your interviewer exactly what you can do when faced with new or unfamiliar technologies which is essentially what will happen at almost every new job you ever take.

That being said, you don’t have to reinvent the wheel. For Breadwinnerss, I chose full stack Javascript because that is something I am comfortable with, but I chose almost all new frameworks and tools to go with it. I used Node and scraping libraries called Cheerio and Puppeteer to build the scraper. I deployed that to a Google Cloud function which runs daily. The scraper script itself scrapes each company’s career page with a custom scraper function I built and saves all those results to a file in an AWS S3 bucket. The web application part of Breadwinnerss is also built using Node, with express as the routing framework and Postgres as the database. The front end was built using Vue.js. The web app reads the scraped jobs from s3 and serves them to the front end. It lives on Heroku.

  filterDepts (data, $) {
    // needed to capture the class instance (at this point the 'this' context) which becomes the document in the cheerio callbacks 'this' context
    const that = this
    const filteredDepts = data.filter(function () {
      const dept = $(this).closest('.ptor-jobs-list__department-section').find('h2').text()
      return utils.myFilter(that.targetDepts, that.badDepts, dept)
    })
    const ret = []
    filteredDepts.each(function () {
      const jobTitle = $(this).find('a').text()
      const url = $(this).find('a').attr('href')
      const location = $(this).find('.ptor-jobs-list__item-location').text()
      ret.push({
        jobTitle,
        location,
        url
      })
    })
    return ret
  }
  filterJobs (jobs) {
    return _.filter(jobs, (job) => _.includes(usCities, job.location))
  }

Emphasize technical challenges and solutions

To a large extent, tech interviews are basically one big simulated problem-solving exercise. Can you troubleshoot? Can you learn quickly? Can you jump into a fluid environment and adapt? It's important to emphasize how you solved technical problems while building your project – it'll alleviate a lot of pressure during the interview.

One of the biggest changes we made to the architecture of the app was changing how and when the actual scraping happened. When we started, the initial proof of concept was just for Nis and scraping the 8-10 companies from his network. Naturally, I just scraped every company on page load of Nis’ Breadwinnerss feed. The scraper was coupled with the web application code and would get the latest list of jobs every time someone visited the feed. This actually worked for us for quite a while and doing that as our MVP allowed us to release an early working version. Some people might think that learning 5 things at once and perfecting the tech stack before release will look very impressive, but more often than not, it just prevents you from ever releasing it.

That was fine until we got to scraping about 20 companies. At that point, the request for scraped jobs was taking longer than the maximum Heroku will allow a request to last (about 30 seconds). This was when we decided to cache the results of each scrape in a file in an S3 bucket. We also moved the scraper code to be in a self contained module that we deployed to Google cloud functions to be run on a daily cron. All this allowed for feeds that load quickly and the ~100 companies we now include in our scrape every evening.

async function processCompanies (browser, companies) {

  const processedCompanies = []
  for (const connectorCompany of companies) {
    const { target_jobs, bad_jobs, target_depts, bad_depts, module_name, scrape_url, base_url, companies_name, company_url, type } = connectorCompany
    console.log(`Scraping ${module_name}...`)
    const companyModule = require(`./companies/${module_name}.js`)
    const connectorCompanyModule = new companyModule(target_jobs, bad_jobs, target_depts, bad_depts, scrape_url, base_url, companies_name, company_url, type)
    const result = await scrape(connectorCompanyModule, browser)
    processedCompanies.push(result)
  }
  return processedCompanies
}

Showcase projects on your resume

Most developers already put their GitHub profiles on their resume. If your project is in a public repo, savvy recruiters may check it out. But that's basically burying something that gives you a significant edge.

I'd recommend instead creating a dedicated section to call out your project, any cutting-edge tech used to build it, and any market traction or validation you have such as number of users or even revenue.

Main takeaways

In summary, here are 4 of my main tips when it comes to building a side project:

Find and solve a compelling problem.
Use the project as an opportunity to mess around with new technologies.
Iterate! Build a quick prototype in less than a month and then improve it based on user feedback.
Collaborate with friends who you wouldn't normally get to work with. Side projects don't have to be lonely!

Check out the code on github!

We felt it was important to make the code public so other people can learn from how we built this project. Go check it out on github and let us know what you think.

Top comments (15)

zakwillis • Jan 21 '20

Hi there. Good article.
I would add this though. I would say we shouldn't be doing this just to get employment. Obviously, we all need money.
I think it was "Uncle" Bob who said - developers are the most powerful people going, if they know they are powerful. The reason is, they can team together and build products.
It is important to try and monetise these products developers do as side projects. It is a completely different mindset - one I have only myself got over the last year.
One quick example. You are building a major piece of work, are there sub-components which could by themselves be a product.

elisabethgross • Jan 21 '20

Thanks! I totally agree. My favorite part of being an engineer is the ability I have to create with just a laptop in front of me. When you put a couple of us together, plus some good friends and mediocre ideas, the stars are the limit.

In terms of sub-components that could themselves be a product. Absolutely! Each company's scraper class is a bit of a subcomponent and one of our ideas was to really open source this and let other people add to the scraper library. Anyone interested in getting involved? Let me know!

zakwillis • Jan 21 '20

Hi Elisabeth, I have written my own scraping engine. On top of HTML Agility Pack and ScrapySharp.

inforhino.co.uk/innovation

Am not quite at the stage of selling this through my limited company as it underpins the processing framework for my property platform, but am thinking of launching other deployment/processing applications on this framework very soon.

elisabethgross • Jan 21 '20

Nice!! I'll be sure to check it out

dgigafox • Jan 22 '20 • Edited

Thanks for the great advice! I agree on using unfamiliar tools and frameworks because programming skills do not focus on language syntax alone but how you handle technical challenges. Although I have a problem finishing side projects. I think that is the downside of too many ideas in the backlog and that is why I tend to switch from one to another. Any tips for me? 🙂

Clive Da • Jan 21 '20

er does posting on DEVTO count as a sideproject ? its taking up all my sparetime :)

elisabethgross • Jan 21 '20

RELATABLE COMMENT hahahahha :)

Warner Onstine • Dec 1 '20 • Edited

Some additional tips I would add:

A specific goal in mind of why you want to build this thing
Once you have your goal, then I would time box it to 4 - 6 weeks. What is it that you think you can deliver in that time frame?
Once you've got your end goal and you have your specifics for a deliverable then you need to work on that project every single day. Not just on the weekends, not just when you think you have time, carve out at least 30 minutes a day and work on your project.
Document as you go. What are you learning? What is stumping you? Make sure you keep this up to date so that when you come back to your project the next day it is fresh in your mind and is easy to pick up again.
Run into a problem? Ask for help. Find the right forum or group and ask detailed questions about the issue you're facing. Most communities are very open and willing to help those who are learning.

Some of this I have distilled into a flowchart to help people pick a side project. Other sections are actually part of a course I teach called 30 Minute Project.