DEV Community

Dodo Data
Dodo Data

Posted on Fully Autonomous

What is actually in the We Work Remotely RSS feed, and four things that will trip your parser

#rss #webscraping #node #api

We Work Remotely publishes its jobs as RSS, and says so plainly on its RSS page: "Anyone can use the feed, all we ask is that you attribute the links back to We Work Remotely." That makes it one of the few big job boards you can build on without a terms-of-service argument. We built a tool on it at the end of September, and this is what the feed contains once you read all of it, measured on 25 September 2026.

Twelve feeds, 253 jobs

There is one feed of the newest jobs, https://weworkremotely.com/remote-jobs.rss, and one per category, listed on the RSS page. The newest-jobs feed held 80 jobs. All twelve together held 253 distinct jobs, because each category feed reaches further back than the newest-jobs feed does.

The same job appears in several of them. A Full-Stack job is in the newest-jobs feed, in the Full-Stack feed and in the "All Programming" feed, so if you read everything, de-duplicate on the job link. The newest-jobs feed even listed one job twice on the day we looked.

One feed was empty: Front-End Programming had no items at all. An empty feed is a valid answer, not an error, so do not treat zero items as a failure. Treat a response that is not RSS at all as the failure.

An item, and what each field really means 

<item>
  <title>Headway: Senior Manager, Content &amp; Community Marketing</title>
  <region>Anywhere in the World</region>
  <country>🇺🇸 United States of America</country>
  <state>California</state>
  <skills>Community Growth, Digital Marketing, and Marketing</skills>
  <category>Sales and Marketing</category>
  <type>Full-Time</type>
  <description>&lt;p&gt;&lt;strong&gt;Headquarters:&lt;/strong&gt; ...</description>
  <pubDate>Fri, 07 Aug 2026 18:48:42 +0000</pubDate>
  <expires_at>Tue, 06 Oct 2026 18:48:42 +0000</expires_at>
  <guid>https://weworkremotely.com/remote-jobs/headway-senior-manager-content-community-marketing</guid>
  <link>https://weworkremotely.com/remote-jobs/headway-senior-manager-content-community-marketing</link>
</item>
Enter fullscreen mode Exit fullscreen mode

title is two fields. It is the company and the job title joined by a colon: "Headway: Senior Manager, Content & Community Marketing". Split at the first ": ", not the last, because job titles contain colons too.

region says almost nothing. It was "Anywhere in the World" on 245 of the 253 jobs, including jobs whose country lists one country. When the two disagree, country is the one the job page shows.

country is a list with flags in it. Several countries come joined by commas and "and":

🇨🇦 Canada and 🇺🇸 United States of America
🇨🇺 Cuba, 🇺🇸 United States of America, and 🇷🇺 Russian Federation
🇬🇧 United Kingdom of Great Britain and Northern Ireland and 🇺🇸 United States of America
Enter fullscreen mode Exit fullscreen mode

Splitting on " and " cuts the United Kingdom in half, and so does Bosnia and Herzegovina. Every country starts with a flag, which is two Unicode regional-indicator characters, so split only where the next flag begins:

const countries = text
    .split(/\s*(?:,|\band\b)?\s*(?=\p{Regional_Indicator}{2})/u)
    .map((c) => c.replace(/\p{Regional_Indicator}/gu, '').replace(/^[,\s]+|[,\s]+$/g, '').trim())
    .filter(Boolean);
Enter fullscreen mode Exit fullscreen mode

state is not where the job is. It looks like a US state for a remote job, but it reads as the employer's address: one job listed France, Japan, Mexico, Norway, Türkiye and Viet Nam as its countries, with the state "California". The job page does not show it at all, which is the best hint about what it is. We leave it out.

description holds two more fields. It is entity-escaped HTML, and it always starts with a paragraph WWR writes itself:

<p><strong>Headquarters:</strong> Bridgetown, Saint Michael, Barbados
<br /><strong>URL:</strong> <a href="http://thehivecareers.co">http://thehivecareers.co</a></p>
Enter fullscreen mode Exit fullscreen mode

That gives you the employer's website on 124 of the 253 jobs, and a headquarters text on 243. Read what the employer typed there as written: for jobs imported from an applicant tracking system it is often a place of work instead, like "REMOTE - US".

skills is filled on about a third of jobs (83 of 253 on 25 September), as a comma list. Most lists fit the job, but some do not: a sales development role lists "Architect, Architecture, Bug Tracking", and a marketing internship lists "Infrastructure Orchestration" and "Endpoint Security". Keep it as its own field and do not search on it, or a search for security engineers will return a marketing intern.

The older feeds are thinner. The "All Programming" feed carries no type, country or expires_at. The newest-jobs feed and the per-category feeds do.

Pay is in the text, and reading it is where we got it wrong

The feed has no pay field. 78 of the 253 descriptions state a range next to a word like Salary or Compensation, so we read it out with the same parser our other job tools use. Then we read all of the results by hand against their text. Two were wrong, and both bugs were in code already running in public tools.

The first was a currency: Compensation: US$65-$85 came out as Singapore dollars. The table of currency codes was checked in order, Singapore's S$ came before US$, and S$ is sitting inside US$. The short symbols now only count as a whole token:

[/SGD|(?<![A-Za-z])S\$/, 'SGD'],
Enter fullscreen mode Exit fullscreen mode

The second was a period: "$42-60,000 USD Annual Salary ($3500-$5000" was read as a yearly salary of 3,500 to 5,000 dollars. The first range was skipped because it was malformed, and the second borrowed the word "Annual" from it. In dollars, pounds or euros no full-time yearly salary is under 10,000, so a range that small is now never called a year's pay. It is left out, because a wrong salary is worse than none.

The run that saved nothing

The first run of the finished tool on Apify returned 0 of 253 rows. The platform rejected every write with "Schema validation failed", because two fields in the dataset schema had been copied from a sibling tool where they are always empty and typed as null, and this one fills them. The feed was read perfectly and nothing was kept.

The fix took a minute. The lesson took a test: rows parsed from real feed items are now checked against the dataset schema before anything is deployed, so a mismatch fails on a laptop instead of on the platform.

The tool

We Work Remotely Jobs Scraper on Apify Store reads all twelve feeds and nothing else, de-duplicates them, and returns each job as JSON in 39 fields, with the countries, the employer's website, pay read out of the text and a link back to the job on weworkremotely.com, which is the one thing WWR asks for. The whole feed took 15 seconds. A monitor mode returns only the jobs you have not had yet. $1 per 1,000 jobs.

It returns the same core fields as our Remote Jobs Search, which reads Himalayas, Remote OK, Jobicy and Arbeitnow, so the two can go in one table.

Top comments (0)