How to get the tags of a StackOverflow question via API call from Node

I found myself lately adding more StackOverflow questions to my dev bookmarks collection, so I took the challenge over the weekend to make this experience more pleasant. I am fascinated by the Stackoverflow tags, so I wanted them added automatically when bookmarking a Stackoverflow question. In this blog post I will present you how to do that and how to call the functionality from an Angular front end.

Improve auto completion of stackoverflow bookmarks

The existing scrapper adds the title of stackoverflow questions, but more was certainly possible. Indeed by using the stackexchange api I was able to automatically add extra the tags and the creation date of the question. Unfortunately the correct answer is not made available via the API, or I did not find out how yet. So let's see how that works.

Backend

In backend the changes are minimal. Check if the scrape path contains a stackoverflowQuestionId query param and then we invoke the api with a key registered on stackapps to get the data

Router

/* GET stackoverflow question data */
router.get('/scrape', async function (request, response, next) {
  const stackoverflowQuestionId = request.query.stackoverflowQuestionId;
  if (stackoverflowQuestionId) {
    const webpageData = await PublicBookmarksService.getStackoverflowQuestionData(stackoverflowQuestionId)

    return response.send(webpageData);
  } else {
    next();
  }
});

Service

I use superagent to make the REST api call

let getStackoverflowQuestionData = async (stackoverflowQuestionId) => {
  const response = await request
    .get(`https://api.stackexchange.com/2.2/questions/${stackoverflowQuestionId}`)
    .query({site: 'stackoverflow'})
    .query({key: process.env.STACK_EXCHANGE_API_KEY || "change-me-with-a-valid-stackexchange-key-if-you-need-me"});


  const tags = response.body.items[0].tags;
  const title = response.body.items[0].title;
  const creationDateMillis = response.body.items[0].creation_date * 1000;
  const creationDate = new Date(creationDateMillis).toISOString();
  const publishedOn = creationDate.substring(0, creationDate.indexOf('T'));

  const webpageData = {
    title: title,
    tags: tags,
    publishedOn: publishedOn
  }

  return webpageData;
}

With a key your rate is currently limited to 10000 calls per day, otherwise to just 300

Front-end

In front-end a little bit more work was needed, since some refactoring was also involved.

  private getScrapeData(location) {
    this.personalBookmarkPresent = false;
    const youtubeVideoId = this.getYoutubeVideoId(location);
    if (youtubeVideoId) {
      this.bookmarkForm.get('youtubeVideoId').patchValue(youtubeVideoId, {emitEvent: false});
      this.publicBookmarksService.getYoutubeVideoData(youtubeVideoId).subscribe((webpageData: WebpageData) => {
          this.patchFormAttributesWithScrapedData(webpageData);
        },
        error => {
          console.error(`Problems when scraping data for youtube id ${youtubeVideoId}`, error);
          this.updateFormWithScrapingDataFromLocation(location);
        });
    } else {
      const stackoverflowQuestionId = this.getStackoverflowQuestionId(location);
      if (stackoverflowQuestionId) {
        this.bookmarkForm.get('stackoverflowQuestionId').patchValue(stackoverflowQuestionId, {emitEvent: false});
        this.publicBookmarksService.getStackoverflowQuestionData(stackoverflowQuestionId).subscribe((webpageData: WebpageData) => {
            this.patchFormAttributesWithScrapedData(webpageData);
          },
          error => {
            console.error(`Problems when scraping data for stackoverflow id ${stackoverflowQuestionId}`, error);
            this.updateFormWithScrapingDataFromLocation(location);
          });
      } else {
        this.updateFormWithScrapingDataFromLocation(location);
      }
    }
  }


  private getStackoverflowQuestionId(location: string) {
    let stackoverflowQuestionId = null;
    const regExpMatchArray = location.match(/stackoverflow\.com\/questions\/(\d+)/);
    if (regExpMatchArray) {
      stackoverflowQuestionId = regExpMatchArray[1];
    }

    return stackoverflowQuestionId;
  }

  private patchFormAttributesWithScrapedData(webpageData) {
    if (webpageData.title) {
      this.bookmarkForm.get('name').patchValue(webpageData.title, {emitEvent: false});
    }
    if (webpageData.publishedOn) {
      this.bookmarkForm.get('publishedOn').patchValue(webpageData.publishedOn, {emitEvent: false});
    }
    if (webpageData.metaDescription) {
      this.bookmarkForm.get('description').patchValue(webpageData.metaDescription, {emitEvent: false});
    }
    if (webpageData.tags) {
      for (let i = 0; i < webpageData.tags.length; i++) {
        const formTags = this.bookmarkForm.get('tags') as FormArray;
        formTags.push(this.formBuilder.control(webpageData.tags[i]));
      }

      this.tagsControl.setValue(null);
      this.tags.markAsDirty();
    }
  }  

If it is recognised it involves a stackoverflow question via the getStackoverflowQuestionId(location: string) method.
The backend api is then called to receive the question metadata.

The API invoking part:

  getStackoverflowQuestionData(stackoverflowQuestionId: string) {
    const params = new HttpParams()
      .set('stackoverflowQuestionId', stackoverflowQuestionId)
    return this.httpClient
      .get<WebpageData>(`${this.publicBookmarksApiBaseUrl}/scrape`, {params: params});
  }

With this data the title, tags and creation date autocomplete for you:

Learn about additional improvements like visual hints for StackOverflow questions or Single-Sign-On with StackOverflow account in the original post - Improved bookmarking of StackOverflow questions