DEV Community: Jia Hua Zou

Telescope 3.0 (soon)!

Jia Hua Zou — Fri, 22 Apr 2022 00:18:08 +0000

The release of 3.0 is tomorrow!! This project has come a long way. There are lots of new technologies implemented in a 14 week period. I had learn a lot and I am very glad to have this unique experience with other students(and contributor).

I haven't been posting due to my other workload from my other courses. Now that it is done I can focus my attention to Telescope. Since there isn't much time before the 3.0, I will be helping out other people's issue by either trying to help them or review on their PRs. Other than that there isn't much to say.

I want to say thank you David for teaching this course (His other courses are 😫 👌 💯 🔥) and the team members that are with me till now. This was a very unique course and I'm glad it was not a regret. I am planning to continue to contribute to Telescope after this semester.

Goals for Telescope

Jia Hua Zou — Thu, 17 Mar 2022 21:14:03 +0000

We only have 2 more releases left till 3.0 and we need to focus on what we need to to be shipped before then.

Which part of Telescope 3.0 do I own?

I've had worked on creating different tests where jest did not touch on.
I've worked on the eslint configuration. There are some work left to do and I want to get it done by this week.
I will also focus on getting the search service.

Who can I depend on if I need help?

I can depend on everyone on the team. It is better than to rely on one or two people.

What are the risks that I see, which could prevent me from getting this shipped?

Other than the time, a possible thing that might be preventing me from shipping could be the e2e test not working properly.

More Tests (e2e)

Jia Hua Zou — Mon, 14 Mar 2022 15:13:38 +0000

This week it was very busy week for me. All those midterm took most of my time. This blog will be short. This time I had to create a E2E(end-2-end) test for the signup function, but I ran into some problems. The test is to test if the function can throw a 409 status code if a user creates an account with an invalid or an already existing feed. Pr to the code It took sometime to understand how the code is ran because what I have should done was to read the PR instead of looking at the codes first.

Problem

First thing I did was to understand the code. I look through the Pr. I follow the instruction for the steps to signing up. It was fine until I created the 2nd account. When I got to the confirm page I was expecting an error 409 in the console, but it somehow manage to create the account with the same feed.

Search test pt.2

Jia Hua Zou — Mon, 21 Feb 2022 22:52:46 +0000

If you haven't read my part 1, Link here. TLDR, it's all about the connection to a mock Elasticsearch and the problems I faced.

Tests

This time it's all about the testing that I have created (Github PR). So creating the test was very easy once I got the mock connection working. My focus for these are makin sure query are sending the proper text and receiving proper data. The first setup I did was to create the describe for the normal search and then I put in all of the individual tests with it function like this:

describe('/query routers', () => {
  it('return error 400 if no params given', async () => {
    const res = await request(app).get('/');
    expect(res.status).toBe(400);
    expect(res.statusCode).toBe(400);
  });
});

This first test was create before(Link to blog) when I setup the initial testing. Before adding different test, I have to make sure the mock has to be clear before each tests. To do that, jest has a function that can easy perform this like so:

afterEach(() => {
    mock.clearAll();
  });

Testing for Statuscode

After that, my first few test will be related to anything with the status code 400 or 503. To cause the status code 400, the query send will have mistakes. So the query filter has to be either post or author anything else will cause a status 400. Also if the text query is empty it will also return the same error. code block.

returns for status code 400
 () => {
        return {};
      }

//proper return
() => {
        return {
          results: 0,
          hits: {
            total: { value: 0 },
            hits: [],
          },
        };
      }

If your wondering why I have it return nothing and not return with a proper format(returning results and hits), it because the query isn't proper sent correctly, the result values will nothing but an error.

If the query was sent correctly, and it returns not a proper format it will cause a error 503 because the search results is expecting a results and hits.Error 503 test. request format

Testing for results

After testing the error 400 and 503 now I can start testing for results. There are only 2 tests, one for post and the other for author. Tests for them. So for these tests, I made sure that I can expect a status code 200 with results and total number of results.
This part was a little tricky because if any of the format is wrong, it will cause a status code 503. The return must have a results with a hits. Within that hits it will display the total result values with a list of url link with the id. This is the format that it needs to return.

() => {
        return {
          results: 2,
          hits: {
            total: { value: 2 },
            hits: [
              {
                _id: '1234',
                url: `${POSTS_URL}/`,
              },
              {
                _id: '5678',
                url: `${POSTS_URL}/`,
              },
            ],
          },
        };

Conclusion

Tackling this issue felt it was hard and easy. Mostly the mock Elasticsearch connection was the most challenging for me. It did felt very rewarding when I was able to merge my work to the repo. After this I felt like I can do more issues related to Elasticsearch.

Search test pt.1

Jia Hua Zou — Mon, 14 Feb 2022 18:56:54 +0000

It took longer than I expect, but I manage to get it to work with the help from Roxanne. There was a lot meetings and keeping each other up to date on our progression. I have learned a lot in the past few weeks.

setup

If you haven't read my last post. It was about how to create the initial setup. Link. TLDR. We needed to create the jest setup and the config file. After that we create the test file. This will contain all of the different tests.

Mock Elastic search connection

I was stuck on this part for the longest time.

Problems

First thing I did was, I read the Elastic search mock [blog}(https://www.elastic.co/blog/smooth-mocking-with-the-elasticsearch-node-js-client). It has a lot of information. First thing I try was I use the client and the mock from the Elasticsearch library. Example:

const { Client } = require('@elastic/elasticsearch') 
const Mock = require('@elastic/elasticsearch-mock') 
const mock = new Mock() 
const client = new Client({ 
  node: 'http://localhost:9200', 
  Connection: mock.getConnection() 
}) 
mock.add({ 
  method: 'GET', 
  path: '/' 
}, () => { 
  return { status: 'ok' } 
})

I have pass the mock.getConnection() to the Connection for client. From this, all of the connection is going through the mock Elasticsearch instead of the real Elasticsearch. Then I add a mock. It requires the method and paths and returns a status. After ward I use the supertest library to request the first mock created.

const res = await request(app).get('/');
expect(res.status).toBe(200);

When I ran this test I was expecting a status 200, but I got was an 400. I try to add paths to the mock to see if I can get results.

mock.add({ 
  method: 'GET', 
  path: '/test' 
}, () => { 
  return { status: 'ok' } 
})

I still got the 400 status.

correct way of connecting

After talking to Roxanne, I found out what I did wrong. Firstly, when I add my mock, the method and path was wrong. My method was missing the POST.

method: ['POST', 'GET'],

Secondly, my path was incorrect. It should not be use as a path parameters, instead it should've been a query parameters.

path: '/posts/post/_search',

The query format is found in the search.js file.

  const {
    body: { hits },
  } = await client.search({
    from: calculateFrom(page, perPage),
    size: perPage,
    _source: ['id'],
    index,
    type,
    body: query,
  });

The reason why it's /posts/post/_search because from the search.js the variable are index = 'posts' and type='post'. If the path was not correctly call exactly, the test will return 400.
The request now need the query function.

const res = await request(app).get('/').query({ text: 'Telescope', filter: 'post' });
expect(res.statusCode).toBe(200);

I test run the test and it finally return status 200.

Conclusion

I learn that the setup for mock Elasticsearch has to be very exact, anything wrong will results in a status 400. It was frustrating trying to understand what I did wrong even though it was a very simple fix.

Prerelease 2.6

Jia Hua Zou — Sun, 06 Feb 2022 22:39:16 +0000

Version 2.6 is coming soon. I have been working on the tests for the search function for Telescope.

At first it sounded like a simple task, but when I start to dig into the problem it was getting a lot more difficult. The simplest thing that I did was to setup the initial setup for the tests. Issue-2757. Even that had some problems. I thought the config setup was correct but later found out that the configuration was not properly setup even tho the test can still work fine. This was because when we ran the jest command to run our test, the command included the setup config. We don't want that because different not all test will need the same setup.

Before:
 "jest": "cross-env NODE_ENV=test LOG_LEVEL=error MOCK_REDIS=1 FEED_URL_INTERVAL_MS=200 jest -c jest.config.js --",
After:
"jest": "cross-env LOG_LEVEL=error jest -c jest.config.js --",

With this change, each test can use their correct config file. After that, in the jest.config.js for the search. I would need to add the proper setting so like so:

const baseConfig = require('../../../jest.config.base');

module.exports = {
  ...baseConfig,
  rootDir: '../../..',
  setupFiles: ['<rootDir>/src/api/search/jest.setup.js'],
  testMatch: ['<rootDir>/src/api/search/test/*.test.js'],
  collectCoverageFrom: ['<rootDir>/src.api/search/src/**/*.js'],
};

The jest.setup.js is in the same directory as the jest.config.js. All it does is to set a mock Elastic to 1(true).

Once that is all setup, I can create my test file. If I do not create a test, Jest will throw an error when every we try to run the test command. So what I add was a simple test to test out if it can return error 400.

const request = require('supertest');
const { app } = require('../src');

describe('/query routers', () => {
  it('return error 400 if no params given', async () => {
    const res = await request(app).get('/');
    expect(res.status).toBe(400);
  });
});```



I know this will always be true because the connection to a mock elastic was not created. My next step would be trying to understand how we can create the connection to a mock elastic and dummy data.

Coming back for more - OSD700

Jia Hua Zou — Sat, 15 Jan 2022 23:17:44 +0000

After finishing my OSD600, I felt like I want to come back and learn more. This time it is more focus on the project telescope. I am glad that my other courses did not overlap the slots for OSD700.

My Goal

This time my goal for this course is to get in more depth on open-source code. I want to understand the back-end of Telescope. As of right now I had to setup on my local computer. It took a while since there are a lot of documents to read. Did it felt like a Easter hunt.

I had my first experience of the Triage meeting with my classmates. We went over PR to see if we can close it and assign one or more of us to that PR if needed. After that we quickly went over the Issues. There was a lot of issues, so are from 3 years ago. Also, I will be a Sheriff with Tim Roberts for next week.

My first issue that I took is to add testing for the search function. Issue-2621 This won't be an easy task to do since there are endless possibility of testing.

Conclusion

This course will get really busy very soon but, I look very forward to contributing to this community.

TESTING

Jia Hua Zou — Sat, 15 Jan 2022 22:41:33 +0000

JFDGKShtsrthhtdbrtgtnyuntbebgmgnmgm

Hello World

Jia Hua Zou — Tue, 07 Sep 2021 19:02:11 +0000

I'm Jia Hua and in the next 4 months my blog will be related to open source programs. I always had an interested in solving problems.
I wanted to take the open source class because back when I first try to contribute to open source program, I was overwhelm with the hundreds of files and lines of codes. I did not know where to start or how to contribute. Hopefully after taking this course, I will be more confidents on contributing to open source projects.