DEV Community: Francesco Menghi

Telescope 3.0

Francesco Menghi — Sat, 23 Apr 2022 22:29:47 +0000

Telescope 3.0 is finally out! For the final week of OSD700 I finished two issues that I mentioned in my previous blog post and some other small fixes.

I successfully added the YouTube and Twitch sign-up page and it now looks like this:

This PR was supposed to only add support for Twitch while Thanh was in charge of adding support for YouTube. However, after Dave's feedback, I ended up adding YouTube in the sign-up instructions and finally Thanh reviewed and approved my PR.

Many fixes

When something doesn't work and you try to find a solution, often the fix turns out to be something really simple. This is what happed with our Turborepo CI cache implementation.

The reason why the CI cache didn't work is because we were missing the team flag in the command. I previously assumed it was not required by looking at the Turborepo docs but Felix (the maintainer of the GitHub Action that we're using) came to the rescue and suggest the fix. In the end, this simple PR made everything work!

ESlint change

Our ESLint implementation included TIMING=1 to get a performance overview each time we lint our code. This however felt unnecessary most of the time so I decided to make it a separate command.

Going forward we have two ESLint commands that are run using Turborepo:

pnpm lint: runs ESLint in our repo.
pnpm lint-time: runs ESLint in our repo with the TIMING=1 option.

Docusaurus fix

Another quick fix I made was removing these two options from Docusaurus:

showLastUpdateAuthor: true,
showLastUpdateTime: true,

These two options allow you to use the git history to retrieve information on when a page was last updated and by whom. Unfortunately since our Docusaurus app lives inside the monorepo, .git is only found in the root of the monorepo. So, when building the Docusaurus app in a Docker container, there was no git info being copied over and it resulted in a long list of errors.

The end of OSD700

This was the last week of OSD700 and last week of my studies at Seneca. I cannot believe I already reached the end. It feels like yesterday that I was learning things like navigating around a terminal or learning the basics of html, css and javascript.

Both OSD600 and OSD700 were an incredible and unique experience. Last semester I learned how to use git and GitHub and made open source contributions during Hacktoberfest and beyond. This semester I participated into taking the Telescope project from version 2.0 to version 3.0 with an amazing group of people.

Working in open source feels completely different than getting an assignment done and submitting it before a deadline. When you open a Pull Request, your code is there in the open ready for other people to take apart, find potential problems and eventually merge. In my experience, there is often a peer with better experience with a certain tool or technology that can provide help or guidance when you are stuck.

Looking back at my first post of the semester where I wrote some goals for the class, I feel like I have accomplished what I setup to do. I worked a lot on monorepo tooling, I used Docker, NGinx, React, Docusaurus and more.

I am excited for the future of Telescope and how the next group of students will influence its direction. I plan to stick around and continue to make contributions to the project!

Close to the finish line

Francesco Menghi — Sat, 16 Apr 2022 22:52:23 +0000

We are now less than a week away from the final Telescope release: version 3.0. Here's a quick update of what I've been up to.

Turborepo CI cache attempt

With my latest PR, I added support for Turborepo for our tests. I also tried to add the ability to keep a Turborepo cache as GitHub Artifacts in CI.

I used this GitHub Action by Felix Mosheev which basically creates a local server in CI that can be used to upload/download the cache during a CI run. The Turborepo docs mention that Remote Cache can be self-hosted and used with the command:

turbo run build --api="https://my-server.example.com" --token="xxxxxxxxxxxxxxxxx"

Felix has a fully working example that I used as a guide. To run the Action we need a token that is used to connect to the local server. We are using the unique GitHub sha as the token each time.

Unfortunately even after Dave's follow-up attempts to make this work, the local server starts but no cache is saved. I don't think this is an urgent issue to get done before the 3.0 release so I might end up keeping this one for later.

Add Twitch URL to Sign Up flow

The other issue I started working on is allowing the frontend Sign Up page to take a Twitch URL. Thanh is also doing the same but for Youtube.

During a meeting, Dave suggested to rework the feed-discovery to handle not only blogs, but also Youtube and Twitch channels and he did so in a live coding session with Thanh and I. I reviewed his PR and I think it should be ready merged soon.

Youtube provides RSS feeds for all channels. You just need the Youtube channel id number and then put it at the end of this URL: https://www.youtube.com/feeds/videos.xml?channel_id=

Twitch however does not do the same. This is were RSS-Bridge comes in handy. It creates an RSS feed for websites that don't provide one.

Finishing up the Sign Up flow is important so that future students can get started with Telescope by signing up directly on the website. Other students have already done the heavy lifting in this area but adding Youtube and Twitch support will be the cherry on top. I'll have an update on this in next week's blog post.

Telescope 3.0 alpha

Francesco Menghi — Mon, 11 Apr 2022 02:09:40 +0000

With Telescope 3.0-alpha released this past Friday, we are now one release away from the big 3.0!

With this release the Docusaurus app is now live on production. It is available at https://telescope.cdot.systems/docs.

After last week's work to have a Dockerfile for both the web app and our docs, this week I added the necessary config to Nginx to deploy the Docusaurus app. It was actually easier than I expected, I just had to add the following to config/nginx.conf.template just like I previously did in our dev config:

location /docs {
  root /usr/share/nginx/html;
}

Docusaurus path

I also had to fix the Docusaurus path since after adding it to nginx we ended up with /docs/docs in the path (The first /docs from nginx and the second from Docusarus). I couldn't find a good way to do it and then I remembered that pnpm uses Docusaurus for their docs and they do not have /docs in their path. In fact if you click the "Docs" link from their homepage, you are redirected to their first documentation page which is: https://pnpm.io/motivation.

Their docs are open source so I went digging and found that they have routeBasePath: '/', in their config. This did the trick for our Telescope docs.

Finally to get the app production ready I had to change the Telescope green to our Telescope blue! We still have dinosaurs on the homepage but Alex is working on a new one that should be up by the 3.0 release.

Goodbye Satellite

I moved the Satellite project inside of the Telescope monorepo last month and, this week, the time came to archive the original repo. Before clicking the big red button, I had to do some spring cleaning of the issues and PRs, change some outdated links and also remove the Satellite contribution info from the Dashboard (Something I actually added last semester).

We archived the Satellite repo but the project lives on an is healthier than ever!

Getting Nginx to work

Francesco Menghi — Mon, 04 Apr 2022 02:29:36 +0000

This past week I finally finished the work necessary for the issue about sharing a single nginx container for all static sites.

I moved docusaurus and test-web-content into src/web in separate PRs and then started working on the Nginx config and Dockerfile.

I originally posted a PR with an Nginx config that would put the Telescope app and the Docusaurus app in two different ports but I was told in the review process to keep everything on port 8000 and simply have Docusaurus at /docs which makes a lot more sense.

I was stuck for a while on getting the Docusaurus app to show on localhost:8000/docs as I kept getting a 403 Error message. Eventually, with help from Josue, I managed to make the configuration work.

Dockefile

The Dockerfile is divided into different layers. It starts with node:16 as base and then goes through separate layers to install dependencies and build the apps. Each layer is specific to either the Telescope app or the Docusaurus app so that if there's a change in one, not everything needs to be rebuilt from scratch.

The final layer uses the Nginx Alpine image (Alpine images are famous for being much smaller in size) and inside we copy the builds from previous layers like this:

FROM nginx:1.20.2-alpine as deploy

WORKDIR /

COPY --from=build-app /app/out /var/www/data

COPY --from=build-docs /docs/build /usr/share/nginx/html/docs

Nginx config

For Nginx, I had to add following to the config template:

location /docs {
    root /usr/share/nginx/html;
}

and change the baseUrl in docusaurus.config.js to /docs/.

It definitely took longer than I expected to get all of this working since I am still new to Docker but it was very rewarding to see it work. Hopefully this PR can be merged soon.

Finally, since we are getting close to release the Docusaurus app to production, I also took the opportunity to fix all the broken links with this PR.

Slow and steady

Francesco Menghi — Mon, 28 Mar 2022 15:11:26 +0000

This past Friday we released version 2.9 of Telescope and we are now closer and closer to the big final 3.0 release.

Breaking things down

I received feedback on my PR from the previous week about sharing a single Nginx container for all static sites. The restructuring of the directories made the PR too big with hundreds of file changes (although most just renaming). To make it easier for reviewer (and for myself!) I decided to split the work into multiple PRs. I started with moving the src/web directory to src/web/app.

I searched again through the whole project for paths that needed to be updated. This more narrow focus on a smaller task helped me find some relative paths that I missed in the original PR. This really shows the benefits of splitting a big task into smaller pieces to make it more manageable.

Before merging this PR we needed to update the path on Vercel and give a heads-up to other students working on frontend issues that they needed to rebase their branch before moving forward.

I still have to do the same with the docusaurus and test-web-content directories and then make the necessary Docker and Nginx changes.

Frontend changes

The other issue I worked on this week was adding a copy button for code snippets. I had to revisit a lot of Web APIs features so I spent a lot of time on MDN (I love this website, there's always something new to learn on it). I originally used MouseEvent to capture the mouse movement and add a copy button to the DOM. Tue helped out by suggesting to use mouseenter and mouseleave so I made the appropriate changes.

As of Friday the copy button was working but the code was not great and it used the text 'copy' instead of an icon. Because of this it did not make it into the 2.9 release.

Busy with an endless Android assignment that took over my weekend, I did not manage to continue to work on it. To my surprise, Duc took over the PR and made it truly great! His final version turned the button into a React component that feels a lot more at home on Telescope than my original version.

Working on this reminded me that Open Source is a team sport and we really shouldn't be afraid to get help from others that might have more experience.

Tue and I got the ball rolling and then Duc scored the goal. In the end we all win! 🔭

Towards Telescope 2.9

Francesco Menghi — Sat, 19 Mar 2022 21:38:46 +0000

By the time we ship the last release this semester (Telescope 3.0 on April 21th), I would like to have finished our monorepo tooling setup with Turborepo. So far we have made great progress as a group but there are still problems popping up left and right, especially with ESlint.

Monorepo tooling

As of right now I am not assigned to any of these remaining ESLint bugs but I have been active fixing bugs related to CI, reviewing PRs and offering help to my peers.

We are using Turborepo to manage ESLint in each package but there is more that can be done. We can use Turborepo for our tests as well so I will have an issue posted about it.

Docker and Nginx

In the meantime, I have been working with Docker and Nginx because I wanted to get out of my comfort zone and learn more about these two very popular tools.

I am very interested in Docker but I have only played with it on my own so far. I have learned the basics after watching all of the CCP555 lectures related to it.

I picked up an issue about sharing a single nginx container for all static sites. To do this I had to restructure the project to have both the Telescope app and our new Docusaurus docs to share a single Dockerfile. Inside this new Dockerfile, there will be a single Nginx container to deploy both sites.

I like this change because it will group all of our static sites into src/web. The PR is still a work in progress but I am hoping to have it ready for Telescope 2.9 next week.

Last piece of the puzzle: Moving Satellite inside Telescope

Francesco Menghi — Sat, 12 Mar 2022 17:06:40 +0000

This week we released Telescope 2.8 and, as part of the new release, we moved the Satellite project inside of the Telescope monorepo. Satellite is the microservice framework used by Telescope.

I am excited about this change because until now Satellite was the only piece of the telescope project that was in a separate repo. Now we will be able to work on Satellite all from within the monorepo!

This is only possible because we can publish the Satellite package to the Npmjs registry using pnpm -r publish as I discussed in my previous blog post.

The initial idea was to copy the project's files over to src/satellite in Telescope and then add to package.json a section about the original contributors like this:

{ "name" : "Barney Rubble",
  "email" : "b@rubble.com",
  "url" : "http://barnyrubble.tumblr.com/"
}

However this was not great because we would have lost all of the git history. Thanks to Duke, I discovered a very useful git command called subtree.

Git subtree

This is the full command:

git subtree add --prefix src/satellite/ git@github.com:Seneca-CDOT/satellite.git HEAD

Let's break it down.

--prefix: this tells git where to add the repo. In our case it's in src/satellite.

git@github.com:Seneca-CDOT/satellite.git: Satellite repo remote address.

HEAD: reference to the latest commit in the Telescope branch.

After successfully running the above command, we get this commit:

commit 85522ffc4205da8c6a6f736073f55aaff595c382
Merge: 8360d5dc 5491bbb0
Author: Francesco Menghi <53121061+menghif@users.noreply.github.com>
Date:   Tue Mar 8 09:55:48 2022 -0500

    Add 'src/satellite/' from commit '5491bbb058d121552a3191b49199d75cfc56956b'

    git-subtree-dir: src/satellite
    git-subtree-mainline: 8360d5dc434878d92527bc134882ea1aed003e53
    git-subtree-split: 5491bbb058d121552a3191b49199d75cfc56956b

git-subtree-mainline is the last Telescope commit and git-subtree-split is the last Satellite commit.

ESLint fixes

Once the Satellite PR was merged, I worked on the ESLint config for Satellite (Issue, PR). While doing this, I discovered that all of the other package's ESLint configurations were not setup properly and so we were not actually linting our code at all.

Since we had an ESLint config for the whole repo before and now we have a config for each package within the repo, we are still getting used to the change.

Basically we were missing the eslint and @senecacdot/eslint-config-telescope dependencies in each package so the command pnpm lint was not actually doing anything.

This is now being fixed and tracked in this Metabug. Soon ESLint will be fully functional again and faster than ever thanks to Turborepo.

Publish packages from within a Monorepo

Francesco Menghi — Tue, 01 Mar 2022 02:47:24 +0000

In this blog post I will go over the details of how we manage and publish packages inside our Telescope monorepo.

David recently shared a link where I found a good definition for monorepo that I wanted to share:

A monorepo is a single repository containing multiple distinct projects, with well-defined relationships.

Within Telescope, we have just that:

└── src
   ├── api
   ├── backend
   ├── docs
   ├── mobile
   └── web

We have all of the APIs, backend, frontend, mobile app and the new Docusaurus docs all inside the same repo.

We recently made the switch to Turborepo and to try out its features, I have changed the way we lint our project. We used to run eslint from the root of Telescope, but now we have individual .eslintrc.js files inside each project and let Turborepo manage it all.

Since version 1.1, Turborepo config moved from package.json to its own turbo.json file.

This is our turbo.json that defines the lint command inside its pipeline:

{
  "pipeline": {
    "build": {
      "dependsOn": ["^build"]
    },
    "lint": {
      "outputs": []
    }
  }
}

When we run pnpm turbo run lint, Turborepo will go over each package defined in pnpm-workspace.yaml and run the lint command. This process is very efficient because it keeps a cache and reuses it if there are no changes to the files in the project.

For example, if I make changes to the docs, Turborepo will only run eslint in the docs, and use the previously created cache for everywhere else.

We have a new @senecacdot/eslint-config-telescope package that contains the rules we reuse with every .eslintrc.js and it lives inside the tools directory:

└── tools
   └── eslint
      ├── index.js
      └── package.json

This package is published to the npmjs registry. To call it inside a .eslintrc.js file we use extends:

module.exports = {
  extends: '@senecacdot/eslint-config-telescope',

  overrides: [
        // more rules here
  ],
};

Since @senecacdot/eslint-config-telescope lives inside the monorepo, we have automated the release of the package thanks to pnpm publish.

Release to npmjs registry

There is a GitHub Actions workflow that handles the release of Telescope and, at the end of it, I added:

      - uses: pnpm/action-setup@v2.0.1
        with:
          version: 6.30.1
      - name: NPM Publish
        uses: actions/setup-node@v2
        with:
          node-version: '16.x'
          registry-url: 'https://registry.npmjs.org'
          cache: 'pnpm'
      - run: pnpm install
      - run: pnpm -r publish --no-git-checks --access=public
        env:
          NODE_AUTH_TOKEN: ${{ secrets.NPM_TOKEN }}

The key part here is pnpm -r publish. The -r stands for recursive: pnpm will go look at each package.json to see if a package has an updated version and, if so, will publish it to the registry.

Telescope Sheriff

Francesco Menghi — Mon, 21 Feb 2022 16:37:17 +0000

This past week it was my turn to be sheriff for the Telescope project. There were a lot of responsibilities and a lot of tasks to complete.

A big part of the role is to lead the weekly meetings, go over outstanding PRs and assign tasks for the following release. It was my turn to be at the helm of the ship and to always be available on Slack if any problem occurred. Looking at all the incoming PRs forced me to have a more holistic view of Telescope and learn more about all the great things other students are working on.

My outstanding PR about Turborepo linting was merged and I will have another blog post to talk about it in more details.

Breaking Telescope

Everything went pretty smoothly until it was time for the release. While on a call with Tue and David, releasing version 2.7 of Telescope first broke production and then staging.
The call quickly turned into a one hour debugging session and I experienced first hand how to debug Docker containers from David (Which is something still new for me).

The issue was caused by a recent PR that changed SECRET to JWT_SECRET (To use with the latest version of Satellite) without adding the required variables in env.staging and env.production.

Project management

Over the week I reviewed many more PRs than in the past and merged a lot of them into master myself. I also added milestone labels and tags to keep things organized.

This experience made me realize how much non coding related work there is to manage an open source project. I now understand why teams in big projects have people specialized in doing this all day and keep team members accountable and on track with their work.

Packages, Docs and Accessibility

Francesco Menghi — Sat, 12 Feb 2022 03:40:03 +0000

There has been a lot of activity within the Telescope project and it's going so fast that it is hard to follow everything. From my end, I am still working a lot on monorepo tooling with Turborepo, but I also decided to pick other interesting issues.

Monorepo local package

After reviewing Thanh's Pull Request about starting to migrate eslint to our microservices (with the goal to speed things up with Turborepo), I realized that we would end up with many .eslint.js files containing a lot of duplicate code.

It turns out that it is possible to have a local package in a monorepo and share it to our microservices without having to publish it. So, the idea I had was to create a local eslint-config package and reuse it in all of our microsevices, thus reducing any duplicate code.

This is possible thanks to the pnpm workspaces feature. For example to add the eslint-config package to a microservice, we would add it as a dependency in its package.json like this:

"eslint-config": "workspace:*"

This will search for the eslint-config package inside each workspace defined in pnpm-workspaces.yaml instead of searching the npmjs registry.

A local package seemed like a great solution in my mind by after opening the PR, we quickly realized that we still rely on npm in some places, so we will not be able to follow this path.

The other solution suggested by Dave is to actually publish the eslint-config package to the npmjs registry. The question is, how do you automate the release of a package that lives inside a monorepo?

Zoltan, the author of pnpm responded to Dave's tweet and said that we can actually use pnpm publish.

According to the docs:

When running this command recursively (pnpm -r publish), pnpm will publish all the packages that have versions not yet published to the registry.

I still have to test and see if it will work, I will have an update on next week's blog, but I am hopeful!

Accessibility Improvements

Something that I really enjoyed this week was diving deeper into accessibility standards for the web. I started by picking this issue about improving the accessibility of Telescope.

Since the main Telescope page is a feed of many blogs, we cannot control if the authors use proper accessibility standards. So what I decided to tackle was Telescope's About page.

Using Google Lighthouse, I found three accessibility issues:

The fixes take the page to a perfect 100 accessibility score which is nice to see.

Docusaurus!

Last semester I loved working with Docusaurus but I was left hungry for more. This week, Cindy migrated the telescope docs to Docusaurus and I had to get involved!

I reviewed Cindy's PR and also worked on improving the Getting Started page by taking advantage of a cool Docusaurus feature called tabs.

Since Docusaurus is build with React, we can actually use JSX in Markdown (which is actually MDX... mind blown!) and that's how the Tabs feature is implemented.

The idea was to separate the Docker installation guide based on the operating system (Windows, macOS, Linux) to simplify the reading experience.

I think the end result is great and I cannot wait for the Docusaurus documentation to ship!

Fixes and documentation

Francesco Menghi — Sat, 05 Feb 2022 02:48:53 +0000

This week I spent some time helping other students getting acquainted with Turborepo by running a meeting and answer questions. It was a good exercise since I don't get to present very often and I was reminded that having to explain something to others is one of the best ways to learn. I'm happy that there is excitement around Turborepo and that more people are getting involved.

After the meeting I wrote a small documentation page (Issue, Pull Request) for how we use Turborepo within Telescope to be able to explain the basic commands to newcomers.

CSS Fix

We have recently finished the mobile version of the GitHub info sidebar and it looks great but, after every new feature there are always bugs trailing behind.

One bug resurfaced recently. There was a horizontal overflow at certain window sizes that was caused by the addition of display: 'flex'. We used the flex layout for the mobile view but we don't need it for the desktop view.

To fix this I put display: 'flex' and flexWrap: 'wrap' inside of theme.breakpoints.down(1205) to use flexbox only for the mobile view (Window size smaller than 1205px).

Towards Telescope 2.7

There is an ongoing issue that I am trying to address: moving the Satellite repo inside Telescope to complete the monorepo. I haven't been successful so far, so I will ask other students to collaborate on this one, and maybe it can be done by Release 2.7.

Finally I am looking to expand my knowledge of Docker so I assigned myself to this issue about modifying one of our containers to use a smaller image. I like everything that has to do with optimization and speeding up workflows, so this seems like a great fit.

There's a lot of activity within the Telescope community so I'll be on the lookout for more issues to hack on.

Turborepo, React Native and more

Francesco Menghi — Sat, 29 Jan 2022 16:22:48 +0000

As I was writing this blog post last night, I got distracted by a dependency vulnerability alert on telescope. I ended up merging the automatic PR from renovatebot and then did a patch release. We are now at version 2.5.4. Our fourth patch release in the span of one week!

One thing I love about Open Source is the flexibility. We have plans and a release schedule to keep everyone on track but, if I have an idea (or if I find a bug), I can bring it up, send a Pull Request and the unplanned change quickly finds its way to the master branch.

This week I had two of these moments while working on bigger tasks.

The first was fixing a css bug I noticed while using telescope on Safari on my mac. It can be tedious to find the reason why Safari is not acting the same as Firefox and Chrome (Safari is generally behind when it comes to implementing some CSS properties). However, after some trial and error and some luck, I managed to fix the bug by removing just on line of code.

The second small fix was an update that I suggested for our GitHub Actions CI implementation. I noticed that all of our tests (that usually take about 10 minutes to complete) would run even for a simple change that does not impact the code such as updating the docs.

I followed the GitHub Actions documentation and I added a paths-ignore option to ignore the /docs folder for the workflow that we run every time something is pushed to a PR or to master. There is now a docs specific workflow that is much faster because it only includes Prettier and ESLint (1 minute in total run time).

Other than these two small changes, what I worked on this week (and what took most of my time) involved Turborepo and Expo.

Turborepo

I briefly talked about Turborepo in my previous blog post: It is a fantastic tool to manage a monorepo like telescope. This week my PR to add support for Turborepo was merged and we are now ready to use it with pnpm. As of right, now it is only used for the build command: Using pnpm turbo run build will go look inside each workspaces (using pnpm-workspace.yaml) to look for a build command and then will run them all.

Other students have shown interest in the Turborepo setup. In the coming weeks, we will work together to get to a point where we can use this tool to its full potential.

React Native with Expo

The last thing I worked on this week was setting up a new React Native project within the telescope monorepo.

As a group we decided to use Expo, which is a framework to manage a React Native app. The problem I ran into was that Expo is not currently compatible with pnpm. As a workaround we now change directory to src/mobile where the React Native app lives and then use npm there.

I tested the app on iOS, Android and the web and it works on all 3 platforms (The PR was merged yesterday). The app is empty for now but by the end of the semester we should have something functional and maybe even publish it to the App Store and Play Store!