DEV Community: Richard Rosko

Setting up an infrastructure for a monorepo on Render

Richard Rosko — Fri, 08 May 2020 07:58:23 +0000

I feel that DevOps is something that has not yet been solved and all the existing providers (AWS, GCP, Azure) leave much to be desired. That is why every time we are starting a new project, I always do extensive research on existing solutions and new players on the market. At the beginning of 2020, we were, once again, in a situation where we had to research and pick a provider for a new project we started working on. During that research, we stumbled upon Render.

We were looking for a quick setup and wanted to spend less than a full day on infrastructure. That is why we were looking for something simple, fast and reliable. We decided to give Render a try and knew that there was always the option to fall back to AWS. Render is a fairly new cloud provider you might have never heard of. They have launched in 2019 and won the TechCrunch Disrupt that year. They have a lot going for them.

I'd like to first talk about the infrastructure of our app, so you can see what moving parts we wanted to have. We will also take a look at all that Render has to offer and see how we managed to utilize it in our project. Lastly, I'd like to talk about some shortcomings and take a look at what Render can not help you with (yet, since they are adding new features all the time).

Our infrastructure and Render

We are building a travel platform for companies. The app that we are building consists of a GraphQL backend, combined with a Postgres database for data persistence. On the frontend, we have a user-facing React application that uses Apollo for communicating with our backend. We will also have a standalone Node application that is doing ELT for us. This background application also has its own Postgres database. We will also host a static landing page, that will be built with gatsby.js.

Render has our back on most of the things we will need. But first, let's set up our environment for local development.
For this particular project, we will use a monorepo, which means all of our services will live in one directory and under one repository. Fortunately, it's easy to work with monorepos on Render even though it requires some workarounds. But more on that later.

API

On the backend, we will be running our development environment in Docker, since we want to make sure everybody on the team has the same environment. Another advantage here is, that most machines do not come with Postgres pre-installed and even if it did, it might not be the same version. We can avoid a lot of headaches doing it this way. In production on Render, we will not be using Docker (although we could because it is supported). Let's spin up two services in our Docker with this docker-compose.yml in our api project:

version: "3"
services:
  db:
    image: postgres
    restart: always
    ports:
      - "5432:5432"
    environment:
      POSTGRES_USER: "postgres"
      POSTGRES_PASSWORD: "postgres"
      POSTGRES_DB: "nextretreat_dev"
    volumes:
      - db-volume:/var/lib/postgresql/data
  app:
    build: .
    depends_on:
      - db
    ports:
      - "3001:3001"
    env_file: .env
    volumes:
      - .:/home/app/
      - /home/app/node_modules
volumes: db-volume:

Our Dockerfile will look like this:

FROM node:12.14.1-alpine
EXPOSE 3001
WORKDIR /home/app

COPY package.json /home/app/
COPY package-lock.json /home/app/
RUN npm install

COPY . /home/app

CMD ["npm", "run", "start:dev"]

This is a pretty standard configuration if you're used to working with Docker, but there is one thing to keep in mind that is specific to Render and that is the way of specifying npm version. Locally, it's always a good idea to have an .nvmrc file that will help the developers on the team to use the same npm version. In order to let Render know what version it should use, we need to add a .node-version file in the root of the repository. For this project we are using 12.4.1, so the only thing you need to do is paste that in the .node-version file. Every time a service is built / re-built on Render servers, it reads the file and uses that for npm.

Long-running service

Next up is our ELT Node service. It will live in our monorepo, in a directory called data-service. It is a simple app with an index.js file in the root of the directory that will manage the extraction and loading of the data into our warehouse. As we mentioned before, it has its own database that it stores the information in, so we will once again use Docker for that. We will re-use our docker configuration from our api service. There are only a couple of things we will change here:

We have to use a different port on our computer (the first one of the port:port directive in docker-compose.yml) because that would prevent us from running both api and data-service at the same time. Let's change it 5433:5432 instead of 5432:5432
We can remove the EXPOSE 3001 from our Dockerfile, since this app does not require it

For these kinds of long-running services, you would most likely want to leverage some sort of restart-on-error tool, like forever, or pm2. But we will leave all of this up to Render and its Background Worker service. The start command for this service is a simple node index.js.

Static site

For our static site we will be using Gatsby, which is an awesome tool for building static applications. We will use a simple template, add in our code for the landing page in the landing-page directory and specify the build command as gatsby build. Not much else to do here.

User-facing client

The last app in our repository is the web client. Following the super simple documentation, we will create and move into our web-client directory in the monorepo and scaffold our application using npx create-react-app my-app. We will specify two main scripts for this app. For local development, we will have a react-scripts start command and the build command will be react-scripts build. Super easy. Now we can start developing our app as a regular CRA app.

The only thing left to do for now is to git push this project to Github (or another git provider).

Setting up Render

We now have our monorepo ready with all the services we will be using locally.
Let's recreate our services on Render for the production environment and hook it all up. After creating an account and linking it to our github repository we are presented with a plethora of services we can choose from. For this project, we will not be using Docker in production and this is still going to be extremely easy to set up. We are going to start with databases.

Databases

As of writing this article, Render only supports one type of database and that is Postgres. Fortunately, that is exactly what we want to use in this case anyway. Setting up a database is very easy, we just head to the Databases section and create a new database. If your database increases in size during later stages, no worries, you can easily scale it up with a click of a button. Scaling down, however, is not supported yet, so make sure you pick right at this point so that you won't pay for something you will not end up using. We will create 2 databases for now, one for the api and one for our data-service.

When you click on a detail of an existing database you can see a pair of connection strings. It is pretty self-explanatory, but if you are connecting to a database from within a Render-hosted service, you use the internal connection string. If you want to connect to the database from outside (GUI app, or a service that is running elsewhere), you use the external connection string.

API

Let's navigate to the Services tab and create our api service. We will be using a Web Service and choose Node as our environment. Next up, we specify the master branch for the production version of our app (we follow the Git-flow Workflow).

Since this project is living inside of a monorepo we have to remember that when it's cloned, it is located in the root of the repository. We therefore need to prepend all our commands with a cd <service-directory> in order to be in the correct folder. The build command is here to make sure you have any prep work, like installing dependencies, or setting up external configuration ready before you start the service. In our case, it will be: cd api && npm install. The start command is cd api && npm start.

After the service is created, there is only one thing left to do and that is to create our list of production environment variables. For the database connection string, we can just copy the internal connection string and, voilà, our api service is hooked up to our database. What's more, if you have more services that will use the same environment variables you can also create environment groups and assign those to different services here.

Long-running service

For our long-running service, we will return to the Services tab and create a New Background Worker. As a sidenote, background worker is ideal for processes like this one. They need to run indefinitely and do not expose a port, whereas web services are meant to be accessed by the outside world. We will once again opt-in for the Node environment and specify the master branch for building. Our build command is: cd data-service && npm install. Our start command is: cd data-service && npm start. You can also scale up all of your services horizontally by running multiple instances of that service. This might be useful for a data acquisition service like this one, but let's leave it on 1 for now and see what we will require in the future.

Static site and user-facing client

Setting up static sites is extremely straight forward and if you want to try Render out, your first static site is even free of charge.
For our landing page, we will select Static Site as our environment when creating this service and make cd landing-page && npm install && npx run gatsby build our build command. We specify our publish directory to be ./landing-page as instructed in our template.

In order to deploy our web-client, we create another Static Site and make our build command cd web-client && npm install && npm run build. The publish directory will be ./web-client/build, since this is a CRA app.

After all of this, we can kick back and relax, because we are now running all our services in the cloud on Render. Everything is hooked up, and we can turn on auto deploys if we choose to. This will make sure, that every time something is pushed to our repository, on the specified master branch, our services get updated and are re-built with the newest additions.

Shortcomings of Render

At the time of writing this article, Render doesn't currently support CDNs and object storage, whereas something like AWS along with S3 has these capabilities. For now, it is a disadvantage, because we have to use another service for our CDN and object storage.

It also doesn't have real support for monorepos. If you do it the way I described, every time you push something to any service in your repository, the hook registers the command on all the other services you have and rebuilds everything. This might not be ideal, but it works and is not that bad.

Conclusion

We usually use AWS on larger projects and my main beef with Amazon is, that it is very complicated to set up. Render on the other hand (kind of like Heroku, or Digital Ocean), has preexisting services that are super easy to set up and scale well at the same time. Although there were inquiries from the community, they do not plan to open source the configurations for their services, because they believe that it would defeat the purpose. It would go against the one-click setup functionality, and I have to say I am with them on this one. This is what makes Render special and so easy to use.
For us, it met the requirements we had for it, as we've been able to easily set up this infrastructure in less than a day. That was what we initially wanted to achieve and Render delivered.

Even though it is missing some services and features, the team is very active in delivering new features. While the documentation is pretty complete, some edge cases are not covered and since Render is still pretty new, the internet might be short on answers. But the community is super helpful and the Slack channel is bustling. Whenever I had an issue, a team member always reached out and helped me with whatever I was dealing with.

This post was originally published on sudolabs.io's blog

React Native Navigation and its caveats

Richard Rosko — Tue, 14 May 2019 12:48:41 +0000

So you went out on a journey to create a react-native application and you feel like you have superpowers. You create your components, style them, make API calls just like you do on the web and you're practically walking on sunshine.

But then you want to implement navigation and the world starts crumbling around you. You try out the navigation that comes pre-packaged in React Native, but it's not cross-platform consistent. Then you discover a JavaScript solution, but the performance is sub-optimal for most apps. You need a solution that uses native navigation definitions. And then you discover react-native-navigation (RNN from now on) from Wix and you are saved as it has all you need.

It is a native solution, that boasts great performance, uses native events on the background and is cross-platform in its true meaning.

Unfortunately, it has some weird issues and you are once again lost. Do not give up hope yet though! Rather than that, let's walk through the breaking points and jot down some notes on how to work with/around those weird issues.

`v1` VS `v2`

The guys over at Wix are working really hard on providing all of us with a native cross-platform solution to navigate in React Native apps - huge kudos to them for maintaining the library. With that being said, the documentation is a bit rough around the edges and sometimes it's hard to find what you are looking for.

A quick look into the the ecosystem of RNN will reveal, that there are two versions of the library that are currently being maintained - the older v1 and the "brand" new v2.

The differences between the two versions are marginal and although there is a way to migrate v1 to v2, it's best to make your mind up before starting a new project.

The question is, which one should you use for your new React Native project?

The quick answer is: v2.

The long answer is, that both versions are still actively maintained, but v2 has a priority in the development process. And although there are some features, that are not yet implemented in v2, the team is very active in making that a thing of the past. Also, the official recommendation from Wix is to use v2 in your projects and so, we should stick with what the authors are saying in this case.

Stacks

This is something you will probably use in any app that uses navigation. A general consensus of what a stack actually is would probably be that is it a collection of screens that are somehow related and, therefore, can be navigated to and from. This does not, however, directly translate to how the stacks work in RNN applications.

In my opinion, it's just an issue with naming here. stack in RNN simply represents an arbitrary screen, that you can use to navigate to any other arbitrary screen. Don't try to find any deeper meaning in the name of the definition here.

The way you define a stack in and RNN app is this:

stack: {
  children: [{
    component: {
      name: 'screen.Home'
    }
  }]
}

If the Home screen is registered with this definition you can use Navigation.push() (and all other navigation events) from the Home screen to any other registered screen. They do not have to be defined in the same stack: {} in order for those transition to work. This was an issue for me when I started using RNN as it makes little sense to me given the naming. I hope you learn from my mistake and navigate properly with stack definitions from now on.

One more word of caution here though - I have seen people who define all of their screens in stacks to make sure they are able to navigate from them if the need arises. I wouldn't personally do that, because if you define a screen in stack it naturally adds some overhead code on the native side to your screens that are defined this way and that is not optimal. You are better off paying attention to the definitions of your screens and only define the screens which you will be navigating from with Navigation events. Such is the will of RNN.

Wrapping registered components in Redux (or any other Provider)

Can you use RNN in conjunction with Redux? Yes, you can. But you have to do some prep work to make it possible. You will probably find many different ways to do it if you make a quick Google search, but there is only one official way and that is through the RNN's own API. The only thing you need to do is use

Navigation.registerComponentWithRedux()

to register your components. This function accepts exactly three parameters, the first being the component you want to register, the second being the Provider (import { Provider } from react-redux) and the third being your custom redux store. In the end, you should have something like this:

Navigation.registerComponentWithRedux('screen.Home', () => Root, Provider, store)

Why stop there though? Even though the method is called registerComponentWithRedux, what it actually does is, it wraps all of the defined components with whatever component you provide it as the second parameter. Therefore, this is the ideal spot to wrap your component with whatever else you need to, like a ThemeProvider, or, e.g. React's Contexts if you use those in your app.

Not only is this the ideal place to wrap your app in any wrappers, but it's probably one of the only places you can do so. Why is that you might ask? Well, you probably already figured it out when you were initializing RNN in your app. The way RNN works is it pre-registers all of your screens ahead of the time for optimization purposes. This is a good thing, but it also unavoidably means, that the entry point of your app changes from being declarative (that's the way, uh huh huh, we like it, uh huh huh) to being imperative.

React likes it much more, if we use declarative definitions for our codebase (layouts and all). But if you use RNN in your app it's just not possible to do that for the root components of your app. This is my main beef with the library, but that's a topic for another time and at least there is a way to wrap your top-level components, albeit not being the most user-friendly.

`componentId` and making peace with it using `React.Context`

As mentioned in the first article, there are two versions of the library. Let's assume that you are using the preferred v2. One of the main changes to the API of the library is the presence of componentId - a unique identifier of any RNN registered component. You have to use it anytime you want to perform a Navigation event (e.g. .push(), .pop()) and each root component that is registered in the initialization step of an RNN app receives it as a prop. This fact could prove problematic and here's why.

Let's say you want to create a reusable component for your app that needs to perform a navigation event as part of its functionality. You could define it as part of your route stack, but that wouldn't really make sense from a functional point of view. It is not a specific screen that the app displays, but just an arbitrary component that could be reused. This approach could bring its own fair share of problems, like unnecessary code duplication and complicated definitions in the scope of a component.

This is better demonstrated on an example. Let's create a component that acts as a select screen with many options that displays an input and when tapped, it presents a separate screen that shows all the possible values for this input in a table. One of these components, let's call it ListSelect:

Since you need to navigate from the screen that is currently displayed when the user interacts with this component you need to have the componentId of the mounted screen ready for the Navigation event in our ListSelect component. If the ListSelect component is used directly in the screen that has a componentId, that wouldn't be a problem. You'd just need to pass it as a prop to the component and then inside of it you would receive it and perform the event, like this:

const navigateToSelectScreen = ({ rootComponentId }) => {
  Navigate.push(rootComponentId, {
    component: 'screen.ListSelect'
  })
}

const ListSelect = ({ rootComponentId }) => (
  <View>
    <TouchableOpacity onPress={() => { navigateToSelectScreen({ rootComponentId }) }}>
      <Text>
        Select a value
      </Text>
    </TouchableOpacity>
  </View>
)

But what if you use it extensively and you are sick of doing the <ListSelect rootComponentId={componentId} />? Or even worse, what if the component is nested deeper in the hierarchy and you'd need to prop drill it three, or even more levels? There has to be a better solution. Enter Provider.

In the previous section we were talking about how we can wrap our root components with a custom provider. What we could do here, is wrap our top level components with a Context Provider that stores the componentId property which we can then access in a Consumer to use literally anywhere else in the app and down the hierarchy tree. Context is a great way to lift such a UI specific piece of data and make it accessible in the hierarchy, but you could use any other global state management library, e.g. Redux, or Flux with the same results.

Let's get back to our example and trace the steps we need to make to ensure we can use the component anywhere with no hassle. First of all, we define the screen we will be navigating to, from inside the component in our stack, like this:

Navigation.registerComponentWithRedux('screen.ListSelect', () => ListSelect, Provider, store)

The next thing we need to do is create a Context that will keep track of the currently visible componentId for us, let's call it NavigationComponentIdProvider (duh). It's going to look something like this:

class NavigationComponentIdProvider extends React.Component {
  state = { navigationComponentId: '' }
  componentDidMount() {
    this.navigationEventsListener =
      Navigation.events().registerComponentDidAppearListener(({ componentId }) => {
        this.setState({
          navigationComponentId: componentId
        })
      })
  }
  componentWillUnmount() {
    if (this.navigationEventsListener) {
      this.navigationEventsListener.remove()
    }
  }
}

In the componentDidMount we set up a listener that updates the state anytime a navigation event changes the active component. RNN has a useful listener for that using the provided

  Navigation.events().registerComponentDidAppearListener()

method that receives the componentId when it is called. We then simply update the state with this information. Do not forget to remove the listener in componentWillUnmount.

We wrap it up by finishing the render method, which renders the children wrapped with the Context's Provider. This should be the end result of our newly created Context that knows the currently displayed componentId at any given time.

class NavigationComponentIdProvider extends React.Component {
  state = { navigationComponentId: '' }
  componentDidMount() {
    this.navigationEventsListener =
      Navigation.events().registerComponentDidAppearListener(({ componentId }) => {
        this.setState({
          navigationComponentId: componentId
        })
      })
  }
  componentWillUnmount() {
    if (this.navigationEventsListener) {
      this.navigationEventsListener.remove()
    }
  }
  render() {
    const { children } = this.props
    const { navigationComponentId } = this.state
    return (
      <NavigationComponentIdContext.Provider value={navigationComponentId}>
        {children}
      </NavigationComponentIdContext.Provider>
    )
  }
}

const NavigationComponentIdConsumer = NavigationComponentIdContext.Consumer

export { NavigationComponentIdConsumer, NavigationComponentIdProvider }

The next step is to update our global Provider that we wrap all our top-level components with to include the NavigationComponentIdProvider.

Our Provider should look something like this:

const Provider = ({ children, store }) => (
  <ReduxProvider store={store}>
    <ThemeProvider theme={theme}>
      <AnyOtherProviderThatYouMightBeUsing>
        <NavigationComponentIdProvider>
          {children}
        </NavigationComponentIdProvider>
      </AnyOtherProviderThatYouMightBeUsing>
    </ThemeProvider>
  </ReduxProvider>
)

If you have followed the tutorial up to this point, you can now import { NavigationComponentIdConsumer }' and get the componentId of a displayed screen anywhere in your app.

To make the whole process even simpler and cleaner, we can create a higher order component (HOC) that wraps the component with the NavigationComponentIdConsumer. It's a very simple abstraction and it will help us to keep the resulting code much cleaner.

The HOC could look something like this:

import { NavigationComponentIdConsumer } from './NavigationComponentIdContext'

const withNavigationComponentIdContext = Comp => props => (
  <NavigationComponentIdConsumer>
    {componentId => <Comp componentId={componentId} {...props} />}
  </NavigationComponentIdConsumer>
)

export default withNavigationComponentIdContext

The last thing we do is wrap the export of our ListSelect component in withNavigationComponentIdContext, like so.

ListSelect.js
...

export default withNavigationComponentIdContext(ListSelect)

The component now has access to the componentId from its props and we do not need to pass it in manually.

The great thing about this is, that you can use this HOC anywhere in your app, so you needn't worry about the componentId anymore if you don't want to . If you are in a situation where a component that is on a lower level in hierarchy needs to make a Navigation event, don't drill the componentId manually, just wrap it in withNavigationComponentIdContext and get it in the props.

Another good example for this usage would be creating a modal component. Although if modals are very specific in your app, it might make sense to define them as separate screens to make more sense from the semantical point of view.

Takeaways

There are a few more RNN specific issues that come to mind. But most of those are version specific (RNN is updated almost every day!). It's probably a good idea to leave them be and not try to work around them that much as they are bound to be resolved in the near future. This article should serve as a quick write-up with solutions to common issues in RNN that are probably not likely to change, since they are tied to the API of the library.

Other than that, if you have any more issues, I can definitely recommend the Discord community for react-native-navigation. Folks there are very helpful and saved my skin loads of times before. Thanks for reading!

First published on sudolabs.io

DEV Community: Richard Rosko

Setting up an infrastructure for a monorepo on Render

Our infrastructure and Render

API

Long-running service

Static site

User-facing client

Setting up Render

Databases

API

Long-running service

Static site and user-facing client

Shortcomings of Render

Conclusion

React Native Navigation and its caveats

v1 VS v2

Stacks

Wrapping registered components in Redux (or any other Provider)

componentId and making peace with it using React.Context

Takeaways

`v1` VS `v2`

`componentId` and making peace with it using `React.Context`