DEV Community: Ibrahim Awad

Large GeoTiff (raster) files on GeoServer using COG

Ibrahim Awad — Sat, 04 Sep 2021 19:17:30 +0000

Publish large geotiff (raster) files on GeoServer and be able to access them in no time with COG (Cloud Optimized Geotiff)

The challenge here is that you might have a really large geotiff file (or a normal one). Publishing that file on geoserver would require any client to download the whole file before rendering it (not a good thing if you have a 500MB geotiff!).

The solution is fairly simple with COG (Cloud Optimized Geotiff). You can check all about COG on the official website, or you can simply read their explanation of COG as:

A Cloud Optimized GeoTIFF (COG) is a regular GeoTIFF file, aimed at being hosted on a HTTP file server, with an internal organization that enables more efficient workflows on the cloud. It does this by leveraging the ability of clients issuing HTTP GET range requests to ask for just the parts of a file they need.

Let's do it through these steps together:

Download and use COG validation script.

Thanks to rouault for creating this awesome script. Save it to your local machine, we will use it in a second.

Get your large (or normal) geotiff file in the same directory as the validation script.

For the sake of this demo, I will use this geotiff, download it, rename it to envisat.tif and save it next to the validation script.

Validate the file that it is a geotiff, but not yet a COG.

run the following command:



python validate_cloud_optimized_geotiff.py envisat.tif

you should see an output telling you that it is NOT a valid cloud optimized geotiff like this screenshot.

Convert the geotiff file into a cloud optimized geotiff file (COG).

In order to do that, we will use GDAL, make sure you have it installed on your machine. for GDAL 3.2 and later, it's a simple one line command, for older versions than 3.2, it's two separate commands. We will cover both cases.

GDAL >= 3.2 run this command on the geotiff file ```

gdal_translate envisat.tif envisat_cog.tif -of COG -co COMPRESS=LZW

The larger the file, the more time it will take to convert it, at the end you should see an output like this.
![convert to cog gdal > 3.2](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/gzzvhrqxfhc38z9dpmao.png)

Now let's try and validate the file again, run the validation command again, but this time on the **output** file:

python validate_cloud_optimized_geotiff.py envisat_cog.tif

you should see and output message telling you that `envisat_cog.tif is a valid cloud optimized GeoTIFF` like this one.
![valid cog file 1](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/eiko44bdr4zlnnkzqsog.png)

* GDAL < 3.2
We have to make this manually by creating the internal overviews, then create the tiles in the files.

Run the following command to create the internal overviews:

gdaladdo -r average envisat.tif 2 4 8 16

It can take some time depending on the file size, but when it completes, we are ready for the next step which is creating the tiles. Run the following command on the file:

gdal_translate envisat.tif envisat_cog.tif -co COMPRESS=LZW -co TILED=YES -co INTERLEAVE=BAND

This command also can take some time depending on the file size, but when it completes, you should have a valid COG file. We can easily validate that, by running the validation script on the **output** file:

python validate_cloud_optimized_geotiff.py envisat_cog.tif

It should tell you that `envisat_cog.tif is a valid cloud optimized GeoTIFF`

## Halfway done!

You can now have a break and bring in a cup of coffee ☕ (or tea) since we made some good progress here. We have our COG file ready to be served by GeoServer.

Let's continue.

GeoServer doesn't support COG files out of the box, luckily there is a community module that adds support to that.

- Go ahead and open GeoServer's [nightly build](https://build.geoserver.org/geoserver/) directory.

- Choose your version of GeoServer `2.16.x` `2.17.x` `2.18.x` `2.19.x`

- Open `community-latest`

- Search for the word `cog`, you should have only **1** search hit, which is a module named `geoserver-2.x-SNAPSHOT-cog-plugin.zip` where `x` is GeoServer minor version.

- Download the module's zip file and extract the content to GeoServer's `WEB-INF/lib` directory then **restart** GeoServer.

### Congratulations! GeoServer now supports COG

- Open up GeoServer's homepage (usually at localhost:8080/geoserver) and login with your credentials (usually admin:geoserver)

- From the left side menu, under `Data`, select `Workspaces`, then click on `Add new workspace`
![GeoServer Data menu](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/kn8zi6sa1cwxh1dj7wun.png)
![Add new workspace](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/hw4agfdayi7we3qsyrj6.png)

- Create a new workspace called `cog`
![cog workspace](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/not8cyvf0d2dqoct064u.png)

- Navigate to GeoServer's data directory, create a folder called `cogs` then copy and paste or cog file `envisat_cog.tif` into that directory.

- Back to GeoServer's console, from the left side menu, under `Data`, select `Stores`. Every geotiff (COG) file you publish is a complete store. click on `Add new store`.
![GeoServer Data menu](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/kn8zi6sa1cwxh1dj7wun.png)
![Add new store](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/t0ffru7oc3c3okiezuti.png)

- From the long menu of GeoServer's data sources, under `Raster Data Sources`, click on `GeoTIFF`.
![New GeoTIFF](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/46dwr55zmcy3fwmcdkqk.png)

- A wizard will appear, for `Workspace` select our newly created workspace called `cog`.
- For the `Data source name`, I will type `envisat`.
- For the `Description`, I will type `envisat_store`.
- Next to the `URL` field, click on the `Browse` button and select the `envisat_cog.tif` file.
![Browse for Tif file](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/ewlgfilvautwe6kfxic0.png)
- Mark the checkbox `Cloud Optimized GeoTIFF (COG)`
Leave all the other settings as is, you should have something like this.
![Add new Raster Data Source](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/9p33d405yln9eg8onrfi.png)
- Click `Save`

- You'll find yourself in the wizard that creates a new layer (that is a good thing!), and your new layer is ready to be published.
![New Layer](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/6jy5tftxwjis3szmaymp.png)
- Click on `Publish`

- The wizard for `Edit Layer` is opened automatically, I will change the layer name to `envisat` and the title to `Envisat`, then I will click `Save`.
![Save Layer](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/o0lppooiur0csk9w95m5.png)

- The layer is created and added to the layers list.
![Layer List](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/dr8otzio88txjwbcekfb.png)

- Preview the layer from the `Layer Preview` menu item under `Data` from the left menu.
![Layer Preview Menu](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/5tcsjaygh8gs1f04a0kr.png)

- Click on `OpenLayers`

It Does not matter how large your layer is, it will load up in seconds (or less), and when you zoom in/out it will load up almost instantly thanks to **COG**.

![Preview](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/4puhsgwepmayw1wetn7y.png)

Ionic React Manual Dark Mode Switch

Ibrahim Awad — Thu, 30 Apr 2020 11:21:20 +0000

Hello there!
With growing support for dark mode in native apps, developers are now looking to add it to their apps to support user preferences. Ionic makes it easy to change the themes of your app, including supporting dark color schemes.

In a normal scenario, when the user switches to Dark Mode in their operating system (Android - iOS - Web), your application will switch automatically to the dark theme. This happens using Media Queries and to be exact, one media query for the user's preferred color scheme.

In my situation I want to have a manual switch in my application that the user can flip to turn on/off the Dark Mode.

Let's dive right into it.

Steps:

Create a new ionic/react project
Modify the current theme variables to allow for manual Dark Mode switching
Implement the controller to do the magic

Create a new ionic/react project

refering to the official documentation of ionic, I will simply do the following.

npm install -g @ionic/cli
ionic start AppWithDarkMode blank --type=react

Open up your newly created project in your preferred IDE, I like Visual Studio Code, it is super lightweight and supports ton of extensions witch makes your development much easier and much faster.

Now check if your application is running normally by going into the project directory and running ionic serve

cd AppWithDarkMode
ionic serve

Your browser will open up at http://localhost:8100/ by default and you're supposed to see something similar to this

Now moving to the next step.

Modify the current theme variables to allow for manual Dark Mode switching

Open up the project in your IDE, under src => theme there's a file called variables.css, open that up.
You can find two sections, one for the :root tag, and another for the media query we talked about. Inside the media query, there are three sections body, .ios body, and .md body. You can easily guess what these three sections are for. These are the dark theme colors for all different platforms running your application.

Now copy all those three sections (body, .ios body, and .md body) outside of the media query and at the end of the file itself. Sure you can create a separate file and include that in your application, but that's up to you.

Now simply add .dark to each of the newly created body tags so that they can be like this (body.dark, .ios body.dark, and .md body.dark).

Now we're ready for our final step.

Implement the controller to do the magic

I will begin by modifying the Home.tsx file, which is the home page that displays in the app.
My current/auto-generated Home.tsx file looks like this.

const Home: React.FC = () => {
  return (
    <IonPage>
      <IonHeader>
        <IonToolbar>
          <IonTitle>Blank</IonTitle>
        </IonToolbar>
      </IonHeader>
      <IonContent>
        <IonHeader collapse="condense">
          <IonToolbar>
            <IonTitle size="large">Blank</IonTitle>
          </IonToolbar>
        </IonHeader>
        <ExploreContainer />
      </IonContent>
    </IonPage>
  );
};

I will remove the second IonHeader tag in the IonContent and leave the ExploreContainer component, so it will look like this.

const Home: React.FC = () => {
  return (
    <IonPage>
      <IonHeader>
        <IonToolbar>
          <IonTitle>Blank</IonTitle>
        </IonToolbar>
      </IonHeader>
      <IonContent>
        <ExploreContainer />
      </IonContent>
    </IonPage>
  );
};

Now we'll need a couple of imports. Please note that most of the following is to make the UI look good, the functionality itself is fairly simple.
Now the imports will look like this

import React from "react";
import { IonContent, IonHeader, IonPage, IonTitle, IonToolbar, IonList, IonItem, IonIcon, IonLabel, IonToggle} from "@ionic/react";
import { moon } from "ionicons/icons";
import ExploreContainer from "../components/ExploreContainer";
import "./Home.css";

We can use our newly imported components like this:

const Home: React.FC = () => {
  return (
    <IonPage>
      <IonHeader>
        <IonToolbar>
          <IonTitle>Blank</IonTitle>
        </IonToolbar>
      </IonHeader>
      <IonContent>
        <IonList className="ion-margin-top">
          <IonItem>
            <IonIcon slot="start" icon={moon} />
            <IonLabel>Dark Mode</IonLabel>
            <IonToggle
              slot="end"
              name="darkMode"
              onIonChange={toggleDarkModeHandler}
            />
          </IonItem>
        </IonList>
        <ExploreContainer />
      </IonContent>
    </IonPage>
  );
};

You can tell directly that it's complaining about onIonChange={toggleDarkModeHandler}, that's because we haven't implemented that method yet.

Go ahead and create that method inside the Home component like this:

const toggleDarkModeHandler = () => {
    document.body.classList.toggle("dark");
  };

Like I said, the functionality itself is fairly simple.

Now return to your browser, and you'll see that the switch is there and is working!

Extra

We can do a little bit of an extra styling to the icon so that it looks like this

In the Home.css file add the following css classes.

.component-icon {
  border-radius: 50%;

  padding: 7px;
  height: 18px;
  width: 18px;

  margin-top: 5px;
  margin-bottom: 5px;
}

.component-icon-dark {
  background: var(--ion-color-step-850, #27323e);
  color: var(--ion-item-background, #fff);
}

And simply use the new CSS classes on the IonIcon component.

<IonIcon
    slot="start"
    icon={moon}
    className="component-icon component-icon-dark"
/>

That's it!

Complete `Home.tsx` file

import React from "react";
import {IonContent, IonHeader, IonPage, IonTitle, IonToolbar, IonList, IonItem, IonIcon, IonLabel, IonToggle} from "@ionic/react";
import { moon } from "ionicons/icons";
import ExploreContainer from "../components/ExploreContainer";
import "./Home.css";

const Home: React.FC = () => {
  const toggleDarkModeHandler = () => {
    document.body.classList.toggle("dark");
  };
  return (
    <IonPage>
      <IonHeader>
        <IonToolbar>
          <IonTitle>Blank</IonTitle>
        </IonToolbar>
      </IonHeader>
      <IonContent>
        <IonList className="ion-margin-top">
          <IonItem>
            <IonIcon
              slot="start" icon={moon} className="component-icon component-icon-dark" />
            <IonLabel>Dark Mode</IonLabel>
            <IonToggle slot="end" name="darkMode" onIonChange={toggleDarkModeHandler} />
          </IonItem>
        </IonList>
        <ExploreContainer />
      </IonContent>
    </IonPage>
  );
};

export default Home;

Update/Publish Styles on GeoServer from Javascript using REST

Ibrahim Awad — Wed, 29 Apr 2020 22:11:34 +0000

Hi there!
Do you want to publish or update some styles on GeoServer, that's easy, check out the official documentation for that. After you read it and can't use it in your javascript code, come here, this post will help you.

I'll skip the unnecessary chat about various situations you might need this and get right to it.
I'm using axios to handle my connections (because I like it :D), but you can do it with other libraries or even with vanilla JS.

Update an existing style (using PUT)

First prepare the StyleContent which is the content of the style file as if you normally would upload it through geoserver interface.
Make sure you have the right access, I'm using default username and password for geoserver admin and geoserver for this.

const geoserverURL = 'http://{GEOSERVER_HOST}/geoserver/rest/workspaces/{WORKSPACE}/styles/{STYLE_NAME}'
axios({
    method: 'put',
    url: geoserverURL,
    data: StyleContent,
    auth: {
        username: 'admin',
        password: 'geoserver'
    },
    headers: {'Content-Type': 'application/vnd.ogc.se+xml'}
}).then((response) => {}, (error) => {console.log(error);});

Publish a new style

Guess what happens if you change the method from put to post from the previous code? That's correct! you publish a new style to geoserver.

How to create a private route in React (Route Guard) - example: for authenticated users only

Ibrahim Awad — Wed, 29 Apr 2020 02:52:37 +0000

Hello there!
It always happen when you're building a react app and configuring your routes, you find out that you don't want your routes to be all public all the time. Perhaps you want to make some routes accessible only for authenticated/authorized users, or make them available at a condition that makes sense to your business logic.

I have some good news for you, you don't need some fancy third-party library to achieve that, or even be an expert in routing techniques in single-page-applications.
I will demonstrate a simple solution that you can easily configure into your application logic.

Prerequisites:

A working React application with your routes configured and ready to use.
A good cup of coffee (only if you're a coffee fan).

Problem

I have my routes here all setup and working fine.

<Route  path="/page/home"  component={Home}  exact  />
<Route  path="/page/news"  component={News}  exact  />
<Route  path="/page/login"  component={Login}  exact  />
<Route  path="/page/profile"  component={Profile}  exact  />
<Redirect  from="/"  to="/page/home"  exact  />

The only problem with these routes is that any one can access the URL /page/profile and I only want that route to be available when the user is authenticated, otherwise I want him to be redirected to the login page /page/login first.

Solution

I will create a simple jsx or tsx component (works for React JS or Typescript) that wraps around the Route component from the react-router-dom library. It will check for my given condition first, if that's true, it will render that component like it's supposed to do, otherwise it will redirect me to the login page.

Now let's take a look at my component:

import  React from  "react";
import { Route, Redirect } from  "react-router-dom";

const  PrivateRoute: React.FC<{
        component: React.FC;
        path: string;
        exact: boolean;
    }> = (props) => {

    const condition = performValidationHere();

    return  condition ? (<Route  path={props.path}  exact={props.exact} component={props.component} />) : 
        (<Redirect  to="/page/login"  />);
};
export  default  PrivateRoute;

Code walk-through

The functional component expects three(3) props:

The component to render in case of a valid condition Profile Page for example
The path to that component /page/profile for example
Any additional parameters like the exact attribute

Final Result

<Route  path="/page/home"  component={Home}  exact  />
<Route  path="/page/news"  component={News}  exact  />
<Route  path="/page/login"  component={Login}  exact  />
<PrivateRoute  path="/page/profile"  component={Profile}  exact  />
<Redirect  from="/"  to="/page/home"  exact  />

That's it! you've done it, you now have your routes protected.