DEV Community: Petra Grunheidt

Geolocation in web apps made easy with haversine

Petra Grunheidt — Mon, 04 Nov 2024 16:59:30 +0000

In this post, I will be implementing a solution to find the estimated distance of a user to Places saved in my application.

For simplicity's sake, let's imagine we have a web app that displays many restaurants from various cities and states. The web app is connected to a backend service in Rails with a relational database. Before the implementation of the geolocation feature, restaurants were displayed in a random order, which means a user might see a restaurant from another state before seeing one they are more likely to visit.

The requirements for the feature are:

When a user accesses the app, they will have the option to share their geolocation.
Upon accepting, the page should be more personalized based on the distance of the user to the restaurants.
The location doesn't have to be precise, like GPS precision, but it shouldn't be imprecise within a 0.5 km margin.

Collecting the user's location

In the context of a web-app, the easiest way to collect a user's estimated location is using the browser's Geolocation API.

Calling the getCurrentPosition or watchCurrentPosition functions will prompt the user to allow or deny sharing their geolocation
Here is a sample implementation that should prompt the user to share their Geolocation, saves it to localStorage and handles errors with console.log

  const getGeolocation = () => {
    if (navigator.geolocation) {
      return new Promise((resolve, reject) => {
        navigator.geolocation.getCurrentPosition(
          (position) => {
            const location = {
              latitude: position.coords.latitude,
              longitude: position.coords.longitude
            };
            resolve(localStorage.setItem('geolocation', JSON.stringify(location)));
          },
          (err) => {
            if (err.code === 1) {
              reject(console.log('User denied the request for Geolocation.'));
            } else {
              reject(console.log('An error occurred while retrieving location.'));
            }
          },
        ); 
      });
    }
    return Promise.resolve(console.log('Geolocation is not supported by this browser.'));
  };

The data retrieved by this function is the user's approximate latitude and longitude. Keep in mind, this location data can be derived from various sources such as GPS, WiFi, or the user's internet connection network, so it may not always be highly precise. However, it is accurate enough for our requirements and for most e-commerce apps.

You can then send these coordinates to your backend, where you can handle the proximity of the restaurants with an appropriate query.

Sorting Records by Geolocation

The defined requirement is that the page should be more personalized based on location. One way to do that is by ordering the Restaurants by proximity.
Now consider that we have a Restaurants table with address information, such as city, state, street address, but no coordinates. How can we find the distance between coordinates and an address?

There are ways to find the distance between two coordinates, such as the Haversine formula, which is a trigonometric function to find the distance between two points in a perfect sphere. The earth isn't exactly a sphere, but given our precision requirements it's close enough to a sphere. If you want to read more about a precise calculation of Geolocation, you can checkout Postgis.

A possible implementation of the Haversine formula looks like this:

  def self.order_by_distance(lat, lon)
    select("restaurants.*, 
            (6371 * acos(cos(radians(#{lat})) * cos(radians(latitude)) * 
            cos(radians(longitude) - radians(#{lon})) + 
            sin(radians(#{lat})) * sin(radians(latitude)))) AS distance")
    .order("distance")
  end

This is an active record query on the Restaurant model that takes as argument latitude and longitude and compares it to that of all Restaurant records, calculating the distance. It then orders restaurants by the closest ones.
It is based off of this post on applying Haversine to SQL queries.

Luckily, the Postgres team has already implemented this, and it can be used by adding the earthdistance extension.

If you're using Rails and Postgres, you would need a migration like this

class AddsEarthdistanceExtensionToPostgres < ActiveRecord::Migration[7.0]
  def change
    enable_extension 'cube' #a dependency of the earthdistance extension 
    enable_extension 'earthdistance'
  end
end

The previous query with the extension looks like this:

def self.order_by_distance(lat, lon)
  select("restaurants.*, 
          earth_distance(ll_to_earth(#{lat}, #{lon}), 
          ll_to_earth(latitude, longitude)) 
          AS distance")
  .order('distance')
end

The ll_to_earth function, transforms latitude and longitude values to spatial coordinates, and the earth_distance function calculates the distance between these coordinates using the Haversine formula.

This formula is also implemented in many other databases, for instance Redis GEODIST.

Address to coordinates

Ok, now we know how to find the distance between coordinates, and we know how to find the user's coordinates, but we don't have the Restaurant's coordinates. How do we get them?

There are many apis that can be used to get the coordinates of an address. Most of them are freemium services. Here are some of the most popular ones:

You can test a few records yourself using Google Maps on their web page to see how it works. In my implementation, I chose Google Maps because it handles various address formats and styles more flexibly than other apis.

That's it, now we have everything we need to implement our geolocation feature, just plug these coordinates into your Restaurants and we are good to go.

References

Geolocation API.
Haversine formula
Postindustria geo queries made easy
Postgis
Redis GEODIST
Google Maps Geocoding API
OpenStreetMap

Obfuscating your create react app and routes

Petra Grunheidt — Wed, 17 Jan 2024 14:40:06 +0000

This week, I was tasked with enhancing the security of our Create React App by employing obfuscation and minification techniques.

Obfuscation, as the name implies, obscures the source code, intentionally making it complex for humans to decipher while retaining its functionality. Minification, on the flip side, compresses code, trims the fat, and optimizes size for efficient delivery.

During my intial search i came across some outdated libraries like javascript-obfuscator and uglify-js(as if javascript code can get any uglier, am I right?). Then, I stumbled upon Terser, a modern library that supports ES6.

Reading through the documentation I was able to apply a simple enough script for my package.json:

"obfuscate:" "terser build/static/js/main.*.js --compress --mangle --output main.min.js"

Which I then tinkered with using xargs for it to update the same file that was used instead of generating a new one:

"obfuscate": "find build/static/js -type f -name 'main.*.js' -print0 | xargs -0 -I {} terser {} --compress --mangle --output {}",

I also came across an article recommending setting the GENERATE_SOURCEMAP to false. A source map is a file that maps the minified or transpiled code back to its original source code, aiding in debugging. While source maps may be valuable during development for debugging purposes, they can potentially expose sensitive information about your code structure and logic. So my final script looks like this:

"build": "GENERATE_SOURCEMAP=false react-scripts build && yarn obfuscate",

Now when I open the website, the files are obfuscated and minified:

(Apologies for the light mode, i don't usually use the browser i printed this on. This is not an obfuscation technique, as it is local to my browser)

So, job well done, right?

However, a colleague conducting security checks on the application discovered that it remained remarkably easy to discern all the routes by searching for the path. This is understandable since the route names can't be effectively obfuscated, as the pathnames must be preserved for users to be able to access the routes. While this is acceptable for public routes, every file in the application is somehow present in the /build/static/js/main.*.js file, given that it's a JavaScript SPA.

The issue arose because our app contained routes meant to be 'secret' and hidden from regular users. Although these routes were protected by authentication, exposing them wasn't an ideal scenario

Chunkification

Now, here comes the really interesting part of this article. The key to obfuscating routes lies in leveraging the power of React's lazy loading and suspense mechanisms.

Lazy Loading: Lazy loading allows you to load specific parts of your application only when they are needed. This is particularly beneficial for large applications with numerous components. React's lazy function enables you to dynamically import components, ensuring that they are only fetched and rendered when required. This helps in reducing the initial loading time and improving the overall performance of your application.
Suspense: Suspense is a React feature that allows components to wait for something before rendering. It enables you to manage the loading states of your components more efficiently. With the Suspense component, you can specify fallback content to display while waiting for the main content to load. This creates a smoother and more seamless user experience, especially when dealing with asynchronous operations such as lazy loading.

By strategically employing these features, we can dynamically split and load code associated with specific routes into separate chunks within your build/static folder, and it will only be loaded when the route is accessed. This not only enhances the security of your application but also optimizes the loading of resources, providing a more efficient and seamless user experience.

When applying this technique and building the app, our Chunk will appear like this:

Now, let's go to the implementation

Implementation in Routing

In your main routing configuration (app/index.js), you can structure it as follows:

// app/index.js
<Router>
  <Routes>
    <Route path="/*" element={<IndexRoutes />} /> {/* Regular routes */}
    <Route path="/admin/*" element={<Suspense fallback={<></>}><AdminRoutes /></Suspense>} /> {/* Routes you want to hide */}
  </Routes>
</Router>

The AdminRoutes component, which contains the routes you wish to secure (routes/admin.jsx), could look something like this:

// routes/admin.jsx
<Routes>
  <Route path="/secret-clients/:slug" element={<AdministrativePage />} />
  <Route path="/super-secret-page" element={<SuperSecretPage />} />
  {/* ... */}
</Routes>

This approach allows the existence of the /admin namespace to be evident in your main.*.js file. However, the actual code related to this namespace is encapsulated in a separate chunk that won't load until the corresponding routes are accessed.

There is just one potential issue: While our admin chunk is inaccessible from regular routes, accessing host/admin/{anything} would load a page. The page would be blank, but unfortunately for us, this also triggers the loading of the associated chunk, potentially exposing information about the routes.

To add an extra layer of protection against unauthorized access to our chunk, we can implement a redirect strategy within our routes/admin.jsx file:

  // routes/admin.jsx
  <Routes>
    <Route path="/actions/:slug" element={<AdministrativePage />} />
    <Route path="/super-secret-page" element={<SuperSecretPage />} />
    {/* ... */}
    <Route path="*" element={<Navigate to="/" />} />
  </Routes>

By including <Route path="*" element={<Navigate to="/" />} />, we ensure that any attempt to access our chunk with an unknown path will result in an immediate redirect to the homepage. While this provides improved security, there's still a brief moment when the chunk is accessible before the redirect occurs, and you know hackers these days, with their black hoodies and big brains. If they can download cars, they could certainly download our chunk if they tried hard enough.

To further enhance security, we can integrate this strategy with user authentication. Returning to our app/index.js, we can modify the routing configuration:

// app/index.js
import { isAdminLoggedIn } from './dir' /* function to verify authentication*/

{...}

<Router>
  <Routes>
    <Route path="/*" element={<IndexRoutes />} /> {/* Regular routes */}
    <Route path="/auth/*" element={<IndexRoutes />} /> {/* Auth routes */}
    {isAdminLoggedIn() && <Route path="/admin/*" element={<Suspense fallback={<></>}><AdminRoutes /></Suspense>} />} {/* Routes you want to hide */}
  </Routes>
</Router>

By implementing authentication in combination to the chunk strategy, even if an unauthenticated user were to try and access the /admin namespace, it would not trigger the chunk loading, therefore, protecting our app further.

Effective email HTML and image handling

Petra Grunheidt — Mon, 04 Dec 2023 12:15:11 +0000

If you are trying to fire an email you coded with all of the beautiful assets your UX team provided, only to find an unaligned mess, without images when opening it on your email provider, this article is for you.

The reason behind this is quite simple: Modern CSS is not supported in most email providers. As a developer of the 2020s, you're likely accustomed to aligning your pages with flexbox, grid or other modern CSS features.

Now, on reading the above statement you might be thinking this is probably an issue with small email providers that nobody uses anyway so why should you care?

To our dismay, major platforms like Gmail do not support flexbox align-items, and there are no plans to include support for SVG (and don't get me started with outlook's desktop app, where even basic styles go very wrong). You can refer to an illustrative guide of what you can or can't do across email providers on Caniemail.

I first learned about these issues in a presentation by a work colleage (@mfornaciari), and was later on assigned a task to develop e-mails for an e-commerce app. This article is a way for me to summarize the main points I learned while studying for my tasks.

Basic HTML Alignment and Layout

So, if you can't use modern CSS, what is the go to way of aligning emails? the answer is, a simple html table:

  <table
    role="presentation"
    width="100%"
    border="0"
    cellpadding="0"
    cellspacing="0"
    style="min-width: 100%;"
  >
    <!-- since we are using the table tag only for styling, it is important for acessibility purposes to use a semantic role -->
    <!-- the other properties, except for the width, are style resets -->
    <tr>
      <td align="center">
        {content}
      </td>
    </tr>
  </table>

This simple centered table layout is widely supported and ensures a consistent appearance across various email clients, even those that might not interpret advanced styling techniques.

With this structure, I separated my email content into rows. Since media queries are not reliable for emails, it's best to think of them as a verticalized media that must fit web and mobile screen resolutions. For this reason, it is a common practice to limit an email's body width to 600px.

Since I was using Ruby on Rails, I added this rule to my mailer layout file along with basic text styles and style resets:

    <!DOCTYPE html>
      <html>
        <head>
          <meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
          <style>

            body, p, span, ul, li {
              font-size: 16px;
              font-family: Urbanist, sans-serif;
              color: #000;
              margin: 0;
              padding: 0;
              text-align: left;
              line-height: 1.75
            }

            h1, h2, h3 {
              font-family: Urbanist, sans-serif;
              margin: 16px 0;
              padding: 0;
              line-height: normal;
              color: #000;
            }

          </style>
        </head>

        <body>
          <table role='presentation' width='100%' border='0' cellpadding='0' cellspacing='0' style='background: grey; min-width: 100%;'>
            <tr>
              <td align='center'>
                <table role='presentation' width='100%' border='0' cellpadding='0' cellspacing='0' style='background: #fff; max-width: 600px'>
                  <tr>
                    <td align='center'>
                      <%= yield %> <!-- For those unfamiliar with html.erb, this is where the content of my specific e-mails will go -->
                    </td>
                  </tr>
                </table>
              </td>
            </tr>
          </table>
        </body>
      </html>

Now, lets create a simple welcome mail and check out the result in mailcatcher (keep in mind that in a real use scenario, you should test that your email actually worked in real providers):

      <body>

        <!-- header, this can be extracted into a shared partial -->
          <header>
            <table
              role="presentation"
              width="100%"
              border="0"
              cellpadding="0"
              cellspacing="0"
              style="min-width: 100%;"
            >
              <tr>
                <td align="center" style='background: red'>
                  <h1 style='color: yellow'>Super e-commerce</h1>
                </td>
              </tr>
            </table>
          </header>

          <!-- content -->
          <table
            role="presentation"
            width="100%"
            border="0"
            cellpadding="0"
            cellspacing="0"
            style="min-width: 100%;"
          >
            <tr>
              <td align="center">
                <h1>Welcome to this plataform, champ</h1>
                <p>Thanks for joining, you're a real superstar, we love you</p>
              </td>
            </tr>
          </table>

          <!-- footer, this can be extracted into a shared partial -->
          <footer>
            <table
              role="presentation"
              width="100%"
              border="0"
              cellpadding="0"
              cellspacing="0"
              style="min-width: 100%;"
            >
              <tr>
                <td align="center" style='background: red'>
                  <h1 style='color: yellow'>cool logo</h1>
                </td>
              </tr>
            </table>
          </footer>

      </body>

now let's fire our email, aaaand:

Your UX team might have a murderous instinc towards me upon seeing this, but that's what healthy emails are made of.

Strategies for Handling Images

Another issue I ran into was dealing with images. As I mentioned at the beginning of this article, Gmail does not support inline image methods such as SVG or base64.

Also, emails have a limited size limit; Gmail, for instance, allows only a maximum of 25MB per email. Additionally, if you send multiple emails with the same subject, for instance, a user spamming password recovery emails and receiving many stacked emails with (1), (2)..., this limit counts for all the collective emails, not for each one.

Here are the two best approaches I found for handling images

Image File attachment

When it comes to including images in your emails, one simple method is to attach image files directly to the email. This allows the recipient to download and view the images separately. Keep in mind that the total size of email attachments may be subject to limitations imposed by email service providers.

Example:

    class YourMailer < ApplicationMailer
      def send_email_with_attachment
        # Other mail settings (to, from, subject, etc.)
        mail(to: 'recipient@example.com', subject: 'Email with Attachment') do |format|
          format.html do
            # Your HTML content here
          end

          format.text do
            # Your plain text content here
          end
        end

        # Attach the image file
        attachments['image.jpg'] = File.read('path/to/image.jpg')
      end
    end

I feel like this method should be used with some consideration, since it increases the total size of your email.
Additionally, it may not be ideal to clutter your email with image attachments, considering that the final user isn't intended to download an Instagram logo, for example.

External URL Hosting

The approach I find most effective for all image sizes, is storing them in an external host such as on an S3 bucket, google drive, a github repository or imgur. This approach doesn't impact the email's full size limit, providing a more scalable solution, but the external services apis may have requests and bandwith limitations and may not be free.

Example:

    <img src="https://external-hosting.com/path/to/image.jpg" alt="External Image" />

The challenge I found with this approach is that free services for hosting are not really optimized for this use and may lead to broken images in your email.

The mobile screenshot above depicts the footer of the email I was developing for my task. It was designed to feature a company logo and three social media links. Initially, I chose to use Google Drive to store my images because it's free.

While the web version of my app consistently loaded images upon most page reloads, the mobile version experienced frequent breaks. Unfortunately, relying on a simple JavaScript reload for images when a request fails is not an available feature in email providers, so the first request on your img tag must succeed.

Consequently, I shifted to a paid solution – the S3 bucket. Although there is a cost associated with this approach, it proved to be a reliable solution for image loading. At first this solution did not seem ideal for someone like me who does not really believe in paying for stuff (or, in Brazilian Portuguese, a 'muquirana'). However, I later discovered the option to integrate the S3 bucket with a Content Delivery Network (CDN), a global network of servers that accelerates image loading times and caches results, reducing the need for frequent payments for image requests each time an email is opened. The use of a CDN can be particularly beneficial for optimizing costs and enhancing overall performance in your email image hosting strategy.

Overall I found this to be the best approach, and it is also very useful when some style your UX team wants does not work on an e-mail provider (yes, i'm looking at you outlook desktop app, which for some reason is still used by a lot of people).

Conclusion

In summary, emails present unique challenges and limitations, yet they can still exude aesthetic appeal. Now, while I made some jokes about irritating your UX team, it is imperative that you as a developer can effectively communicate these restrictions to them so that they can best create a design that is beautiful and possible to make. This collaborative approach ensures a balance between aesthetics and functionality. Explore https://mosaico.io/ for valuable, free insights into crafting impressive email designs!