DEV Community: Mike Elsmore

How to Stream Video with Just JavaScript and HTML5

Mike Elsmore — Mon, 02 Feb 2026 13:43:41 +0000

If you're old enough to remember Internet Explorer, CSS shim files, and spending way too much time on Newgrounds, first off—how are your knees these days? But secondly, you probably remember Flash Player and Silverlight as being the way you watched video in web browsers. Those were the days of browser plugins, mysterious crashes, and that wonderful "Click to enable Flash" banner that haunted every second website.

But since 2008, we've had the option to use the native <video> HTML5 element to handle video playback and streaming. No more plugins. No more Adobe update prompts. And here's the thing that might surprise you: we don't even need a modern framework to do this.

The requirements to play with the <video> element are as simple as any web browser on a modern operating system, and an ImageKit account to host your video files. We'll also cover some fun upgrades and extras you can do later on with JavaScript and ImageKit's video API.

See it in action

If you want to skip ahead and see the finished result, I've got you covered:

Live demo: https://elsmore.me/imagekit-javascript-video-streaming/
Source code: https://github.com/ukmadlz/imagekit-astro-video-streaming

Feel free to poke around the code, fork it, break it, and make it your own. Now let's walk through how it all works.

Using the HTML5 <video> element
Adding playback controls
Defining multiple video sources
Displaying a video poster (thumbnail)
Auto-playing videos the right way
Building a custom video player with JavaScript
Adaptive Bitrate Streaming (ABS)
Using Video.js for advanced streaming
Optimising video delivery with ImageKit
Conclusion

Using the HTML5 `<video>` element

To start displaying a video on any given webpage is as easy as this:

<!DOCTYPE html>
<html lang="en">
  <head>
    <meta charset="UTF-8" />
    <meta name="viewport" content="width=device-width, initial-scale=1.0" />
    <title>An Example HTML5 Video</title>
  </head>
  <body>
    <video
      src="https://ik.imagekit.io/ikmedia/example_video.mp4"
      width="640"
      height="360"
    />
  </body>
</html>

That's right—just like an <img> tag takes a source location to display an image, the only thing the <video> tag strictly needs is a file passed into the src attribute. And just like an <img>, it's best to assign both a width and height to the <video> element to avoid layout shifts as the video loads.

If you're thinking "wait, that's it?"—yes, that's genuinely it for the bare minimum. But of course, we're going to make it better.

Adding playback controls

That bare-bones example above will display a video, but your users won't be able to do much with it. They can't play, pause, adjust volume, or seek through the timeline. Let's fix that:

<video
  src="https://ik.imagekit.io/ikmedia/example_video.mp4"
  width="640"
  height="360"
  controls
/>

The controls attribute adds the browser's default video controls. This includes play/pause, a progress bar, volume control, and fullscreen toggle. Each browser renders these slightly differently (Firefox, Chrome, and Safari all have their own flavour), but they're functional across the board.

Here are some other useful attributes you'll want to know about:

Attribute	What it does
`controls`	Shows play, pause, volume, progress bar
`autoplay`	Starts playing automatically (with caveats)
`muted`	Starts the video muted
`loop`	Loops the video when it ends
`playsinline`	Plays inline on iOS instead of fullscreen
`preload`	Hints how much to preload (`none`, `metadata`, `auto`)
`poster`	Shows an image before the video plays

Defining multiple video sources

Not all browsers support all video formats. While MP4 (H.264) is nearly universal these days, WebM can offer better compression, and you might have legacy content in other formats. The <source> element lets you provide fallbacks:

<video width="640" height="360" controls>
  <source src="https://ik.imagekit.io/ikmedia/example_video.webm" type="video/webm">
  <source src="https://ik.imagekit.io/ikmedia/example_video.mp4" type="video/mp4">
  Your browser does not support the video tag.
</video>

The browser reads top to bottom and picks the first format it supports. So if you want users to get WebM (smaller files, same quality), put that first. MP4 acts as the fallback.

The text between the tags ("Your browser does not support...") only displays if the browser doesn't support the <video> element at all. This is increasingly rare, but it's good practice to include it.

Pro tip: With ImageKit, you can transform video formats on the fly using URL parameters. Instead of hosting multiple files, use a single source and let ImageKit handle the conversion:

<video width="640" height="360" controls>
  <source src="https://ik.imagekit.io/ikmedia/example_video.mp4?tr=f-webm" type="video/webm">
  <source src="https://ik.imagekit.io/ikmedia/example_video.mp4?tr=f-mp4" type="video/mp4">
</video>

Displaying a video poster (thumbnail)

Before your video loads, you'll want to show something other than a black rectangle. The poster attribute lets you display an image:

<video
  src="https://ik.imagekit.io/ikmedia/example_video.mp4"
  width="640"
  height="360"
  controls
  poster="https://ik.imagekit.io/ikmedia/example_video.mp4/ik-thumbnail.jpg"
/>

ImageKit has a neat trick here: you can generate thumbnails automatically from your video using the /ik-thumbnail.jpg suffix. Want a thumbnail from a specific timestamp? Add a transformation parameter:

https://ik.imagekit.io/ikmedia/example_video.mp4/ik-thumbnail.jpg?tr=so-5

This grabs a frame from 5 seconds into the video. You can also resize it:

https://ik.imagekit.io/ikmedia/example_video.mp4/ik-thumbnail.jpg?tr=so-5,w-640,h-360

No need to manually create and host poster images—let the API do the work.

Auto-playing videos the right way

Autoplay is a minefield. Browsers have cracked down hard on autoplaying videos because nobody likes visiting a site and having sound blast at them unexpectedly. Here's how to do it properly:

<video
  src="https://ik.imagekit.io/ikmedia/example_video.mp4"
  width="640"
  height="360"
  autoplay
  muted
  playsinline
/>

The key points:

autoplay alone won't work in most browsers - they require the video to also be muted
playsinline is essential for iOS - without it, Safari on mobile opens videos in fullscreen mode
Consider your users - autoplaying videos (even muted) can be jarring

This pattern is perfect for hero videos, background loops, or product demos where you don't need audio. If you do need audio, you'll need to wait for user interaction first (like a click) before calling video.play() in JavaScript.

Building a custom video player with JavaScript

The native controls are fine, but sometimes you want something custom. The HTML5 video element exposes a comprehensive JavaScript API for building your own player. Here's a basic example:

<!DOCTYPE html>
<html lang="en">
<head>
  <meta charset="UTF-8">
  <meta name="viewport" content="width=device-width, initial-scale=1.0">
  <title>Custom Video Player</title>
  <style>
    .video-container {
      max-width: 640px;
      margin: 0 auto;
    }
    video {
      width: 100%;
      display: block;
    }
    .controls {
      display: flex;
      gap: 10px;
      padding: 10px;
      background: #222;
      align-items: center;
    }
    .controls button {
      padding: 8px 16px;
      cursor: pointer;
    }
    .progress-bar {
      flex-grow: 1;
      height: 8px;
      background: #555;
      cursor: pointer;
      border-radius: 4px;
    }
    .progress {
      height: 100%;
      background: #4CAF50;
      width: 0%;
      border-radius: 4px;
    }
    .time {
      color: white;
      font-family: monospace;
    }
  </style>
</head>
<body>
  <div class="video-container">
    <video id="video" src="https://ik.imagekit.io/ikmedia/example_video.mp4"></video>
    <div class="controls">
      <button id="playPause">Play</button>
      <div class="progress-bar" id="progressBar">
        <div class="progress" id="progress"></div>
      </div>
      <span class="time" id="time">0:00 / 0:00</span>
    </div>
  </div>

  <script>
    const video = document.getElementById('video');
    const playPauseBtn = document.getElementById('playPause');
    const progressBar = document.getElementById('progressBar');
    const progress = document.getElementById('progress');
    const timeDisplay = document.getElementById('time');

    // Play/Pause toggle
    playPauseBtn.addEventListener('click', () => {
      if (video.paused) {
        video.play();
        playPauseBtn.textContent = 'Pause';
      } else {
        video.pause();
        playPauseBtn.textContent = 'Play';
      }
    });

    // Update progress bar
    video.addEventListener('timeupdate', () => {
      const percentage = (video.currentTime / video.duration) * 100;
      progress.style.width = percentage + '%';
      timeDisplay.textContent = formatTime(video.currentTime) + ' / ' + formatTime(video.duration);
    });

    // Click to seek
    progressBar.addEventListener('click', (e) => {
      const rect = progressBar.getBoundingClientRect();
      const percentage = (e.clientX - rect.left) / rect.width;
      video.currentTime = percentage * video.duration;
    });

    // Format seconds to M:SS
    function formatTime(seconds) {
      if (isNaN(seconds)) return '0:00';
      const mins = Math.floor(seconds / 60);
      const secs = Math.floor(seconds % 60).toString().padStart(2, '0');
      return mins + ':' + secs;
    }
  </script>
</body>
</html>

The video element gives you access to a wealth of properties and methods:

Properties:

video.currentTime - current playback position in seconds
video.duration - total duration in seconds
video.paused - boolean indicating if video is paused
video.volume - volume level from 0.0 to 1.0
video.playbackRate - playback speed (1.0 is normal)

Methods:

video.play() - start playback (returns a Promise)
video.pause() - pause playback
video.load() - reload the video source

Events:

play - fired when playback starts
pause - fired when playback pauses
timeupdate - fired as playback progresses
ended - fired when video reaches the end
loadedmetadata - fired when duration/dimensions are available

This gives you everything you need to build players that match your brand, add custom features, or integrate with your application's state management.

Adaptive Bitrate Streaming (ABS)

Here's where things get interesting. So far, we've been dealing with single video files—one quality level, one file size, one experience for everyone. But what happens when someone's on a dodgy coffee shop WiFi versus a fibre connection at home?

Adaptive Bitrate Streaming (ABS) solves this by encoding your video at multiple quality levels and letting the player switch between them based on network conditions. The video starts quickly at lower quality, then upgrades as bandwidth allows. If the connection drops, it downgrades gracefully instead of buffering endlessly.

There are two main protocols:

HLS (HTTP Live Streaming) - Apple's format, uses .m3u8 playlist files
MPEG-DASH (Dynamic Adaptive Streaming over HTTP) - Uses .mpd manifest files

The catch? The native <video> element doesn't support these formats directly in most browsers. Safari supports HLS natively (it's Apple's thing, after all), but Chrome and Firefox don't. That's where JavaScript libraries come in.

Using Video.js for advanced streaming

Video.js is an open-source HTML5 video player that handles ABS protocols, provides a consistent UI across browsers, and offers a plugin ecosystem for extended functionality.

First, include Video.js in your project:

<!DOCTYPE html>
<html lang="en">
<head>
  <meta charset="UTF-8">
  <meta name="viewport" content="width=device-width, initial-scale=1.0">
  <title>Video.js Streaming Example</title>
  <link href="https://vjs.zencdn.net/8.10.0/video-js.css" rel="stylesheet">
</head>
<body>
  <video
    id="my-video"
    class="video-js vjs-default-skin"
    controls
    width="640"
    height="360"
  >
    <source src="https://ik.imagekit.io/demo/sample-video.mp4/ik-master.mpd?tr=sr-240_360_480_720_1080" type="application/dash+xml">
  </video>

  <script src="https://vjs.zencdn.net/8.10.0/video.min.js"></script>
  <script>
    const player = videojs('my-video', {
      fluid: true, // Responsive sizing
      playbackRates: [0.5, 1, 1.5, 2], // Speed options
    });
  </script>
</body>
</html>

For HLS streams, just swap the source:

<source src="https://ik.imagekit.io/demo/sample-video.mp4/ik-master.m3u8?tr=sr-240_360_480_720_1080" type="application/x-mpegURL">

Video.js (version 7+) handles both DASH and HLS out of the box, no additional plugins required. The player will automatically select the appropriate quality based on the viewer's bandwidth.

Quality selection

Video.js also lets users manually select quality levels if they prefer. You can add this functionality with the quality selector plugin, or let the player handle it automatically—most users won't need to touch it.

Optimising video delivery with ImageKit

I've been dropping ImageKit URLs throughout this post, so let's properly cover what it can do for your video streaming setup.

Resize videos to appropriate dimensions

Why send a 4K video to someone watching on a 400px container? ImageKit lets you resize on the fly:

https://ik.imagekit.io/ikmedia/example_video.mp4?tr=w-640,h-360

This doesn't just scale the player—it actually delivers a smaller file. A video at the original 4K resolution might be 20MB, while the same video resized to 640x360 could be under 2MB. Same visual quality for your use case, massively reduced bandwidth.

Automatic format selection

Instead of manually specifying WebM and MP4 sources, let ImageKit figure out what the browser supports:

<video src="https://ik.imagekit.io/ikmedia/example_video.mp4?tr=f-auto" controls></video>

The f-auto transformation detects the viewer's browser and delivers the most optimised format automatically.

Generate ABS playlists

Creating DASH and HLS playlists normally requires encoding your video at multiple bitrates, generating segment files, and creating manifest files. ImageKit does all of this from a single source video:

DASH:

https://ik.imagekit.io/ikmedia/example_video.mp4/ik-master.mpd?tr=sr-240_360_480_720_1080

HLS:

https://ik.imagekit.io/ikmedia/example_video.mp4/ik-master.m3u8?tr=sr-240_360_480_720_1080

The sr-240_360_480_720_1080 parameter specifies which quality levels to include. ImageKit handles the encoding and playlist generation on demand.

Add overlays

Need to watermark your videos or add captions? ImageKit supports text and image overlays:

https://ik.imagekit.io/ikmedia/example_video.mp4?tr=l-text,i-My%20Company,fs-40,co-white,bg-00000080,pa-20,l-end

This adds "My Company" as a text overlay with a semi-transparent black background.

Conclusion

Video streaming with JavaScript and HTML5 has come a long way since the Flash days. The native <video> element handles most common use cases without any external dependencies. When you need adaptive streaming for varying network conditions, Video.js provides a solid foundation that works across browsers.

The real power comes from combining these client-side capabilities with a smart CDN like ImageKit. Instead of pre-encoding dozens of video variants, you can transform and deliver optimised video on the fly. Format conversion, resizing, thumbnail generation, and adaptive streaming playlists all happen automatically from a single source file.

Whether you're building a simple product demo or a full video streaming platform, the tools are all there in the browser—we just need to use them.

Choosing Your Documentation Tooling: A Practical Guide

Mike Elsmore — Tue, 27 Jan 2026 16:47:56 +0000

In our previous post on Developer Experience, we touched on documentation being one of the core pillars of great DX. We mentioned treating your docs as a product rather than a necessary evil, and pointed towards the docs-as-code approach. But we didn't dive into the specifics of what to actually build your documentation with. So let's fix that.

The Landscape

There's no shortage of documentation tools out there, and honestly, that can make the decision harder rather than easier. After working with various clients and our own projects here at Digital Speed, we've found ourselves reaching for a handful of tools repeatedly: Docusaurus, VuePress, Redocly, and Fumadocs.

Each has its strengths, and sometimes the best solution isn't picking one – it's knowing how they can work together.

Docusaurus

If you've spent any time in the open-source world, you've probably read documentation built with Docusaurus. Meta's documentation framework has become something of a default choice, and for good reason.

Docusaurus is React-based, which means if your team already lives in that ecosystem, the learning curve is minimal. It handles versioning out of the box (a genuine lifesaver when you're maintaining multiple API versions), has solid search via Algolia integration, and the plugin ecosystem is mature enough that most common needs are covered.

Several members of the team here at Digital Speed have used Docusaurus extensively, and what keeps us coming back is the balance it strikes between flexibility and convention. You can get something professional-looking deployed in an afternoon, but you're not boxed in when requirements get more complex down the line.

The main consideration? It's opinionated about structure, which is actually a feature if you're starting fresh but can be friction if you're migrating existing docs with unusual organisation.

VuePress

For teams in the Vue ecosystem, VuePress is the natural counterpart to Docusaurus. It's lightweight, fast, and the Vue-powered theming system makes customisation straightforward if you know your way around Vue components.

VuePress 2 (the current version) has matured significantly and offers a clean developer experience. The markdown-centred approach means your content stays portable, and the plugin architecture is sensible without being overwhelming.

We've seen VuePress work particularly well for smaller documentation sites or teams who want something less heavyweight than Docusaurus. If you're documenting a Vue-based library or your team simply prefers Vue's patterns, it's worth serious consideration.

Redocly

Now we're getting into more specialised territory. Redocly is laser-focused on API documentation, specifically OpenAPI-based docs. If you're building products with REST APIs (and let's be honest, most of us are), Redocly deserves a look.

The open-source Redoc renderer produces three-panel API documentation from your OpenAPI spec. It's the kind of documentation that makes developers actually want to integrate with your API. We've used the open-source version at Digital Speed and found it does exactly what it promises with minimal fuss. The only issue with the open-source version is you're pretty much locked in to the limited out-of-the-box styling and theming options, so you can't expect to customise the design much.

Beyond the renderer, Redocly offers a CLI for linting your OpenAPI specs and a broader platform for teams who need more – but the open-source tools alone cover a lot of ground.

The key thing to understand about Redocly is that it's not trying to be a general-purpose documentation platform. It's purpose-built for API reference documentation, and it excels at that specific job.

Fumadocs

Fumadocs is the newer entrant here, built specifically for the Next.js ecosystem. If you're already running Next.js (and plenty of teams are these days), Fumadocs integrates naturally into your existing setup rather than being a separate concern.

We've recently started using Fumadocs at Digital Speed and have been impressed by how it handles the basics. The search works well, the theming is highly customisable and clean, and it doesn't fight against Next.js conventions – it embraces them.

It's worth noting that Fumadocs is younger than the other options, which means the ecosystem is smaller and you might occasionally hit edges that haven't been smoothed yet. Additionally, the documentation is directly tied to Next.js versions and isn't versioned itself, so you might not be able to find documentation for older versions of Next.js. But for Next.js teams, it's a compelling option that's maturing quickly.

Using Tools in Concert

Here's where it gets interesting. These tools don't have to be mutually exclusive.

A pattern we've seen work well is using Docusaurus (or Fumadocs, depending on your stack) as the primary documentation platform for guides, tutorials, and conceptual content – then embedding Redocly-generated API reference documentation within it.

This gives you the best of both worlds: a flexible, customisable platform for your narrative documentation, paired with purpose-built tooling for your API reference. Your developers get a cohesive experience, but you're using the right tool for each job.

The integration is typically straightforward. Redocly can generate static HTML that you host alongside your main docs, or you can embed the Redoc React component directly if you're in a React-based framework.

Making the Choice

So how do you actually decide? A few questions worth asking:

What's your team's existing stack? If you're React-heavy, Docusaurus or Fumadocs (for Next.js specifically) will feel natural. Vue teams should look at VuePress.

Do you have OpenAPI specs to document? Strongly consider Redocly for that portion of your docs, even if you use something else for the rest.

How much customisation do you need? All of these tools are customisable, but they sit on a spectrum from "works great with defaults" to "expects you to build on top." Know where your needs fall.

What's your team's capacity? A smaller team might prefer something lightweight like VuePress or Fumadocs over Docusaurus's broader feature set.

Conclusion

There's no single "best" documentation tool – there's only the best tool for your specific situation. What matters more than the specific choice is that you make a choice and invest in your documentation as a proper product.

We'll be covering more specifics around SDK development and API documentation in upcoming posts. In the meantime, if you're struggling with your documentation strategy or need help implementing any of these tools, we at Digital Speed are happy to chat.

After all, great documentation is just another way of helping developers get things done.

How to Add Video to Your Astro Site with ImageKit

Mike Elsmore — Thu, 22 Jan 2026 16:33:43 +0000

If you've been anywhere near web development Twitter (or X, or Bluesky, or whatever we're calling it this week) in the last few days, you've probably seen the news: Cloudflare has acquired The Astro Technology Company. The team behind one of the most interesting web frameworks to emerge in recent years is now part of the connectivity cloud giant.

For those of us who've been building with Astro, this is genuinely exciting news. Not "exciting" in the Silicon Valley way where a company gets acqui-hired and the product slowly dies, but actual excitement. Cloudflare has committed to keeping Astro open source, MIT-licensed, and platform-agnostic. Fred Schott, Astro's CEO, wrote that joining Cloudflare means the team can finally stop worrying about building a business on top of Astro and focus 100% on the framework itself.

The timing is particularly interesting because Astro 6 just hit beta, bringing a redesigned development server, better runtime support, and improved performance. With Cloudflare's resources behind it, Astro is positioned to become the definitive framework for content-driven websites—and content-driven websites almost always need video.

So let's talk about how to implement video in Astro, specifically using ImageKit's video API to handle the heavy lifting of transcoding, streaming, and optimisation.

See it in action

If you'd rather jump straight to the code or see the finished result:

Live demo: https://elsmore.me/imagekit-astro-video-streaming/
Source code: https://github.com/ukmadlz/imagekit-astro-video-streaming

Clone it, deploy it, tear it apart—it's all there for you to play with. Now let's dig into the details.

Why Astro for video content?

Astro was born in 2021 out of frustration with a web that had become obsessed with shipping JavaScript to the browser for everything. The prevailing wisdom was that every website should be architected as an application—rendered client-side, hydrated, and heavy with JavaScript bundles.

Astro took a different approach: ship HTML by default, JavaScript only when necessary.

This philosophy—what Astro calls "Islands Architecture"—is perfect for video-heavy content sites. Your video player can be an interactive island while the rest of your page (navigation, text content, footer) remains static HTML. The result? Faster initial page loads, better Core Web Vitals scores, and happier search engines.

Sites like Porsche, IKEA, and OpenAI use Astro. Platforms like Webflow and Wix have built on it. And now with Cloudflare's backing, performance-focused content sites have a framework with serious long-term support.

Setting up an Astro project

Let's start from scratch. If you've got Node.js 18.17.0 or later installed, creating a new Astro project is straightforward:

npm create astro@latest my-video-site

The CLI will ask you a few questions. For this tutorial, I'd recommend:

Template: Empty
TypeScript: Yes (strict)
Dependencies: Yes

Navigate into your project and start the development server:

cd my-video-site
npm run dev

Your site is now running at http://localhost:4321/. Yes, 4321—Astro counts down rather than up. It's a small thing, but I appreciate the personality.

Basic video with HTML5

The simplest way to add video to an Astro page is exactly the same as any HTML page—the <video> element just works:

---
// src/pages/index.astro
---
<html lang="en">
  <head>
    <meta charset="UTF-8" />
    <meta name="viewport" content="width=device-width, initial-scale=1.0" />
    <title>Video Demo</title>
  </head>
  <body>
    <h1>Welcome to my video site</h1>
    <video
      src="https://ik.imagekit.io/ikmedia/example_video.mp4"
      width="640"
      height="360"
      controls
    >
      Your browser does not support the video tag.
    </video>
  </body>
</html>

That's it. No JavaScript shipped to the client. No framework overhead. Just HTML and a video file.

Want multiple format sources for browser compatibility? Same as regular HTML:

<video width="640" height="360" controls>
  <source src="https://ik.imagekit.io/ikmedia/example_video.mp4?tr=f-webm" type="video/webm">
  <source src="https://ik.imagekit.io/ikmedia/example_video.mp4?tr=f-mp4" type="video/mp4">
  Your browser does not support the video tag.
</video>

Notice those URL parameters? That's ImageKit transforming the video format on the fly. One source file, multiple formats, no manual transcoding required.

Creating a reusable Video component

In Astro, components are .astro files with a frontmatter section (between the --- fences) and a template section. Let's create a reusable Video component:

---
// src/components/Video.astro
interface Props {
  src: string;
  width?: number;
  height?: number;
  poster?: string;
  controls?: boolean;
  autoplay?: boolean;
  muted?: boolean;
  loop?: boolean;
  class?: string;
}

const {
  src,
  width = 640,
  height = 360,
  poster,
  controls = true,
  autoplay = false,
  muted = false,
  loop = false,
  class: className,
} = Astro.props;
---

<video
  src={src}
  width={width}
  height={height}
  poster={poster}
  controls={controls}
  autoplay={autoplay}
  muted={muted}
  loop={loop}
  playsinline
  class={className}
>
  <slot>Your browser does not support the video tag.</slot>
</video>

<style>
  video {
    max-width: 100%;
    height: auto;
  }
</style>

Now you can use it anywhere in your site:

---
// src/pages/index.astro
import Video from '../components/Video.astro';
---
<html lang="en">
  <head>
    <meta charset="UTF-8" />
    <meta name="viewport" content="width=device-width, initial-scale=1.0" />
    <title>Video Demo</title>
  </head>
  <body>
    <h1>Product Demo</h1>
    <Video
      src="https://ik.imagekit.io/ikmedia/example_video.mp4"
      poster="https://ik.imagekit.io/ikmedia/example_video.mp4/ik-thumbnail.jpg?tr=so-3"
      width={800}
      height={450}
    />
  </body>
</html>

The component is pre-rendered at build time. Zero JavaScript sent to the browser. The video element works natively.

Adding Video.js for advanced playback

Native HTML5 video covers most use cases, but sometimes you need more. Custom controls, adaptive bitrate streaming, analytics integration—this is where Video.js comes in.

The catch with Astro is that Video.js needs to run in the browser. This is where Astro's Islands Architecture shines—we can add interactivity only where we need it.

First, install Video.js:

npm install video.js

Now create a Video.js component. Since Video.js needs client-side JavaScript, we'll use a framework integration. Astro supports React, Vue, Svelte, Solid, and more. Let's use vanilla JavaScript with Astro's <script> tag:

---
// src/components/VideoPlayer.astro
interface Props {
  src: string;
  poster?: string;
  width?: number;
  height?: number;
}

const { src, poster, width = 640, height = 360 } = Astro.props;
const playerId = `video-${Math.random().toString(36).substr(2, 9)}`;
---

<div class="video-player-wrapper">
  <video
    id={playerId}
    class="video-js vjs-default-skin"
    controls
    preload="auto"
    width={width}
    height={height}
    poster={poster}
    data-src={src}
  >
    <source src={src} type="video/mp4" />
  </video>
</div>

<script>
  import videojs from 'video.js';
  import 'video.js/dist/video-js.css';

  // Initialize all video players on the page
  document.querySelectorAll('.video-js').forEach((element) => {
    if (element instanceof HTMLVideoElement && !element.hasAttribute('data-vjs-player')) {
      const player = videojs(element, {
        fluid: true,
        playbackRates: [0.5, 1, 1.5, 2],
      });

      // Cleanup on navigation (for Astro view transitions)
      document.addEventListener('astro:before-swap', () => {
        player.dispose();
      });
    }
  });
</script>

<style>
  .video-player-wrapper {
    max-width: 100%;
  }
</style>

The <script> tag in Astro components runs on the client. By importing Video.js here, Astro bundles it and ships it only when this component is used.

Adaptive Bitrate Streaming in Astro

For video-heavy sites, Adaptive Bitrate Streaming (ABS) is essential. It delivers different quality levels based on the viewer's network conditions—high quality on fast connections, lower quality (but still watchable) on slow ones.

Video.js handles both HLS and DASH protocols out of the box. Here's how to set it up with ImageKit's automatically generated streaming playlists:

---
// src/components/StreamingPlayer.astro
interface Props {
  src: string;
  poster?: string;
  title?: string;
}

const { src, poster, title = 'Video' } = Astro.props;

// Generate HLS and DASH URLs from the source
const hlsUrl = `${src}/ik-master.m3u8?tr=sr-240_360_480_720_1080`;
const dashUrl = `${src}/ik-master.mpd?tr=sr-240_360_480_720_1080`;
---

<div class="streaming-player">
  <video
    id="streaming-video"
    class="video-js vjs-default-skin vjs-big-play-centered"
    controls
    preload="auto"
    poster={poster}
    data-hls={hlsUrl}
    data-dash={dashUrl}
  >
    <!-- HLS source (Safari native, others via Video.js) -->
    <source src={hlsUrl} type="application/x-mpegURL" />
    <!-- Fallback MP4 for older browsers -->
    <source src={`${src}?tr=f-mp4`} type="video/mp4" />
  </video>
</div>

<script>
  import videojs from 'video.js';
  import 'video.js/dist/video-js.css';

  const videoElement = document.getElementById('streaming-video') as HTMLVideoElement;

  if (videoElement) {
    const player = videojs(videoElement, {
      fluid: true,
      html5: {
        vhs: {
          overrideNative: true, // Use Video.js HLS instead of native
        },
      },
    });

    // Quality selector (if you add the plugin)
    player.on('loadedmetadata', () => {
      console.log('Stream loaded:', player.currentSource());
    });

    document.addEventListener('astro:before-swap', () => {
      player.dispose();
    });
  }
</script>

<style>
  .streaming-player {
    max-width: 100%;
    margin: 0 auto;
  }

  .video-js {
    width: 100%;
    aspect-ratio: 16 / 9;
  }
</style>

The magic here is in the ImageKit URLs. The /ik-master.m3u8 suffix tells ImageKit to generate an HLS playlist, while tr=sr-240_360_480_720_1080 specifies which quality levels to include. ImageKit handles all the transcoding and segment generation automatically.

Use it like this:

---
import StreamingPlayer from '../components/StreamingPlayer.astro';
---
<StreamingPlayer
  src="https://ik.imagekit.io/demo/sample-video.mp4"
  poster="https://ik.imagekit.io/demo/sample-video.mp4/ik-thumbnail.jpg"
  title="Product Demo"
/>

Optimising video with ImageKit

ImageKit's video API does more than just serve files. Here are the transformations that make the biggest difference for Astro sites:

Resize to match your layout

Don't send 4K video to a 640px container:

https://ik.imagekit.io/ikmedia/example_video.mp4?tr=w-640,h-360

This transformation happens on ImageKit's servers. Your users download a smaller file, your bandwidth bills go down, and videos start playing faster.

Automatic format selection

Let ImageKit figure out the best format for each browser:

<Video src="https://ik.imagekit.io/ikmedia/example_video.mp4?tr=f-auto" />

WebM for browsers that support it (better compression), MP4 for everything else.

Generate thumbnails from the video

No need to create and host separate poster images:

https://ik.imagekit.io/ikmedia/example_video.mp4/ik-thumbnail.jpg?tr=so-5,w-640,h-360

This grabs a frame from 5 seconds into the video, resized to 640x360. Perfect for poster images and video galleries.

Add watermarks and overlays

Protect your content or add branding:

https://ik.imagekit.io/ikmedia/example_video.mp4?tr=l-text,i-My%20Brand,fs-32,co-white,l-end

Lazy loading and performance

For pages with multiple videos, you don't want them all loading at once. Here's a lazy-loading video component:

---
// src/components/LazyVideo.astro
interface Props {
  src: string;
  poster?: string;
  width?: number;
  height?: number;
}

const { src, poster, width = 640, height = 360 } = Astro.props;

// Generate a low-quality poster if none provided
const posterUrl = poster || `${src}/ik-thumbnail.jpg?tr=w-${width},h-${height},q-50`;
---

<div class="lazy-video" data-src={src} style={`aspect-ratio: ${width}/${height};`}>
  <img
    src={posterUrl}
    alt="Video thumbnail"
    loading="lazy"
    width={width}
    height={height}
  />
  <button class="play-button" aria-label="Play video">
    <svg viewBox="0 0 24 24" fill="currentColor" width="48" height="48">
      <path d="M8 5v14l11-7z"/>
    </svg>
  </button>
</div>

<script>
  document.querySelectorAll('.lazy-video').forEach((container) => {
    const button = container.querySelector('.play-button');
    const img = container.querySelector('img');

    button?.addEventListener('click', () => {
      const src = container.getAttribute('data-src');
      if (!src) return;

      const video = document.createElement('video');
      video.src = src;
      video.controls = true;
      video.autoplay = true;
      video.style.width = '100%';
      video.style.height = '100%';

      container.innerHTML = '';
      container.appendChild(video);
    });
  });
</script>

<style>
  .lazy-video {
    position: relative;
    background: #000;
    cursor: pointer;
  }

  .lazy-video img {
    width: 100%;
    height: 100%;
    object-fit: cover;
  }

  .play-button {
    position: absolute;
    top: 50%;
    left: 50%;
    transform: translate(-50%, -50%);
    background: rgba(0, 0, 0, 0.7);
    border: none;
    border-radius: 50%;
    padding: 16px;
    color: white;
    cursor: pointer;
    transition: background 0.2s;
  }

  .play-button:hover {
    background: rgba(0, 0, 0, 0.9);
  }
</style>

This shows a thumbnail and play button. The actual video only loads when clicked—no wasted bandwidth on videos users never watch.

Persisting video across page transitions

Astro 3+ introduced View Transitions, which enable smooth animations between pages. One neat feature: you can persist elements across navigations.

If you have a video playing and the user navigates to another page with the same video, you can keep it playing:

<video
  src="https://ik.imagekit.io/ikmedia/example_video.mp4"
  controls
  transition:persist
>
</video>

The transition:persist directive tells Astro to keep this element instead of replacing it during navigation. The video continues playing without interruption.

This is fantastic for sites with persistent media players—podcasts, background music, or video courses where you don't want playback to stop as users navigate.

Conclusion

Astro's philosophy of shipping minimal JavaScript makes it an excellent choice for video content sites. The framework gives you the performance benefits of static HTML while allowing interactive islands—like video players—exactly where you need them.

With Cloudflare's acquisition, Astro's future looks more stable than ever. The team can focus entirely on the framework without the distraction of building a business model. For those of us building content-driven sites with video, that's genuinely good news.

Combine Astro's architecture with ImageKit's video API, and you get a powerful setup: automatic format optimisation, on-the-fly transcoding, adaptive bitrate streaming, and thumbnail generation—all without maintaining your own video infrastructure.

The web doesn't have to be slow. It doesn't have to ship megabytes of JavaScript for every page load. And video doesn't have to be complicated.

The Differentiator Between B2B products: The Developer Experience

Mike Elsmore — Wed, 21 Jan 2026 12:00:00 +0000

Here at Digital Speed we are active users of AI within our day to day, a lot of us enjoy using Claude Code as part of our tool chain and we even use Spec Kit within some projects, and we can see how it has improved our pace and work.

But the velocity created by the tools we have to hand mean that everything is starting to feel a bit samey when integrating products together. So that’s the biggest separator between the products we’ve used recently? The Developer Experience, otherwise known as DX or DevX.

What is Developer Experience?

Most people see DX and think UX, which is close but not quite right. The dictionary definition of Developer Experience from Wikipedia is:

"Developer experience (DX) is a user experience from a developer's point of view. It is defined by the tools, processes, and software that a developer uses when interacting with a product or system while in the process of production of another one, such as in software development.[23] DX has had increased attention paid to it especially in businesses who primarily offer software as a service to other businesses where ease of use is a key differentiator in the market."
Some people on Wikipedia - https://en.wikipedia.org/wiki/User_experience#Developer_experience

A much nice definition is this quote:

"DevX is about creating frisson with your product. It's about creating thrill, feelings of excitement, and enabling and empowering developers to believe, 'Hey, I can do that.'"
Salma Alam-Naylor, @whitep4nth3r

If you want to watch a video covering the subject, our very own Mike Elsmore gave a talk at WeAreDevelopers. But what are the core things to consider when thinking about your product's Developer Experience?

Documentation
Tooling
Support

We’ll be covering some of these specific topics in follow-up posts, but for now here is a quick overview of the things to consider.

Documentation

When moving at speed, most organisations don’t have time to keep documentation up-to-date. The biggest advice that can be given here is treating your documentation as a product, rather than as a necessary evil.

If you are a tech first organisation, then one of the best things that can be done is a shift to docs-as-code. An amazing article on this topic written by Joanna Suau covers all this in greater detail: how to adopt a docs-as-code approach for developer documentation.

As for what should be used for documentation, or what should be covered, this is very much dependant on the product. And we will cover this more in depth in a follow-up article. However, just as something to think about in the mean time if you are providing technology with an API interface that requires a complete reference and should follow a standard like OpenAPI Specification. On top of this a glossary of all possible errors from the API or tool should be available alongside it.

And the last things on documentation, make sure it’s written from the perspective of someone that has no knowledge or prior skills.

Tooling

When it comes to how developers interact with your product, it can differ widely. But the common consensus is to provide a faster and smoother way to integrate your product with whatever they’re building in whatever language or system they are building it within.

The most direct way, after documenting the basics for any interaction, is through an SDK. SDKs can consist of things such as API wrapper clients, to full CLI tooling, but they all contain the pieces to simplify integrating things such as authentication/authorization. We’ll cover more of the specifics of how to build out SDKs (primarily Client Wrappers) in a follow up post, but for now I’ll present some of the best examples.

The Fly.io CLI (otherwise known as flyctl), is a great example of a command line interface that would allow someone to interact with the product directly or even script out into CI/CD flows.

A great example of a full-service SDK has to be the AWS SDK (in any language they build for). Considering how vast the product array is across the AWS portfolio, it’s consistent and easy to navigate with a lot of documentation and all sorts of helper functionality on top of the APIs.

And now in the age of AI, it’s worth considering MCP servers and the likes, just as a means to allow developers to build agents to interact with your product. Or at the very least, for AI tooling to have a better understanding of the product when developers are doing research or solving a problem.

Support

Unfortunately, this one comes very much down to business and financial budgets. Support tooling and time to manage supporting your end users is a large part of great developer experience. No matter how much documentation you have, or tooling you provide, developers of different skill levels will have issues or edge cases to contend with.

The only suggestions here are, if the product and budgets are small (or non-existent) then allow and enable the community of developers to support and help each other. Provide a place like Slack or Discord for developers to find other developers that may know what they don’t, and at the very least provide a searchable location for Q&A such as discourse or GitHub Wikis.

Conclusion

Developer Experience is more than just these three things, as it can include the onboarding process or even the marketing material that developers consume. But they’re less focused on the problem of solving how developers use the product day-to-day. Another quick quote about what DX is and why it’s great:

"Helping developers get sh❤️t done."
Salma Alam-Naylor, @whitep4nth3r

If you or your company need help with Developer Experience, we at Digital Speed are more than happy to discuss your needs and see how we can help. And if we can’t help you directly, we can recommend organisations to work with that could help.

Dapr in the cloud with Catalyst public beta

Mike Elsmore — Fri, 27 Sep 2024 18:37:11 +0000

I've been playing with this thing recently called Dapr (you can blame @marcduiker for me finding out about the project).

For those wondering what the heck Dapr is, you'd be right it does sound like it's dapper and means you need a fancy hat.

But it's also a cool open-source project that is part of the CNCF, you can check out the GitHub to go and investigate. What it's great at is acting as a common API and system for distributed event-driven applications.

For what reason could I have been interested? Well, like most of us, I've been building a side project or two between contracts. And the one I'm most interested in finishing (feel free to ask me about it) is a distributed app across multiple vendors. I have a selection of the application in Railway doing responsive quick things and handling anything I'd need to worry about being highly available, and then the big static data crunching bits in containers on Digital Ocean. Each one handles its queues and pub/sub, and to communicate with each other they have "private" APIs to send and receive stuff.

Wouldn't this be so much easier with something like Dapr acting as a common API to do service invocation of HA to long processes? Or pub/sub for anything that needs to be done immediately. Would you be happy to pass it back and forth for the user?

Unfortunately for me, I haven't had the time to work out how to stand up and secure Dapr between the two vendors so it got left behind.

But the title gives it away, Dapr is an open-source open-core technology. And as some of you may know, I love open-core tech.

How to produce MVP on Open Core projects | Mike Elsmore posted on the topic | LinkedIn

The greatest thing about Open Core projects is being able to produce the MVP on completely open tooling like Novu or configuring OpenTelemetry before having to look into vendors to use. If you have the time and capacity to do it this way I whole heartedly suggest you do it this way, saves the headache of realising something isn't the right fit and replacing it later.

linkedin.com

One of the organizations behind Dapr is called Diagrid, and they have a managed service product for running Dapr workloads on Kubernetes called Conductor. But what I'm excited about is what they announced the other day, the public beta for Catalyst (read about it here https://www.diagrid.io/blog/announcing-catalyst-public-beta).

This is exciting in my specific use case of multi-vendor IaaS & PaaS without having to manage it myself. That means I can probably replace my RabbitMQ instances in each vendor with Catalyst, as well as use its state management tooling as its key/value storage to handle data being shared between long-running processes without a shared database.

You can see what Catalyst is currently capable of in this diagram.

It even has a diagram explaining how I could use service invocation rather than "private" APIs to handle triggering services across vendors

Ignoring that the diagram is Lambdas rather than Railway to Digital Ocean, this would mean me removing a lot of retry boilerplate!

If you're interested, I'd read about the capabilities of Dapr in general. And I'd jump on to the public beta to try it for yourself.

Catalyst Public Beta

Some thoughts on DevTalks Bucharest 2024

Mike Elsmore — Thu, 27 Jun 2024 15:26:15 +0000

I'm an exceedingly lucky person, over the years I've had the opportunity to attend and speak at (including very last minute) multiple DevTalks events. And I was exceedingly happy to be invited to both moderate and speak at the 11th edition in Bucharest between May 29th & 30th.

If you'd like a full run-down of all the speakers, talks, and activities they had I'd check out the website https://www.devtalks.ro/ and the very active social media they run.

I will admit that on day two, I spent nearly every moment on the product world stage being a moderator so all my thoughts on that are limited. But I can at least share about day one, and the expo booth. So here are my thoughts about the event.

Expo

Let's get the expo out of the way first, as that covers both days. In the event space, it had a HUGE expo between all the stages with amazing brands that I recognised and huge booths from local brands I didn't. You could chill and work in a dedicated workspace area that came with quiet/meeting room pods, and you could play with or watch so many robots including football and drawing… it was nuts.

And the refreshments just kept coming! Amazing food trucks outside, coffee from plenty of Nespresso Professional machines with baristas, beers from Heineken (I can't stand the beer, but at least it was available), and all-you-can-drink soft drinks from Pepsi.

Day One

Unfortunately, after a very stressful travel day into Bucharest from sunny Birmingham I slept in a little so I missed the opening speech and keynote (sorry dear reader). But I did manage to catch some awesome talks between bumping into old friends.

The first talk I managed to stand at the back for was Francesco Ciulla from Daily.dev on the Web Stage. The talk was titled Evolution of Web Development and I enjoyed hearing about how the culmination of standards was being used by Daily.dev to produce an amazing experience that's scalable across browsers.

// Detect dark theme var iframe = document.getElementById('tweet-1795729492985696427-934'); if (document.body.className.includes('dark-theme')) { iframe.src = "https://platform.twitter.com/embed/Tweet.html?id=1795729492985696427&theme=dark" }
Next up I saw some of Vanessa Villa from Pangea on the DevLead Stage. This talk was something a little different for me as I'm not an IoT person beyond a little smart home tech, it was titled "IoT Evolution: Where is it now?". I'll admit she had my curiosity about the industrial applications of IoT right now, but I'm far removed from that, unfortunately.

// Detect dark theme var iframe = document.getElementById('tweet-1795730884118884493-961'); if (document.body.className.includes('dark-theme')) { iframe.src = "https://platform.twitter.com/embed/Tweet.html?id=1795730884118884493&theme=dark" }

Another talk from the DevLead stage had me watching my old friend Alex Lakatos from Interledger. "Building a Developer-first Culture" was the title, and this one was way too close to home for me. Having seen how Alex evolved this process, and how true it is. The biggest key takeaway is, with a small team, set the collaborative and open framework early so that everyone wants to get involved with sharing how awesome the product they work on is!

// Detect dark theme var iframe = document.getElementById('tweet-1795797219745894735-224'); if (document.body.className.includes('dark-theme')) { iframe.src = "https://platform.twitter.com/embed/Tweet.html?id=1795797219745894735&theme=dark" }

The final talk of day one that I got to see was Elad Shechter from Appwrite on the Web Stage. His talk, titled "Update Your < Style >!", involved a lot of CSS, and seemed like black magic. But it was interesting to see how he implemented a grid as someone terrible at CSS.

Day Two

Day two was a little different, I was moderating all day so I sat on one stage and watched it all. I was also a little stupid and forgot to take photos and post them to social media for future reference.

I was moderating the Product World Stage that was powered by METRO.digital (the digital wing of the METRO supermarkets). So it's only fitting that day two was opened up by representatives from METRO.digital.

First up, we had Adrian Postelnicu who is the CPO for METRO.digital setting the scene for the day.

Immediately from Adrian, we were introduced to Aura Virgolici (Engineering Manager at METRO.digital) & Irina Poiana (Domain Owner at METRO.digital). They gave a talk titled "Bridging the Gap: A Journey to High-Performing Teams" and it was about the pain points of a digital transformation program METRO.digital) went through.

The second full talk of the day was from Nesrine Changuel giving a talk named "The Secret to Crafting Lovable Tech Products". Here she covered the idea of "delight" which was an interesting concept for features that don't necessarily bring direct ROI but do engage and make users love a product. You can even download the Product Delight Map from the talk.

Our first AI talk of the day was given by Miro Alexandrov from Ipsos, his talk was "Leveraging Generative AI and Product Management Frameworks: How Ipsos is transforming Market Research". I did enjoy how he described Ipsos's use of AI to classify and enhance the existing research they have to improve and speed up projects.

The last talk before the lunch break was Lucian Gruia from Ciklum. The talk "Coding Privacy - Aware Enterprise AI with RAG Architecture" was an overview of RAG and how it's implemented, which for someone like me who knows little about the implementation was a great starter.

After lunch we had Dan-Mihai Dinu from Bolt who had the most dramatic and interesting start to his talk with costumes and all. The talk itself was called "Bolt Food: The Secret Recipe" and was an interesting run-through on the speed of development and iteration that Bolt has gone through with projects.

The next talk from Stefan Tudor Murariu was titled "The essential tools for early-stage product companies". His talk was about product development, giving some insight into getting over those first hurdles for finding product fit.

Following on we had two speakers from Swissquote, Edwige Fiaclou (Head Software Engineer Tech Talent & Methodologies) and Laetitia Aegerter-Cuello
(Senior Agile Coach). The session "Agility à la carte: Product and Delivery a tasty combination for innovation" was a fun (it had Toblerone!) walk-through the Disciplined Agile process and it's effects on product delivery at Swissquote.

Our last AI talk of the day was from George Dita with a talk titled "Empower Your Product Management with AI: Enhance, Don’t Replace Your Unique Skills". George gave a breakdown of a toolchain they use for decision-making and process automation around product management.

The final talk of the day (hurray it's over) was from yours truly, with the help of my friend Mike Dolha (you can find him on Instagram, LinkedIn, or his podcast). My talk was "The ABC of DX", which was meant to be a practical guide to things organisations can do to improve internal and external Developer Experience so people love what they're building on or with. Mike then drove a quite hilarious Q&A session at the end. If you'd like a write-up of this talk and my thoughts etc please feel free to add a comment.

The conclusion

DevTalks is an event that will always hold a special place in my heart, but Ow My God is it so busy! You cannot convey everything you could learn in a simple post. If you have the opportunity to attend, you should as Romania is a beautiful country with lovely cities (and food). And if you'd like to give a talk, the organisation behind it always runs a CFP. They also have a bunch of other events throughout the year

Exciting times are ahead this autumn, with three extraordinary events… | DevTalksRomania

Exciting times are ahead this autumn, with three extraordinary events bringing the IT Community of Romania together. Make sure to mark your calendars! 🗓 ➡ September 26th: DevTalks Cluj-Napoca at Cluj Innovation Park. Grab your early-bird tickets: https://bit.ly/3zl1xeu ➡ November 6th-7th: DevCon Bucharest at NORD Events Center by Globalworth. Stay tuned for more info! Meanwhile, check out what last year’s edition was like at www.dev-con.ro. ➡ November 21st: Empower - Our newest event, making its debut at NORD Events Center by Globalworth: www.empower-tech.ro/ 🙌 We’re working hard to create the best experience for you. Follow us to stay updated and be the first to know more about our events!

linkedin.com

You should build an SDK

Mike Elsmore — Thu, 22 Feb 2024 17:03:45 +0000

Over the last little while in my career, I’ve been either involved in building or building SDKs, especially here at Infobip with the huge array of tools we have on offer. And this leads me down a rabbit hole of wondering why organisations don’t build them by default once they reach a certain size if they follow an API-first design approach.

I’ve actually given talks on this subject a few times, if you want to skip reading, you can watch the last recording at

What is an SDK?

Let’s cover the basics: what is an SDK? The acronym itself means Software Development Kit, so it’s a toolkit for developing software. According to Wikipedia, the definition of an SDK is

A software development kit (SDK) is a collection of software development tools in one installable package. They facilitate the creation of applications by having a compiler, debugger and sometimes a software framework. They are normally specific to a hardware platform and operating system combination. To create applications with advanced functionalities such as advertisements, push notifications, etc; most application software developers use specific software development kits.

https://en.wikipedia.org/wiki/Software_development_kit

This means that I, and many like me who have treated SDKs as whatever the package manager installs, are a bit wrong (not totally, just a bit). What I mean by this, is that Client Libraries, like those that wrap an HTTP API, are not the SDK but a part of an SDK. An SDK is a collection of tools; this can include your client libraries, but also other dev tools. It could include testing tooling, webhook definitions, or even mathematical models for doing a task. Those of us who build primarily for the web or for mobile applications forget that they can also be physical SDKs. Game developers working on next-gen hardware regularly get access to kits like these for the PS5

At Infobip, we have a lot of SDKs, you can check them out here on our SDK page, but what I’ve been calling our NodeJS SDK is just a client library for now.

Why bother building an SDK?

Most people I know who are building SDKs are trying to enable a developer outside of their organisation, be it open source or proprietary, to consume the tools and products being built. I think this is wrong, and for a very simple reason, internal & external developers deserve the same tools.

Yes, external developers need help and ease of onboarding or use to engage with technology. But so do internal developers, it’s not uncommon for organisations to be running different services, with different technologies, and differing knowledge or experience levels.

Say the organisation you are in has some “legacy” systems running that no one actively works on but are used to grab information or tools that have been brought in over years and never updated. An internal client library providing a consistent set of functions to use them, as well as ETL tooling to make sure older standards are converted to the currently used ones, is a great SDK to build internally and manage. Or you could have REST, GraphQL, and WSDL systems that don’t have specification files or complete documentation on how to handle everything. A well-made SDK could solve that for teams.

Essentially, for internal and external developers, an SDK can lower the learning curve, which leads to speeding up implementation and provides consistency across everything being built.

How can you build one?

SDKs are meant to make life easier for the person using them, so the advice around developing them is rather straightforward.

If you are developing a wrapper library or code to be used or to be imported and used in a code base, follow the instructions and best practices for the languages or frameworks it’s being developed for. For example, if you’re creating it for the Express framework in NodeJS then you’ll need to follow the middleware pattern that it requires.

Where possible, use a namespace structure to allow people to only include the parts of the SDK they require if they do not need to use the whole thing. For example, fastify is a web framework in NodeJS and only the core structure needed for routing and requests are within the main fastify module. For everything else, that is expected of a full nuts and bolts framework, they’re included as an ecosystem of @fastify modules, like so

If you are developing a client library, follow a standard. Don’t hop about code styles, etc., as it’ll defeat the point of being easy to consume. And whenever you have to break the standard, make it known. Most languages and development environments allow you to easily see things like DocBlocks to explain why and also how to consume this edge case.

Let’s move on to something controversial among my peers. Personally, I think it is totally OK to use generators to produce client libraries. If you’re using standards like the OpenAPI Specification, you can use a plethora of generators to produce the first version of your client library codebase. And most importantly, where it isn’t fit for your purposes, you can modify the template, or completely extend the generator to give you the library you want. I definitely endorse this practice, as it’s something we do here at Infobip.

And now for some best practice tips, just to make everyone’s life a little easier to maintain them. Use version control on the code base that’s in-line with the language it’s been developed in. Additionally, where possible, use dependency management tooling and static analysis tools to make the maintenance and security risks manageable.

Automating your CloudQuery Policies with CircleCI

Mike Elsmore — Tue, 01 Mar 2022 10:48:20 +0000

This blog is going to cover running CloudQuery in CircleCI as part of your continuous integration or continuous delivery pipeline.

We all know how awesome a tool CloudQuery is, with a bit of configuration you very quickly get access to the cloudquery fetch command to get information on your cloud assets, and then you can start running cloudquery policy run <policy> against that to get amazing guidance on your cloud infrastructure. But what about automating the process, or making it accessible to the rest of your team?

You could use our helm chart to deploy a dedicated and persistent version of CloudQuery, or if you want to experiment with you could add this to your CI/CD processes. I personally really like using CircleCI and have been an active user since about 2017, so for this example, we’ll be creating a CircleCI workflow (and template) to achieve this.

What is CI/CD?

CI, or continuous integration, in its simplest form is the means of continually adding and integrating code into a shared body of code. In Git terms, this is the creation of changes in a branch, followed by the testing, and then merging into the main branch.

CD, or continuous delivery, is the methodology by which you can deploy your codebase at any time. This means making sure that the codebase is tested, and has an automated release process, rather than manually grabbing the code and loading it to a location.

How to add CloudQuery to CircleCI?

Prerequisites

Make sure you’ve followed one of our Getting Started Guides to make sure you have a valid and functional config.hcl for CloudQuery to use.

Getting started with CircleCI

If you’ve never used CircleCI before, rather than walk through all the steps here I’d recommend reading through the Your First Green Build guide in the CircleCI documentation. This guide will take you through the creation of your .circleci/config.yml and what the core parts are.

Once you’ve got that started, here is how to add CloudQuery to your CircleCI flow.

The first step is to add to the list of jobs, each of these jobs is a distinct task and process to complete. Here is the job to fetch data to a Postgres instance:

# Define a job to be invoked later in a workflow.
# See: https://circleci.com/docs/2.0/configuration-reference/#jobs
jobs:
  build:
    # Specify the execution environment. You can specify an image from Dockerhub or use one of our Convenience Images from CircleCI's Developer Hub.
    # See: https://circleci.com/docs/2.0/configuration-reference/#docker-machine-macos-windows-executor
    docker:
      - image: cimg/base:stable
        environment:
          CQ_DSN: postgresql://postgres:pass@localhost/postgres?sslmode=disable
      - image: postgres:11
        environment:
          POSTGRES_USER: postgres
          POSTGRES_PASSWORD: pass
          POSTGRES_DB: postgres
    # Add steps to the job
    # See: https://circleci.com/docs/2.0/configuration-reference/#steps
    steps:
      - checkout
      - run:
          name: "Install CloudQuery"
          command: curl -L https://github.com/cloudquery/cloudquery/releases/latest/download/cloudquery_linux_x86_64 -o cloudquery && chmod a+x cloudquery
      #  Wait for Postgres to be ready before proceeding
      - run:
          name: "Waiting for Postgres to be ready"
          command: dockerize -wait tcp://localhost:5432 -timeout 1m
      - run: 
          name: "Fetch AWS Resources"
          command: ./cloudquery fetch

When you have a new job, you need to set the container that will execute the task and any supporting containers. In this case, we have an image of cimg/base:stable which is a basic Linux image to execute tasks as our primary container with a supporting postgres:11 image.

We also have to provide the CQ_DSN variable, following this ENV substitution guide, to point to the Postgres instance within the CircleCI job.

Next, we have the steps, these are the commands executed on the primary container to add, configure, and perform tasks. The first task is usually checkout, this command will clone your codebase from your git repository to run against.

The following step is Install CloudQuery, this follows the Download and Install process of adding CloudQuery to a Linux instance and configuring it to be executable.

      - run:
          name: "Install CloudQuery"
          command: curl -L https://github.com/cloudquery/cloudquery/releases/latest/download/cloudquery_linux_x86_64 -o cloudquery && chmod a+x cloudquery

Due to the way CircleCI parallelises its container, it does take a moment for the Postgres image to be ready so we do have a wait step.

      - run:
          name: "Waiting for Postgres to be ready"
          command: dockerize -wait tcp://localhost:5432 -timeout 1m

This step is using dockerize to make sure the Postgres image is available to use before executing any commands against it.

And finally, we run CloudQuery itself.

      - run: 
          name: "Fetch Resources"
          command: ./cloudquery fetch

Using cloudquery fetch the container either uses the .cq information if committed to the repository or downloads a fresh version of the providers configured and grabs the data from your provider to use.

Configuring CircleCI

Now for this to work, you’ll need to add the access keys etc in CircleCI. It’s usually best to do this on a per-project basis. Within CircleCI go to projects, then select the project in question:

From here navigate to Project Settings:

From here select Environment Variables, and from here you can add your ENV variables as a key-value pair:

To make this work, you now need to add a workflows reference. This is how you chain jobs together to accomplish tasks:

# Invoke jobs via workflows
# See: https://circleci.com/docs/2.0/configuration-reference/#workflows
workflows:
  fetch-workflow:
    jobs:
      - build

This workflow fetch-workflow will now execute on every branch pushed to your git repository. But once the fetch is complete the data is lost as the Postgres image is destroyed after use, which isn’t much use for sharing the information gathered from the provider.

Adding a policy check

Once you have added a policy from the CloudQuery Hub or written your Custom Policy, you’ll want to run this nightly or after deployment and to this we can extend the example above quickly:

jobs:
  build:
    […]
  policy:
    # Specify the execution environment. You can specify an image from Dockerhub or use one of our Convenience Images from CircleCI's Developer Hub.
    # See: https://circleci.com/docs/2.0/configuration-reference/#docker-machine-macos-windows-executor
    docker:
      - image: cimg/base:stable
        environment:
          CQ_DSN: postgresql://postgres:pass@localhost/postgres?sslmode=disable
      - image: postgres:11
        environment:
          POSTGRES_USER: postgres
          POSTGRES_PASSWORD: pass
          POSTGRES_DB: postgres
    # Add steps to the job
    # See: https://circleci.com/docs/2.0/configuration-reference/#steps
    steps:
      - checkout
      - run:
          name: "Install CloudQuery"
          command: curl -L https://github.com/cloudquery/cloudquery/releases/latest/download/cloudquery_linux_x86_64 -o cloudquery && chmod a+x cloudquery
      - run:
          name: "Waiting for Postgres to be ready"
          command: dockerize -wait tcp://localhost:5432 -timeout 1m
      - run: 
          name: "Fetch Resources"
          command: ./cloudquery fetch
      - run:
          name: "Check AWS Policy"
          command: ./cloudquery policy run aws --output-dir ./
      - store_artifacts:
          path: ./aws.json
          destination: aws-policy

Our new policy job is the same as the previous build job except it has two additional steps. The first of which is Check AWS Policy step, this is running the entire AWS Policy set against the gathered data.

      - run:
          name: "Check AWS Policy"
          command: ./cloudquery policy run aws --output-dir ./

And because of the use of the --output-dir flag, it’s creating an aws.json for the results of the policy checks.

The final step is storing the aws.json as an artifact so that after the execution of the check, and the subsequent destruction of the image you can still review the results of the policy check:

      - store_artifacts:
          path: ./aws.json
          destination: aws-policy

Early we mentioned running this as a nightly task, CircleCI workflows have a trigger for this:

# Invoke jobs via workflows
# See: https://circleci.com/docs/2.0/configuration-reference/#workflows
workflows:
  fetch-workflow:
    […]
  policy-check:
    triggers:
      - schedule:
          cron: "0 0 * * *"
          filters:
            branches:
              only:
                - main
    jobs:
      - build
      - policy

Once you add the schedule to the triggers you can follow the crontab standard to decide when it’s executed, and any other filters you may want to limit this execution by. This is a legacy method of automating within CircleCI, for a more up-to-date approach that requires a little more configuration I’d use Scheduled Pipelines.

Summary

This blog should have covered all the aspects of using CloudQuery with CircleCI to automate your policy checks, and with that make it shareable with your teammates. If you’d like to discuss other ways to use CloudQuery in CI you can join our Discord and talk to us about your needs. And if we’ve missed anything we’d happily follow up this blog with more details, or even other CI/CD providers.

Running AWS Foundational Security Best Practices with CloudQuery Policies

Mike Elsmore — Mon, 28 Feb 2022 09:32:31 +0000

Back in mid-2020 AWS Security Hub released a new security standard called AWS Foundational Security Best Practices. This new standard sets security controls to detect when an AWS account or deployed resources don’t match up to the best practices set out by the AWS security experts. The complete standard can be found in the AWS Security Hub documentation.

As with any security guidelines, factors such as AWS environments, requirements, and capacity of your security team, will impact how you implement those guidelines.

The new AWS Foundational Security Best Practices CloudQuery policy gives you a powerful way to automate, customize, codify, and run your cloud security & compliance continuously with HCL and SQL.

The CloudQuery AWS Foundational Security Policy covers 200+ checks - you can review them on GitHub or review them in the CloudQuery Hub.

Prerequisites

Please follow the Getting Started documentation on how to install cloudquery, and fetch your AWS configuration into a PostgreSQL database.

Running

After fetching your AWS configuration into a PostgreSQL database, you can use SQL to check your cloud deployment for compliance!

For example, you can check for certificates that are going to expire soon and need to be renewed.

#https://github.com/cloudquery-policies/aws/blob/main/queries/acm/certificates_should_be_renewed.sql

SELECT arn

FROM aws_acm_certificates

WHERE not_after < NOW() AT TIME ZONE 'UTC' + INTERVAL '30' DAY;

You can also use the cloudquery command to run the entire AWS Foundational Security Best Practices policy pack. The policy is split into sections as sub-policies, so you can run either the entire policy, a sub-policy, or even one specific check.

# execute the AWS foundational-security-best-practices policy pack

cloudquery policy run aws//foundational_security

# execute the ACM section in AWS Foundational Security policy

cloudquery policy run aws//foundational_security/acm

# execute the S3 related section in AWS Foundational Security policy

cloudquery policy run aws//foundational_security/s3

# describe all available policies and sub-policies available for AWS on cloudquery

cloudquery policy describe aws

# execute the entire AWS policy pack, including other benchmarks.

cloudquery policy run aws

You can also output the results into a json and pass them to downstream processing for automated monitoring and alerting.

cloudquery policy run aws//foundational_security --output-dir=results

Build your own and share!

Do you have a policy that you want to codify, or that you’ve been running with python or bash scripts? You are welcome to try codifying it with CloudQuery Policies (See our github and docs for how to develop one). Feel free to visit our discord or GitHub to get help - we’ll also be happy to share your policy on CloudQuery Hub.

What are policies and how do you use them with CloudQuery?

Mike Elsmore — Mon, 21 Feb 2022 13:00:10 +0000

Policy is a very broad term and in this blog we’ll be explaining what policies are, how we implement them at CloudQuery, and how you can use this in the security & compliance of your applications.

We have a good definition of what a policy is within our documentation, and it reads as so

Policy compliance is a broad term and can refer to any kind of policy, from internal standards to regulatory requirements. A CloudQuery Policy is a codified form of this that is written with HCL as the logic layer and SQL as the query layer.

Simplified this means a policy is a set of rules to be followed, for example, a password policy requiring you to

Be alphanumeric
Have a special character
Have a number
Changed every 4 weeks
No repeat password for 13 weeks

This is a really basic policy that is standard for a lot of applications or organisations.

In the wild various vendors have policies, and various standards bodies have varying policies that your applications may have to validate against to be classified as safe for certain usage. Even huge providers like AWS have made themselves compliant with ISO 27001 and a whole gamut of others.

What are the common policies?

A few of the most common policies you can come across are things like HIPAA (US healthcare) which has a full summary of security rules to follow. But for most of us in the software industry, we are going to come into contact with these two, CIS (Center for Internet Security) and PCI DSS (Payment Card Industry Data Security Standard)

How does CloudQuery work with policies?

CloudQuery is an open-source cloud asset inventory powered by SQL, so a policy to the tool is simply an HCL configuration file that references SQL queries. How this works is by CloudQuery ingesting the data from your cloud provider, such as AWS or Azure, and then the policy executes the SQL statements against that data as if it were a test suite. Our co-founder Yevgeny recently wrote an amazing piece on running the PCI DSS policy against AWS. If you are curious about the multitude of policies currently available you can check them out at https://hub.cloudquery.io/ which is our centrally available and searchable source for publicly available policies.

How to use policies?

As an example, we shall use AWS to check for compliance. Initially, you will need to install CloudQuery, to do this follow the instructions available in the Getting Started section of our documentation for your given operating system.

Now that you have CloudQuery installed you will need to create the configuration, with the provider you wish to connect it to which in our case is AWS

cloudquery init aws

This will create the config.hcl relative to where you ran this command, for a breakdown of how this is structured it’s available here https://docs.cloudquery.io/docs/configuration/overview.

If you don’t already have a Postgres instance running for the data (you will need to change the connection block of the config.hcl if you do) then run the following command

docker run -p 5432:5432 -e POSTGRES_PASSWORD=pass -d postgres

Once this is started you can run the following command to start the process of getting the data from AWS

cloudquery fetch

Now that you have the asset configuration available you can run policies against them. If you’d like to download them in advance you can execute

cloudquery policy download aws

This downloads the policy configuration and SQL to use later within the .cq directory, it also returns a block that you can add to your config.hcl that may look similar to

policy "aws" {
  source = "aws"
}

If you would like to skip the download step, you could execute the following commands to review and execute the policies:

# execute the whole policy pack (cis, pci dss, etc)
cloudquery policy run aws

# execute specific policy pack
cloudquery policy run aws//pci_dss_v3.2.1

# execute specific section in PCI DSS
cloudquery policy run aws//pci_dss_v3.2.1/autoscaling/1

It’s that simple and you will get a PASS/FAIL return list from all these existing policy statements.

Making a custom policy?

If you want to make a policy to automate some of your checks rather than writing the SQL repeatedly or storing them as a stored procedure, then it’s relatively easy.

For a full tutorial on creating policies, please follow the tutorial in our documentation, but for a quick guide keep reading.

The first step to a custom policy is to allow cloudquery to find it, the easiest way to do that is to add a policy block to the config.hcl and this can look like so

policy "my-custom-policy" {
  source = "./path/to/policy/directory"
}

Inside your new policy you will need a policy.hcl to define what your policy is, a simple example can be just like so:

policy "my-custom-policy" {
  title = "This is a test policy"
  doc = file(“README.md”)
  configuration {
    provider "theprovideritusesname" {
      version = ">= 1.0"
    }
  }
  …
}

After defining what the policy is, and its prerequisites for data such as the provider or documentation, you need to define the actual policy definitions

policy "my-custom-policy" {
  title = "This is a custom test policy"
  doc = file(“README.md”)
  configuration {
    provider "theprovideritusesname" {
      version = ">= 1.0"
    }
  }

  policy "nested-policy-peter" {
    title = "A policy about Peter"
    source = file("peter/policy.hcl")
  }

  policy "nested-policy-james" {
    title = "A policy about James"
    source = file("james/policy.hcl")
  }
}

Now for the policy itself, or the sub-policy in this case, we have out my-custom-policy and we are now creating the “peter” sub-policy. This should look like so:

policy "peter" {
  title = "Policy specifically for Peter"
  doc = file("peter/README.md")
  configuration {
    provider "theprovideritusesname" {
      version = "> 1.0"
    }
  }

  policy "meal" {
    source = file("peter/meal.hcl")
  }
}

You can see that this sub-policy defines the prerequisites, like the provider that supplies its data etc, as the initial policy did before. And as before the policy block here references another hcl, but this one contains the query to execute and looks like so:

policy "meal" {
  title = "Section Meal: Information regarding Peter’s meals"
  doc = file("peter/docs/meals.md")
  check "breakfast" {
    title = "What does Peter have to breakfast"
    doc = file("peter/docs/breakfast.md")
    query = file("queries/meals/breakfast_eaten.sql")
  }

  check "lunch" {
    title = "What does Peter have to lunch"
    doc = file("peter/docs/lunch.md")
    query = file("queries/meals/lunch_eaten.sql")
  }

  check "dinner" {
    title = "What does Peter have to dinner"
    doc = file("peter/docs/dinner.md")
    query = file("queries/meals/dinner_eaten.sql")
  }
}

Now you can see the policy specifically for Peters meals, and we aren’t going to another nested level, instead we have the check block. This block contains the title of the check, any doc concerning it, and a reference to the SQL query that’s contained in a .sql file (NOTE: you can just write the SQL into the query block-like `query = “SELECT 1;”, but we don’t advise that for query reuse).

tip All thefileincludes are from the root directory that you set the firstpolicy.hcl` in, leaving you with a directory structure a little like so:

policy.hcl
peter
- Docs
  - Meals.md
  - Breakfast.md
  - Lunch.md
  - dinner.md
- policy.hcl
- meals.hcl
queries
- meals
  - breakfast_eaten.sql
  - lunch_eaten.sql
  - dinner_eaten.sql `

And with this, you can create policies to automate the checks, or your internal organisation policies, against your assets in the cloud. If you’d like more documentation on policies, you can check out the policy documentation or the hubs policy page.

If you are looking for an open-source cloud asset inventory powered by SQL, check out our GitHub.

Also, Feel free to join our Discord if you run into any bugs/issues, or just want to chat.

AWS SSO Tutorial with Google Workspace (Gsuite) as an IdP Step-by-Step

Mike Elsmore — Fri, 26 Nov 2021 14:44:01 +0000

AWS SSO and AWS Organization were released around 2017 and are probably the best way to manage AWS access at scale.

"AWS Single Sign-On (SSO) is a cloud SSO service that makes it easy to centrally manage SSO access to multiple AWS accounts and business applications. It enables users to sign in to an AWS IAM user with their existing corporate credentials and access all of their assigned accounts and applications from one place."
Quote From AWS SSO page

This is a huge security and operational win, some highlights:

No need to rotate another new password in AWS IAM
2FA is already managed at your IdP (Google Workspace (Gsuite)/Okta/AzureAD) level
When a user is leaving an organization he is automatically removed access from the organization
Easily automate the provisioning of AWS access when a user joins an organisation or department

In this article we, will go through a step-by-step guide to set-up AWS SSO with Google Workspace (previously Gsuite) as an IdP. If you are using Google Workspace and use it as your central directory, this is the guide for you.

Prerequisite

You should have the AWS Organization (If you are not using it, This service combined with AWS SSO is a real game changer) set-up.

You need to signup from the main account (also called "management account" ) and with enough permissions (usually Administrator permissions).

You will also need to make sure that you have access to the Google Workspace Admin and the relevant permissions to manage it.

Setting up AWS

Now that you have all the relevant permissions, everything is ready to configure for AWS SSO. Here is the step by step to set it all up:

From within the AWS Management Console search so single sign on and go to the AWS Single Sign-on.
Once on the service page, click the Enable AWS SSO button to start the service. This will take a few moments to complete.
Now that SSO is enabled, we need to change from the AWS directory to using an external provider. Select Choose your identity source.
Within the Settings page, select Change under the Identity source section.
Now we can change from the AWS SSO directory to an Active Directory (not what we need), or an External identity provider which is what we need to configure Google Workspace as the provider.
After you have selected External identity provider, scroll down to Service provider metadata and click Show individual metadata values.
You should now be presented with three fields that you can use to configure the next step on Google Workspace in the Google Admin console.
Don't close this screen, you will need it shortly after you have done the next section.

Google Workspace SAML setup

With the SSO Urls for our AWS organization, we can go to our Google Workspace Admin console and configure it.

When inside the Google Workspace Admin console, go to the Web and mobile apps settings. You can find this in the left-hand navigation menu under Apps.
Then select Add App from the top navigation, then Add custom SAML app.
Add an App name for the integration, I'm using AWS SSO to make it easier to find later.
We suggest you download the Google IdP metadata ready to put it back into AWS, this is under Option 1: Download IdP metadata.
Now to add the AWS SSO Urls from earlier to configure Google Workspace to point to the correct location. The mapping of data is:

- For ACS URL, enter the AWS SSO ACS URL.
- For Entity ID, enter the AWS SSO Issue URL.
- For Start URL, leave the field blank.
- For Name ID format, choose EMAIL.
- For Name ID, choose Basic Information > Primary email.

We don't need to apply anything to the Attribute mapping settings, so you can just click FINISH to move forward.
Once that's saved, it is time to enable it for everyone. In the User access section, open the settings by selecting the karat in the top right corner.
Now that you're in the Service status screen, select ON for everyone and SAVE. This will enable the service and allow you to manage who can have access to AWS, but to configure what they can access you need to do that in AWS SSO as Google Workspace is unaware of all the possible options.

Adding Google Workspace configuration to AWS SSO

Now that the AWS SSO service is enabled, and the Google Workspace SAML app exists, it's time to make them talk to each other.

Go back to the Change identity source screen in AWS SSO. Scroll to the bottom and add the GoogleIDPMetadata.xml file you downloaded a few moments ago, then click Next: Review.
To confirm this new identity source, you will need to type ACCEPT into the field under the warnings and then select Change identity source.
And now you are done with configuring the SSO and SAML connection between AWS SSO and Google Workspace. However, you aren't quite done as you need to configure the user provisioning at this point.

Setting up Users and Permissions

As of writing this, you can't automatically sync users between AWS and Google (this is being worked on over at OpenID) so we are limited to two options; manually creating the user (which we will go through) and using https://github.com/awslabs/ssosync to automate the process.

To manually add users, you will want to follow these instructions.

Go to Users in the sidebar of the AWS SSO service. Then select Add user.
Use the primary Google Workplace email address as the Username as well as the Email address, and fill the other fields accordingly. Then hit Next: Groups to save.
As part of this process, we aren't going to be adding groups so we can skip these phases by selecting Add user in the bottom right.
The user now needs to be associated with an AWS account. So select AWS accounts from the left navigation, select the checkbox next to the user, and click Assign users to attach them to the account.
On the next screen select the user again so that we can move on to permissions by clicking Next: Permissions sets.
As we haven’t configured and permission sets before we will have to do that now by clicking Create new permission set.
We will be using Use an existing job function policy, these are like AWS managed policies that you will be aware of if you have configured permissions inside AWS previously. Now Next: Details to select the policy.
From here you can select the policy you want to assign, I'll be using AdministratorAccess as this is for myself. But you could use PowerUserAccess as this would allow the user to build whatever they want, but not mess with other users and groups. Then click Next: Tags to apply this to the user.
Tags are optional, but they are advised for auditing and search at a later point. But I don't need them so we select Next: Review to move forward.
A quick once over to make sure everything is set correctly, then we can click Create to do so.
Back to our Assign Users screen, we can click the refresh icon to view our permission sets. From here we select the checkbox for the permissions set we want for the user, then select Finish to apply it.
It'll take a moment to provision, but you should get a Complete screen saying you are done.

And we are done, now the user can authenticate and log in from Google Workplace using the handy link in the Google apps selector.

Summary

By now you should have AWS SSO configured with Google Workspace as an IdP and you can manage access & permissions to your AWS in the AWS SSO service.

What is infrastructure drift?

Mike Elsmore — Tue, 23 Nov 2021 14:05:10 +0000

I’ve spent most of my career as an application/web developer, so primarily glueing X to Y to do a thing that I love doing! And my exposure to operating through DevOps has been through the lens of an application developer.

I’ve recently joined CloudQuery, which is an open-source cloud asset inventory powered by SQL. What this means is that I’m learning more about infrastructure than I’ve ever done before. With that, I’m learning about things like infrastructure as code through Terraform and the concept of infrastructure drift.

What is drift?

Simply put, it’s when the state of your infrastructure is different from the state set by your IaC (Infrastructure-as-Code). For example, if you’ve deployed five lambdas via terraform with the default 128 MB of memory, but now three of them are using 256 MB then you have drift in the states.

What causes drift?

My reading has led me to believe that you can have two kinds of drift, the first is like the above which is Resources Managed by IaC and the second is Resources Not Managed by IaC. The first is when all your resources have been deployed by IaC like Terraform or CloudFormation, the second is resources deployed manually or via another process that doesn’t use IaC. The things that cause drift are the same in both cases, either someone or something manually adding or changing individual resources.

Is drift bad?

Not necessarily, things happen in production, you have to tweak and keep things up and alive when apps are on fire. And sometimes, you need resources outside of your IaC, say security tooling that needs oversight etc.

But how do you know if you have drifted?

And this is where we come to why I went down this rabbit hole, CloudQuery now has drift detection built on top of the open-source cloud asset inventory. You can detect drift via another tool, or via Terraform itself, but these only work against resources managed with Terraform. But with CloudQuery you can see not only the drift from your Terraform state, but also the resources within your Cloud vendor account that aren’t managed as an easy to identify list.

How do I resolve the drift?

If you head to the documentation for cloudquery drift scan you can see an example result, like so:

=== DRIFT RESULTS  ===
5 Resources not managed by Terraform
aws:ec2.ebs_volumes:
- vol-id1
- vol-id2
aws:ec2.instances:
- i-id1
- i-id2
aws:ec2.security_groups:
- sg-id1
93 Resources managed by Terraform (equal IDs)
=== SUMMARY ===
Total number of resources: 98
- 5 not managed by Terraform
- 93 managed by Terraform (equal IDs)
- 94.89% covered by Terraform

You can see the resources that are unmanaged by Terraform, as well as any resources that have drifted from your Terraform state. At this point, you can now decide which resources to leave unmanaged by Terraform, or you can follow the Manage Resource Drift guide in the Terraform documentation to begin managing them and matching the Terraform to production. And for the little piece of extra automation to make your life simpler, you could run CloudQuery as part of your CI to make sure everything is in the right state before or after a deployment.

DEV Community: Mike Elsmore

How to Stream Video with Just JavaScript and HTML5

See it in action

Table of Contents

Using the HTML5 <video> element

Adding playback controls

Defining multiple video sources

Displaying a video poster (thumbnail)

Auto-playing videos the right way

Building a custom video player with JavaScript

Adaptive Bitrate Streaming (ABS)

Using Video.js for advanced streaming

Quality selection

Optimising video delivery with ImageKit

Resize videos to appropriate dimensions

Automatic format selection

Generate ABS playlists

Add overlays

Conclusion

Choosing Your Documentation Tooling: A Practical Guide

The Landscape

Docusaurus

VuePress

Redocly

Fumadocs

Using Tools in Concert

Making the Choice

Conclusion

How to Add Video to Your Astro Site with ImageKit

See it in action

Why Astro for video content?

Setting up an Astro project

Basic video with HTML5

Creating a reusable Video component

Adding Video.js for advanced playback

Adaptive Bitrate Streaming in Astro

Optimising video with ImageKit

Resize to match your layout

Automatic format selection

Generate thumbnails from the video

Add watermarks and overlays

Lazy loading and performance

Persisting video across page transitions

Conclusion

The Differentiator Between B2B products: The Developer Experience

What is Developer Experience?

Documentation

Tooling

Support

Conclusion

Dapr in the cloud with Catalyst public beta

How to produce MVP on Open Core projects | Mike Elsmore posted on the topic | LinkedIn

Some thoughts on DevTalks Bucharest 2024

Expo

Day One

Day Two

The conclusion

Exciting times are ahead this autumn, with three extraordinary events… | DevTalksRomania

You should build an SDK

What is an SDK?

Why bother building an SDK?

How can you build one?

Automating your CloudQuery Policies with CircleCI

What is CI/CD?

How to add CloudQuery to CircleCI?

Prerequisites

Getting started with CircleCI

Configuring CircleCI

Adding a policy check

Summary

Running AWS Foundational Security Best Practices with CloudQuery Policies

Prerequisites

Running

Build your own and share!

What are policies and how do you use them with CloudQuery?

What are the common policies?

How does CloudQuery work with policies?

How to use policies?

Making a custom policy?

AWS SSO Tutorial with Google Workspace (Gsuite) as an IdP Step-by-Step

Using the HTML5 `<video>` element