DEV Community: Farhan Munir

Configurable Video Transition Duration in Reel Quick

Farhan Munir — Sun, 26 Jul 2026 09:38:03 +0000

Video transitions are one of those details that quietly shape the feel of an edit.

In Reel Quick issue #13, the goal was simple: let users control how long a scene transition lasts instead of forcing a fixed value.

Issue URL: https://github.com/ronin1770/reel-quick/issues/13

The problem

The app already supported transition effects between scenes, but the duration was fixed.

That meant creators could choose what transition to use, but not how long it should run.

For short-form video, that matters a lot:

fast transitions create a snappier pace
longer transitions feel smoother or more cinematic
some edits need no transition at all

The feature

The new behavior adds a configurable transition duration:

minimum: 0.0 seconds
maximum: 4.0 seconds
step: 0.5 seconds

A slider in the frontend lets the user choose the duration, and that value is sent to the backend for FFmpeg video generation.

If the value is 0.0, transitions are disabled entirely.

The FFmpeg math

When two clips are joined with a transition, the transition overlaps the end of the first clip and the start of the second clip.

So the final duration is:

final length = clip 1 + clip 2 - transition duration

Example 1

Clip 1 = 7 seconds
Clip 2 = 8 seconds
Transition duration = 4 seconds

Math:

7 + 8 - 4 = 11 seconds

Final video length: 11 seconds

Example 2

Clip 1 = 7 seconds
Clip 2 = 8 seconds
Transition duration = 0.5 seconds

Math:

7 + 8 - 0.5 = 14.5 seconds

Final video length: 14.5 seconds

Why validation matters

This feature also needs guardrails.

The backend validates that:

the duration is between 0 and 4
the duration is a multiple of 0.5
clips are long enough for the selected transition

That last point is important. A 4 second transition cannot work safely if a clip itself is only 3 seconds long.

Implementation notes

The implementation touches both frontend and backend:

Frontend

add a transition duration slider
show the selected value beside it
send transition_duration in the video creation request
show inline validation errors

Backend

accept transition_duration in the API
validate the range and step
preserve the default behavior when no value is supplied
pass the value into FFmpeg transition generation

The key idea

The transition does not add extra time to the video.

It overlaps clips.

That is the core reason the final duration becomes:

sum of clips - overlap

For n clips, the total duration becomes:

sum(all clip durations) - sum(all transition overlaps)

Open source note

Small editing controls like this can have a big impact on creative flexibility.

This is exactly the kind of feature that makes open source video tools more useful in real workflows: not flashy, just practical and user-driven.

If you want to follow the discussion or implementation details, see the issue here:

https://github.com/ronin1770/reel-quick/issues/13

Implementing HTTP 429 Retry-After Handling in Heka Insights Agent

Farhan Munir — Sat, 25 Jul 2026 10:34:29 +0000

We opened a focused reliability issue in the Heka Insights Agent to improve how the OTLP HTTP exporter behaves under backend rate limiting.

Issue URL: https://github.com/ronin1770/heka-insights-agent/issues/22

Repository: https://github.com/ronin1770/heka-insights-agent

The problem is simple: if an ingestion gateway returns 429 Too Many Requests, the agent should not immediately keep retrying on a short generic backoff loop until all attempts are exhausted.

If the server says:

429 Too Many Requests
Retry-After: 20

then the agent should respect that instruction, wait, and retry the same metrics batch at the correct time.

Why This Matters

In telemetry pipelines, rate limiting is normal. Gateways protect themselves, vendors enforce quotas, and burst traffic happens.

What matters is how the agent behaves when that happens.

A weak retry strategy looks like this:

request fails with 429
exporter retries too quickly
all configured attempts are consumed before the rate-limit window resets
the batch fails even though the backend gave a valid recovery signal

That is not just inefficient. It also creates noisy logs, unnecessary traffic, and lower delivery reliability.

For an agent that is expected to stay running continuously, this is the wrong operational behavior.

The Goal of Issue #22

Issue #22 defines a more correct path for 429 Too Many Requests handling in the OTLP HTTP exporter.

The required behavior is:

detect 429 explicitly instead of treating it like a generic transient HTTP error
read the Retry-After header case-insensitively
support integer-second delay values
use a default delay when the header is missing or invalid
cap excessive values with configuration
retry the same OTLP payload
reuse the same idempotency key
avoid stacking exponential backoff on top of server-provided delay
keep the main agent process and collectors running
add logging and timing instrumentation around the retry path

This is a narrow but important reliability improvement.

The Core Design Choice

The most important part of the issue is not just "sleep and retry."

It is retrying the same batch with the same idempotency key after honoring the delay requested by the server.

That protects correctness in a few ways:

the exporter does not replace the pending batch with newer metrics
retry semantics stay tied to the original request
the backend sees the same logical delivery attempt after the rate-limit window
clients do not accidentally create duplicate semantics by regenerating request identity

This is especially important for observability delivery paths, where retries need to be operationally safe, not just convenient.

Configuration Rules

The issue introduces explicit control for Retry-After behavior:

OTLP_RETRY_AFTER_DEFAULT_SECONDS=5
OTLP_RETRY_AFTER_MAX_SECONDS=300
OTLP_MAX_EXPORT_ATTEMPTS=5

That gives the exporter deterministic behavior across common cases:

missing header -> use default
empty header -> use default
invalid value -> use default
negative value -> use default
zero -> retry without additional delay
value above max -> cap it

This is the kind of behavior that operators can reason about quickly.

Async Sleep Matters

One of the stronger requirements in the issue is that the wait must be asynchronous:

await asyncio.sleep(retry_after_seconds)

That detail matters because the agent should continue running normally while one export batch is waiting for its retry window.

The wait should not:

block the event loop
freeze collectors
terminate the process
force a broad retry redesign for unrelated HTTP failures

This keeps the scope disciplined while still improving real runtime behavior.

Logging and Timing

The issue also calls for better observability around exporter retries themselves.

That includes logging:

endpoint
HTTP status
current attempt
maximum attempts
parsed Retry-After value
applied delay
whether defaulting or capping occurred
masked idempotency-key reference
retry outcome

It also asks for timing data such as:

retry_after_parse_ms
retry_after_sleep_seconds
retry_request_ms
total_batch_export_ms

This is important because retry logic without timing visibility becomes difficult to debug in production.

Testing Scope

Issue #22 is also explicit about testing expectations.

The test coverage should verify:

header parsing for valid and invalid cases
case-insensitive header handling
same payload reuse on retry
same idempotency key reuse on retry
correct attempt accounting
no extra exponential backoff layered onto Retry-After
successful retry exit behavior
max-attempt failure without crashing the agent
clean shutdown during the wait
collector continuity while the exporter is waiting
no logging of secrets or raw OTLP payloads

There is also a clear integration expectation: return 429, wait the correct amount of time, retry the same batch, then succeed and exit the retry loop cleanly.

Why I Like This Issue

This is the kind of issue that improves a system in a practical way.

It does not try to redesign all retry behavior in one pass. It focuses on one real failure mode, defines the expected runtime behavior precisely, and forces the implementation to think about:

correctness
idempotency
async behavior
operational visibility
bounded configuration
testability

That is a good pattern for infrastructure work.

Read the Issue

If you want the full requirement set and acceptance criteria, the issue is here:

https://github.com/ronin1770/heka-insights-agent/issues/22

If you're interested in Linux telemetry agents, OTLP delivery behavior, or exporter reliability work, the repository is here:

https://github.com/ronin1770/heka-insights-agent

Reel Quick: Added Video Transition Effects and a Theme Selector

Farhan Munir — Wed, 15 Jul 2026 10:00:00 +0000

Reel Quick: Added Video Transition Effects and a Theme Selector

I’ve been improving Reel Quick, an open-source Instagram reels generator built with FastAPI, Next.js, and ARQ. The latest update adds two practical frontend/video workflow improvements: video transition effects and a theme selector.

The transition work makes reel stitching feel more polished by adding support for scene transition effects between clips. For this release, I also added sample input/output media so the behavior is easy to inspect directly. On the UI side, Reel Quick now supports a theme selector with multiple appearance modes, making the interface more flexible for different working environments and user preferences.

What was added

Video transition effects
Sample transition input clips
Sample output video showing a circle transition
Theme selector demo output video
Updated project documentation

Why this matters

For reel creation workflows, transitions improve the final viewing experience without forcing extra manual editing outside the tool. The theme selector improves usability for longer editing sessions and makes the app feel more production-ready.

Demo assets

Inside the repo docs folder you can find:

Transition input clips
Transition output example
Theme selector demo video

If you want to follow the implementation discussion and scope, check the linked issue above.

Adding RPM Packaging for CentOS 9 in Heka Insights

Farhan Munir — Sun, 12 Jul 2026 10:00:00 +0000

Heka Insights Agent now includes a dedicated RPM packaging workflow for CentOS Stream 9 and other Enterprise Linux 9-compatible distributions.

Repository: https://sl1nk.com/dpi1j6p

The project already supported packaged installation on Debian-based systems, but RPM-based environments need their own build format, package metadata, lifecycle scripts, and installation workflow.

This release adds that missing path.

Supported RPM-based distributions

The new packaging flow targets:

CentOS Stream 9
Red Hat Enterprise Linux 9
Rocky Linux 9
AlmaLinux 9

Instead of attempting to reuse a Debian .deb package, operators can now build and install a native .rpm artifact using the standard dnf and rpm ecosystem.

What was added

The repository now includes a dedicated RPM packaging structure:

packaging/rpm/build_rpm.sh
packaging/rpm/heka-insights-agent.spec
packaging/rpm/heka-insights-agent.service

The build script:

Creates the standalone agent binary with PyInstaller.
Prepares the RPM build directories.
Copies the binary, systemd unit, documentation, and configuration example.
Runs rpmbuild.
Generates the final RPM and SHA-256 checksum.

The generated artifact follows the standard EL9 naming format:

heka-insights-agent-0.1.0-1.el9.x86_64.rpm

Why native RPM packaging matters

A .deb package is not a CentOS release artifact.

Debian and Enterprise Linux distributions use different package formats, package managers, dependency metadata, and installation lifecycle hooks.

CentOS 9 and related systems expect:

RPM package metadata
dnf-compatible dependency handling
RPM installation and removal scriptlets
EL9-compatible binaries
systemd integration
Enterprise Linux filesystem conventions

Building the agent directly on an EL9 system also reduces compatibility risks with native libraries such as glibc.

Consistent runtime model

The packaging format changes, but the agent’s runtime model remains consistent.

The RPM installs:

/usr/local/bin/heka-insights-agent
/usr/lib/systemd/system/heka-insights-agent.service
/etc/heka-insights-agent/
/var/log/heka-insights-agent/

Configuration is completed through the setup command:

sudo /usr/local/bin/heka-insights-agent setup

The service can then be enabled and started with:

sudo systemctl enable --now heka-insights-agent

Service output is available through the systemd journal:

sudo journalctl -u heka-insights-agent -f

The package also creates the agent service account and prepares the required configuration and log directories with restricted permissions.

Cleaner release artifacts

Platform-specific output is now separated inside dist/:

dist/ubuntu/
dist/centos9/

This keeps Debian and RPM artifacts clearly separated and makes multi-platform release automation easier to maintain.

A release can now provide artifacts such as:

heka-insights-agent_0.1.0_amd64.deb
heka-insights-agent-0.1.0-1.el9.x86_64.rpm

Installation on CentOS 9

The generated RPM can be installed using:

sudo dnf install -y \
  ./heka-insights-agent-0.1.0-1.el9.x86_64.rpm

After configuration:

sudo /usr/local/bin/heka-insights-agent setup
sudo systemctl enable --now heka-insights-agent

The service can be verified with:

sudo systemctl status heka-insights-agent --no-pager

A practical release improvement

This is not a change to the telemetry engine itself, but it is an important operational improvement.

Teams using CentOS Stream, RHEL, Rocky Linux, or AlmaLinux can now evaluate and deploy Heka Insights Agent using a package format that matches their infrastructure.

That means:

fewer manual installation steps
clearer release documentation
native package management
easier upgrades and removals
more consistent production rollouts

Review the source, packaging scripts, and release documentation:

https://sl1nk.com/dpi1j6p

I Added Scene Transitions to Reel Quick with FastAPI, Next.js, and FFmpeg

Farhan Munir — Sat, 11 Jul 2026 06:13:26 +0000

Building reels by stitching short clips together works, but hard cuts can make the final video feel rough. I wanted Reel Quick to produce something smoother without forcing creators to leave the app and finish the job in a separate editor.

This update adds scene transitions to Reel Quick. When creating a reel, you can now choose a transition and have it applied automatically at every scene boundary. That keeps the workflow simple while making the final output feel more polished.

On the backend, I added a transition catalog for supported FFmpeg xfade effects, including fade, dissolve, fadeblack, fadewhite, wipeleft, wiperight, slideleft, slideright, circleopen, and circleclose. The API validates the selected transition, stores it with the video, and exposes transition records so the frontend can load only active options.

On the frontend, the create-video flow now fetches available transitions and lets the user pick one before the job is queued. From there, the worker passes that choice into the render pipeline, and FFmpeg applies the selected transition between trimmed clips during output generation.

One detail that matters: this feature depends on FFmpeg support for the xfade filter. If the local FFmpeg build supports it, Reel Quick can now turn plain stitched clips into cleaner, more watchable reels with no extra editing step.

This is the kind of update I like most: small on the surface, but meaningful in the final result. It improves creator output, keeps the API workflow clean, and moves Reel Quick closer to being a practical open source reel-production tool.

Output video: https://drive.google.com/file/d/1HqOj9ZctlApq2AO2ylS16YXYhy6gu7AO/view?usp=sharing

Issue: https://github.com/ronin1770/reel-quick/issues/1

Repo: https://github.com/ronin1770/reel-quick

Feature Request: Theme Switching for Reel-Quick

Farhan Munir — Tue, 07 Jul 2026 10:56:59 +0000

I have opened a feature request to add frontend theme support to Reel-Quick.

The proposed feature would allow users to select between Light, Dark, and System-default themes. The selected preference should persist across sessions and apply consistently throughout the application.

Because Reel-Quick is built with Next.js, this could be implemented using a theme provider such as next-themes, together with CSS variables, Tailwind dark-mode utilities, or the project’s existing styling approach.

Theme support would improve accessibility, reduce visual fatigue in low-light environments, and make the platform more adaptable for individual users and future team deployments.

The initial scope can remain simple: Light, Dark, and System modes. Later enhancements could include accent colors, high-contrast mode, compact layouts, or organization-level branding.

View, discuss, or contribute to the feature request here:

https://github.com/ronin1770/reel-quick/issues/11

Introducing Heka Insights Agent v0.1.0: Our First Release

Farhan Munir — Wed, 01 Jul 2026 18:08:25 +0000

We are shipping the first release of Heka Insights Agent.

Repository: ronin1770/heka-insights-agent

Heka Insights Agent is a lightweight Linux telemetry agent focused on a simple goal: collect host-level signals, normalize them into a consistent format, and make delivery flexible enough for different observability backends.

For this first release, we wanted to establish a solid base instead of trying to do everything at once. That means a working runtime, clear configuration, multiple exporter options, and a packaging story that makes installation easier on real Linux systems.

What Heka Insights Agent Is

At its core, Heka Insights Agent is designed to:

collect essential Linux host telemetry
normalize metrics into a consistent internal model
keep collection separate from delivery
support multiple backend paths without changing collectors
stay lightweight enough for continuous runtime use

That separation is important. It means the agent is not tightly locked to one vendor from day one.

What Ships in v0.1.0

This first release includes several important pieces.

1. Linux host telemetry collection

The project collects core machine-level signals such as:

CPU usage
memory usage
disk I/O
runtime health-oriented metrics

The goal is to keep the foundation practical and extensible.

2. Multiple exporter paths

This release supports a small but meaningful set of delivery modes through EXPORTER_TYPE:

console
otlp_http
datadog_otlp
datadog_native
newrelic_otlp

That gives teams a path to start locally, use standard OTLP HTTP delivery, or integrate with Datadog and New Relic using repo-supported configuration flows.

3. Strict runtime configuration behavior

One of the priorities in this release is predictable startup behavior.

Instead of silently accepting bad configuration, the runtime validates required settings and fails fast when critical exporter values are missing or malformed. That matters a lot for agents that are expected to run unattended.

4. Debian package build and install flow

This release also includes a Debian packaging path for Ubuntu amd64.

The package flow covers:

a bundled executable under /usr/local/bin/heka-insights-agent
a systemd service unit
packaged runtime config under /etc/heka-insights-agent/.env
log output under /var/log/heka-insights-agent/agent.log
an interactive setup path for first-time installation

This is a meaningful step forward compared with a repo-only Python workflow, because it gives users a clearer route for installing the agent as a managed service.

5. Interactive setup flow

The packaged runtime includes a setup mode so installation is not just "copy files and figure it out yourself."

The setup flow is meant to help with:

selecting exporter mode
capturing required environment values
writing packaged runtime configuration
enabling and starting the service on success

That makes the first release much closer to something an operator can actually install and use.

6. Docs and test coverage around the exporter paths

We also shipped supporting documentation and test coverage for the exporter/runtime surface, including milestone-specific integration coverage for the Datadog and New Relic paths.

That does not mean the project is feature-complete. It means the base is being treated seriously.

Installation Direction

For release users, the install direction is now straightforward:

sudo dpkg -i heka-insights-agent_<version>_amd64.deb

If setup is interrupted, it can be resumed with:

sudo /usr/local/bin/heka-insights-agent setup

This first release is intentionally Linux-focused and packaging-aware from the start.

Why We’re Starting Here

There are already many observability tools in the market, but there is still room for agents that are:

simpler to understand
easier to extend
less coupled to a single backend
easier to package and operate on Linux hosts

The first release is about getting those fundamentals in place:

collection
normalization
delivery options
installability
operational clarity

Open Source Repo

If you want to inspect the code, follow the release progress, or contribute:

ronin1770/heka-insights-agent

What Comes Next

From here, the focus is to keep improving the agent in practical ways:

deeper collector coverage
stronger runtime hardening
broader integration coverage
more production-oriented examples
continued packaging and rollout improvements

The important part is that the first release is no longer just an idea. There is now a concrete repo, a real runtime, documented exporter behavior, and a packaged install path to build on.

Heka-Insights-Agent: Milestone 6 Complete: Datadog OTLP + Native Integration Paths

Farhan Munir — Sun, 17 May 2026 06:04:48 +0000

Milestone 6 is complete for the Heka Insights Agent.

Repository: https://github.com/ronin1770/heka-insights-agent

In this milestone, we implemented two Datadog-compatible delivery paths so teams can choose what fits their needs:

Datadog OTLP preset mode (EXPORTER_TYPE=datadog_otlp)
Datadog native API mode (EXPORTER_TYPE=datadog_native)

Milestone 6 Goal

Provide Datadog integration through both:

OTLP-based delivery for portability
Datadog-native delivery for backend-specific control

In common terms: this means your agent can now send metrics to Datadog in either a standards-oriented way (OTLP) or a Datadog-specific way (native API), without changing your collectors.

What We Implemented

1. Datadog OTLP preset mode (`datadog_otlp`)

We added a Datadog preset resolver that derives endpoint + auth headers from Datadog config.

endpoint is derived from site: https://otlp.<DATADOG_SITE>/v1/metrics
auth header is injected: dd-api-key: <DATADOG_API_KEY>
optional Datadog hostname/tags are mapped into resource attributes

2. Datadog native exporter (`datadog_native`)

We implemented a native Datadog metrics exporter for POST /api/v1/series.

endpoint is derived from site: https://api.<DATADOG_SITE>/api/v1/series
canonical gauge -> Datadog gauge
canonical counter -> Datadog count
timestamp_unix_ms -> Unix seconds
count.interval is derived from CPU_POLL_INTERVAL_SECONDS

3. Validation hardening (M6-3)

We fail fast on invalid/missing Datadog config:

DATADOG_SITE must be an allowed full domain
DATADOG_API_KEY must be non-empty
DATADOG_TAGS must be strict key:value format

4. Deterministic mapping rules (M6-4)

We made mapping predictable:

DATADOG_HOSTNAME overrides label-derived host
tag conflicts are deterministic (DATADOG_TAGS override label tags by key)
metric prefixing is idempotent (won't double-prefix)

5. OTLP vs native comparison docs (M6-5)

We added side-by-side docs so teams can quickly decide mode based on:

transport style
portability
mapping behavior
counter interval semantics

6. Docker-based milestone-6 integration tests (M6-6)

We added dedicated tests under:

tests/milestone-6/test_datadog_live_integration.py

These are intentionally gated and only run when explicitly enabled.

Key Files Added/Updated

src/config/runtime.py
src/config/__init__.py
src/exporters/factory.py
src/exporters/datadog_native.py
tests/test_config_otlp_env.py
tests/test_datadog_exporters.py
tests/milestone-6/test_datadog_live_integration.py
docs/configuration.md
docs/architecture.md
docs/development.md
README.md

Implementation Strategy

We followed the existing exporter architecture:

Keep collectors unchanged
Extend runtime config with strict validation and preset resolution
Reuse OTLP HTTP exporter for Datadog OTLP mode
Add a dedicated native exporter for Datadog API v1 series
Add deterministic mapping rules + comprehensive tests
Add Docker-backed live tests in a milestone-specific folder

Test Commands and Outputs

Local Datadog-focused test suite

PYTHONPATH=src python3 -m unittest -v \
  tests.test_config_otlp_env \
  tests.test_datadog_exporters

Example output:

Ran 21 tests in 0.008s

OK

Full local suite

PYTHONPATH=src python3 -m unittest discover -s tests -v

Example output:

Ran 42 tests in 0.014s

OK (skipped=1)

Docker live Datadog integration tests (milestone-6)

docker compose -f docker-compose.test.yml run --rm \
  -e RUN_OTLP_INTEGRATION=1 \
  -e RUN_DATADOG_LIVE_INTEGRATION=1 \
  -e DATADOG_SITE=us5.datadoghq.com \
  -e DATADOG_API_KEY=<REDACTED_DATADOG_API_KEY> \
  test-runner \
  pytest -vv -s -rs tests/milestone-6/test_datadog_live_integration.py

Observed output:

collected 2 items

tests/milestone-6/test_datadog_live_integration.py::DatadogLiveIntegrationTests::test_datadog_native_exports_gauge_and_count_metrics PASSED
tests/milestone-6/test_datadog_live_integration.py::DatadogLiveIntegrationTests::test_datadog_otlp_preset_exports_gauge_metric PASSED

2 passed in 2.63s

Why This Matters

With Milestone 6 complete, Datadog users can now choose the integration path that matches their operations model:

choose OTLP preset for standards-aligned portability
choose native mode for Datadog-specific control and semantics

Both paths are now validated, tested, and documented.

Contribute

If you're interested in observability agents, OTLP pipelines, or exporter design, contributions are welcome:

https://github.com/ronin1770/heka-insights-agent

Useful contribution areas:

additional Datadog live integration coverage
CI automation for gated integration scenarios
docs/examples for production deployments
future milestone reliability and operability improvements

Milestone 5 Complete: New Relic OTLP Integration for Heka Insights Agent

Farhan Munir — Tue, 12 May 2026 05:33:12 +0000

In this milestone, we focused on making New Relic integration first-class in the Heka Insights Agent, while still using the same OTLP HTTP exporter foundation.

Repository: https://github.com/ronin1770/heka-insights-agent

Milestone 5 Goal

Milestone 5 was about shipping a reliable, low-friction New Relic path without creating a separate proprietary exporter.

Scope covered:

New Relic preset configuration layer
Automatic New Relic auth header injection
Endpoint and required field validation
Documentation updates with practical examples
Integration tests for preset behavior

What We Changed

1. New Relic preset mode (`EXPORTER_TYPE=newrelic_otlp`)

We added a preset resolver that maps New Relic-specific environment variables into OTLP HTTP exporter inputs.

Required variables:

NEWRELIC_OTLP_ENDPOINT
NEWRELIC_API_KEY
NEWRELIC_SERVICE_NAME

Optional variables:

NEWRELIC_ENVIRONMENT
NEWRELIC_HOST_NAME

2. Automatic auth header injection

When EXPORTER_TYPE=newrelic_otlp is selected, the runtime now injects:

api-key: <NEWRELIC_API_KEY>

No manual OTLP_HTTP_HEADERS setup is required for baseline New Relic auth.

3. Precedence behavior

In preset mode, NEWRELIC_* values take precedence over conflicting generic OTLP_* values.

Examples:

NEWRELIC_API_KEY overrides any conflicting OTLP_HTTP_HEADERS api-key
NEWRELIC_SERVICE_NAME overrides service.name from OTLP_RESOURCE_ATTRIBUTES

4. Validation hardening

We now fail fast on invalid or missing required New Relic settings:

missing required keys -> startup error
invalid endpoint format -> startup error (must be absolute http:// or https://)

5. Documentation updates

We updated project docs and examples to include New Relic preset usage and expected test flows.

Commands to Run Tests

Unit tests for config behavior

PYTHONPATH=src python3 -m unittest -v tests.test_config_otlp_env

New Relic Docker integration tests (explicit)

docker compose -f docker-compose.test.yml run --rm \
  -e RUN_OTLP_INTEGRATION=1 \
  -e OTLP_IT_HOST=host.docker.internal \
  test-runner \
  pytest -vv -s -rs tests/milestone-5/test_newrelic_otlp_integration.py

Expected summary:

collected 3 items
...
3 passed

Full OTLP/HTTP Docker test stack

docker compose -f docker-compose.test.yml up --build --abort-on-container-exit --exit-code-from test-runner

Why This Matters

With Milestone 5 complete, teams can onboard New Relic using a predictable preset path:

less config ambiguity
safer startup behavior through validation
standardized OTLP transport path
stronger confidence through Docker-backed integration tests

Next steps will continue building on this exporter foundation while keeping the runtime predictable and vendor-friendly.

Milestone 4 Complete — OTLP HTTP Exporter for Heka Insights Agent

Farhan Munir — Wed, 29 Apr 2026 06:49:27 +0000

In this milestone, I completed the OTLP HTTP exporter implementation for Heka Insights Agent and validated it with both unit tests and Docker-backed integration tests.

Repo: https://github.com/ronin1770/heka-insights-agent

What was implemented in Milestone 4

Milestone 4 includes:

OTLP HTTP exporter wiring
OTLP metric payload mapping
OTLP auth header support
OTLP resource attribute mapping
Retry/backoff behavior for transient failures
Unit test coverage for config + sender + exporter + mapping
Docker integration tests against a real OpenTelemetry Collector

Configuration added

OTLP configuration is environment-driven through .env:

LOG_LOCATION=./log/heka_agent.log
CPU_POLL_INTERVAL_SECONDS=10
EXPORTER_TYPE=otlp_http
OTLP_HTTP_ENDPOINT=http://localhost:4318/v1/metrics
OTLP_HTTP_HEADERS=key=Bearer abcd1234
OTLP_RESOURCE_ATTRIBUTES=service.name=heka-insights-agent,host.name=localhost
OTLP_HTTP_TIMEOUT_SECONDS=10
OTLP_HTTP_RETRY_MAX_ATTEMPTS=5
OTLP_HTTP_RETRY_INITIAL_BACKOFF_SECONDS=1
OTLP_HTTP_RETRY_MAX_BACKOFF_SECONDS=5

Notes

OTLP_HTTP_HEADERS format: key=value,key2=value2
OTLP_RESOURCE_ATTRIBUTES format: key=value,key2=value2
Retryable failures:
- transport errors
- HTTP 408, 429, and 5xx
Non-retryable failures:
- HTTP 400, 401, 403, 404, and similar client-side errors

Unit tests

Unit tests cover:

OTLP env parsing and validation
OTLP payload mapping
OTLP HTTP sender behavior
Retry/backoff behavior
Exporter initialization and wiring

Run unit tests:

PYTHONPATH=src python3 -m unittest discover -s tests -v

Unit test output

Ran 27 tests in 0.008s

OK

Integration tests (Docker + real collector)

Integration tests validate:

auth success (200)
auth reject path (401) and no retry on non-retryable status
no-auth collector path

Run OTLP integration tests:

RUN_OTLP_INTEGRATION=1 PYTHONPATH=src python3 -m unittest -v tests.test_otlp_http_integration

Integration test output

test_auth_rejected_without_retry_for_401 ... ok
test_auth_success_with_bearer_header ... ok
test_no_auth_collector_accepts_without_headers ... ok

----------------------------------------------------------------------
Ran 3 tests in 2.417s

OK

Collector config used in tests

The project includes collector fixtures under tests/fixtures/otlp/, including auth-required and no-auth variants for scenario testing.

Final result

Milestone 4 now has:

working OTLP HTTP exporter
production-style config controls
retry/backoff logic for transient failures
unit + integration test coverage
updated docs and run instructions

Repo again: https://github.com/ronin1770/heka-insights-agent

Milestone 4 (Part 1): Implementing OTLP HTTP Core in Heka Insights Agent (M4-1, M4-2)

Farhan Munir — Mon, 27 Apr 2026 09:57:30 +0000

Milestone 4 (Part 1): Implementing OTLP HTTP Core in Heka Insights Agent (M4-1, M4-2)

Heka Insights Agent already had a canonical metrics pipeline from Milestone 3.

In this part of Milestone 4, I implemented the OTLP HTTP core in two focused steps:

M4-1: Canonical metrics -> OTLP payload mapping layer
M4-2: OTLP HTTP request sender and exporter wiring

This post covers only these two items. Auth headers, resource attributes, retry/compression controls are intentionally deferred to later M4 tasks.

Why This Split Matters

By separating mapping from transport, we get:

stable internal metric model
explicit OTLP payload construction
transport logic that can evolve independently
clean foundation for New Relic/Datadog-style OTLP integrations later

What Was Implemented

M4-1: OTLP Payload Mapping Layer

I added a dedicated mapper that converts canonical metric records into OTLP HTTP JSON payloads.

Core behavior:

validates required canonical fields before send
supports explicit type mapping:
gauge -> OTLP gauge.dataPoints
counter -> OTLP sum.dataPoints with cumulative temporality
maps canonical labels to OTLP metric attributes
maps timestamp_unix_ms to OTLP timeUnixNano
rejects malformed metrics early with explicit errors

Result: malformed payloads are blocked before network transport.

M4-2: OTLP HTTP Sender + Exporter

I added OTLP HTTP sender/exporter flow and wired it into exporter selection.

Core behavior:

EXPORTER_TYPE=otlp_http now creates OTLP exporter
validates OTLP endpoint format (http/https absolute URL) at startup
fails fast when endpoint is missing/invalid
sends JSON payload via HTTP POST
treats only 2xx responses as success
raises explicit errors for HTTP failures and transport errors

Result: working end-to-end OTLP HTTP delivery with fail-fast startup safety.

Local Test Setup with OpenTelemetry Collector (Docker)

I used OTel Collector debug exporter to validate incoming metrics.

Collector config (`otel-collector-config.yaml`)

receivers:
  otlp:
    protocols:
      http:
        endpoint: 0.0.0.0:4318

exporters:
  debug:

service:
  pipelines:
    metrics:
      receivers: [otlp]
      exporters: [debug]

Run collector

docker run --rm \
  -p 4318:4318 \
  -v "$(pwd)/otel-collector-config.yaml:/etc/otelcol/config.yaml" \
  otel/opentelemetry-collector:latest \
  --config=/etc/otelcol/config.yaml

Agent `.env` for this test

LOG_LOCATION=./log/heka_agent.log
CPU_POLL_INTERVAL_SECONDS=10
EXPORTER_TYPE=otlp_http
OTLP_HTTP_ENDPOINT=http://localhost:4318/v1/metrics

Run agent

python src/main.py

Verification Signals

From runtime behavior:

agent starts with exporter_type=otlp_http
collector logs periodic metric batches every ~10 seconds
no exporter exceptions during dispatch
first cycle has fewer points due to CPU warm-up, then normalizes

Example collector signal:

resource metrics: 1
metrics: 24
data points: 24

Tests Added

I added focused tests for M4-1/M4-2:

payload mapping correctness (gauge/counter, labels, timestamps)
validation failures for malformed canonical metrics
HTTP sender request behavior and error handling
exporter wiring and missing-endpoint startup failure

All tests pass:

PYTHONPATH=src python3 -m unittest discover -s tests -v

What Is Intentionally Not Included Yet

Deferred to later M4 items:

auth headers (M4-3)
resource attribute mapping (M4-4)
timeout/compression/retry controls (M4-5)
broader OTLP docs and expanded test matrix (M4-6, M4-7)

Closing

M4-1 and M4-2 establish the OTLP core path: canonical metrics are now mapped deterministically and sent over HTTP with fail-fast validation.

This gives a production-friendly base to layer auth, resource metadata, and resiliency controls next.

Repo URL: https://github.com/ronin1770/heka-insights-agent

Heka-Insights-Agent: Milestone M3-4: Fail-Fast Exporter Validation at Startup

Farhan Munir — Fri, 24 Apr 2026 06:30:06 +0000

Milestone M3-4: Fail-Fast Exporter Validation at Startup

On April 24, 2026, I completed M3-4 in heka-insights-agent:

M3-4: Add configuration validation for exporter settings

This milestone closed a critical gap in the exporter foundation by removing silent fallback behavior and enforcing explicit startup failures for invalid exporter configuration.

Context

In Milestone M3-3, we routed output through the exporter interface and lifecycle:

initialize()
export(metrics)
shutdown()

That established the architecture.

M3-4 focused on correctness and operational safety: startup must fail early when exporter configuration is invalid or points to an unimplemented adapter.

The Problem Before M3-4

Prior behavior was permissive:

invalid EXPORTER_TYPE values defaulted back to console
unimplemented exporter types also downgraded to console

Why this was risky:

misconfiguration could go unnoticed in production
users could think they were exporting to one backend while actually exporting to console
behavior violated milestone acceptance criteria requiring explicit errors

M3-4 Goals

The implementation targeted three concrete outcomes:

invalid EXPORTER_TYPE must fail fast with a clear startup error
configured but unimplemented exporter adapters must fail fast with a clear startup error
docs must reflect strict validation (remove pre-M3-4 fallback messaging)

Implementation Summary

1) Strict validation in runtime config

Updated src/config/runtime.py in get_exporter_type(...).

Behavior now:

missing EXPORTER_TYPE still defaults to console
unsupported value now raises RuntimeError with explicit supported values list
optional logger records an error message before raising

This changed exporter selection from “best-effort fallback” to deterministic validation.

2) Removed fallback from exporter factory

Updated src/exporters/factory.py in create_exporter(...).

Behavior now:

console returns ConsoleExporter
other configured values currently raise RuntimeError because adapters are not implemented yet

This is intentional. It prevents false confidence and makes readiness of each exporter explicit.

3) Documentation updated to M3-4 semantics

Updated:

docs/configuration.md
README.md

Both now state that:

unsupported exporter values fail fast
unimplemented exporters fail fast
only missing value defaults to console

This removed the “fallback to console with warning” wording from the pre-M3-4 behavior.

Validation Performed

Validation included compile checks and runtime behavior checks.

Compile validation

python3 -m compileall src passed

Behavior validation

Tested startup resolution paths for:

missing EXPORTER_TYPE
EXPORTER_TYPE=console
EXPORTER_TYPE=invalid_value
EXPORTER_TYPE=otlp_http (declared but not implemented adapter)

Observed results:

missing value -> resolves to console
console -> exporter creation succeeds
invalid_value -> immediate RuntimeError with supported-values message
otlp_http -> immediate RuntimeError stating exporter not implemented

These outcomes align with M3-4 requirements.

Why this change matters operationally

Fail-fast startup validation improves reliability in real deployments:

misconfigurations are caught immediately
no hidden routing to fallback output
deployment behavior is explicit and auditable
future exporter rollout can be gated by implementation readiness

This is especially important when teams automate deployments and rely on env-based configuration.

Architectural Impact

After M3-4, exporter behavior is now strictly layered:

config validates selector
factory enforces implementation availability
runtime starts only on valid+implemented exporter path

That gives us a clean foundation for adding real transports (otlp_http, datadog_native, newrelic_otlp) without changing collector logic.

What’s Next

The natural next step after M3-4 is M3-5:

document exporter lifecycle and responsibilities in architecture docs
include startup validation expectations as part of operational guidance
capture adapter implementation contract for future backend integrations

M3-4 makes sure the system fails loudly when exporter config is wrong, which is exactly what a transport foundation should do before adding real outbound integrations.

DEV Community: Farhan Munir

Configurable Video Transition Duration in Reel Quick

The problem

The feature

The FFmpeg math

Example 1

Example 2

Why validation matters

Implementation notes

Frontend

Backend

The key idea

Open source note

Implementing HTTP 429 Retry-After Handling in Heka Insights Agent

Why This Matters

The Goal of Issue #22

The Core Design Choice

Configuration Rules

Async Sleep Matters

Logging and Timing

Testing Scope

Why I Like This Issue

Read the Issue

Reel Quick: Added Video Transition Effects and a Theme Selector

Reel Quick: Added Video Transition Effects and a Theme Selector

What was added

Why this matters

Links

Demo assets

Adding RPM Packaging for CentOS 9 in Heka Insights

Supported RPM-based distributions

What was added

Why native RPM packaging matters

Consistent runtime model

Cleaner release artifacts

Installation on CentOS 9

A practical release improvement

I Added Scene Transitions to Reel Quick with FastAPI, Next.js, and FFmpeg

Feature Request: Theme Switching for Reel-Quick

Introducing Heka Insights Agent v0.1.0: Our First Release

What Heka Insights Agent Is

What Ships in v0.1.0

1. Linux host telemetry collection

2. Multiple exporter paths

3. Strict runtime configuration behavior

4. Debian package build and install flow

5. Interactive setup flow

6. Docs and test coverage around the exporter paths

Installation Direction

Why We’re Starting Here

Open Source Repo

What Comes Next

Heka-Insights-Agent: Milestone 6 Complete: Datadog OTLP + Native Integration Paths

Milestone 6 Goal

What We Implemented

1. Datadog OTLP preset mode (datadog_otlp)

2. Datadog native exporter (datadog_native)

3. Validation hardening (M6-3)

4. Deterministic mapping rules (M6-4)

5. OTLP vs native comparison docs (M6-5)

6. Docker-based milestone-6 integration tests (M6-6)

Key Files Added/Updated

Implementation Strategy

Test Commands and Outputs

Local Datadog-focused test suite

Full local suite

Docker live Datadog integration tests (milestone-6)

Why This Matters

Contribute

Milestone 5 Complete: New Relic OTLP Integration for Heka Insights Agent

Milestone 5 Goal

What We Changed

1. New Relic preset mode (EXPORTER_TYPE=newrelic_otlp)

2. Automatic auth header injection

3. Precedence behavior

4. Validation hardening

5. Documentation updates

Commands to Run Tests

Unit tests for config behavior

New Relic Docker integration tests (explicit)

1. Datadog OTLP preset mode (`datadog_otlp`)

2. Datadog native exporter (`datadog_native`)

1. New Relic preset mode (`EXPORTER_TYPE=newrelic_otlp`)

Collector config (`otel-collector-config.yaml`)

Agent `.env` for this test