DEV Community: Ran Cohen

Config Confessions: Horror Stories from the Trenches

Ran Cohen — Thu, 07 Mar 2024 14:20:00 +0000

In the shadowy corners of the development world, where the lines of code whisper secrets, there lie tales of terror that chill the very soul of even the most seasoned developers. These are the stories of configurations gone wrong, of nights turned into days by the glow of the monitor, and of deployments that turned into disasters. Welcome to "Config Confessions."

The Tale of the Vanishing Variables

It was a dark and stormy night when Aang, a developer at a thriving startup, decided to push the latest features to production. The code was clean, the tests passed, and everything looked set for a smooth deployment. But as the new version went live, services started failing left and right. The culprit? Environment variables that mysteriously vanished into the void. In the rush to deploy, Aang had overlooked the critical step of updating the production environment variables, leading to a cascade of failures. The team spent the night battling the shadows, restoring order to their digital realm.

Lesson Learned: Always double-check your environment configurations before deployment. And maybe, just maybe, avoid deploying on a dark and stormy night.

The Curse of the Hard-Coded Secrets

Katara, a seasoned developer, had always championed best practices. Yet, under the pressure of a tight deadline, she committed the cardinal sin of hard-coding a third-party API key directly into the configuration file. "It's just temporary," she told herself. But as fate would have it, the temporary fix became permanent, forgotten until the day the application suddenly lost access to the critical third-party service. The key had been compromised, and with it, the security of their application.

Lesson Learned: Temporary solutions often have a way of becoming permanent. Always externalize your secrets and use secure vaults or environment variables.

The Phantom Environment

There was once an environment variable, set by a developer named Sokka, meant to guide an application gently through the stages of development and testing. But a misstep in configuration cursed the application, making it behave as if possessed. In production, the application veered wildly, connecting to test databases and sending debug logs into the void. It took the team days to uncover the phantom variable responsible, hidden deep within the deployment scripts, a silent specter of their oversight.

Lesson Learned: Even the smallest of spirits, like environment variables, can possess great power over the fate of your applications. Treat them with care, and always double-check their bindings.

Conclusion

These tales from the trenches serve as cautionary reminders of the pitfalls that await when we neglect the sacred art of configuration management. Each story, while a horror in its own right, offers a lesson to ward off future nightmares.

We invite you to share your own stories of configuration challenges or successes. Your insights could be the guiding light for someone else's project. Drop your experiences in the comments below.

Stay tuned for the next chapter in our series, where we'll unveil even more horror stories, continuing our collective lessons through the complex world of software configuration.

So heed these confessions, dear reader, and may your deployments be smooth, and your configurations be ever in your favor.

Decoding the Matrix: The Evolution of Environment Variables in Software

Ran Cohen — Tue, 05 Mar 2024 14:20:00 +0000

Dive deep into the heart of the software world's matrix, where a powerful yet often overlooked tool resides—environment variables. These key-value pairs do more than just configure software applications; they carry the secrets of the past, present, and future of software development. From their inception to becoming a cornerstone of modern coding practices, environment variables have shaped the way we build, deploy, and manage applications. Join us on a fascinating journey through time, exploring the origins and evolution of environment variables.

The Dawn of Environment Variables

The story begins in the early days of computing, long before the graphical user interfaces and sophisticated development environments we take for granted today. It was the era of mainframe computers and command-line interfaces, where the need for flexible, configurable software first became apparent. The concept of environment variables is believed to have originated with the Multics operating system in the 1960s, a project that laid the groundwork for many aspects of modern computing.

Multics introduced the idea of a "context" for each user session, allowing for personalized settings and configurations. This innovation paved the way for the UNIX operating system, developed in the early 1970s, to adopt and expand the concept of environment variables. UNIX used these variables to store user preferences, system settings, and to control the behavior of shell scripts and applications, setting a standard that persists in operating systems to this day.

From UNIX to Ubiquity

The adoption of environment variables in UNIX marked the beginning of their journey to becoming a ubiquitous tool in software development. As UNIX-inspired operating systems like Linux took shape, and as the world gradually shifted from mainframe to personal computing, environment variables cemented their place in the software developer's toolkit.

One of the most widely recognized environment variables, PATH, illustrates the utility and enduring relevance of environment variables. PATH tells the operating system where to look for the executables it needs to run commands and scripts. It's a prime example of how environment variables make software configuration dynamic and portable across different systems and setups.

Environment Variables Today

Fast forward to the present, and environment variables are integral to the development, deployment, and operation of software across platforms and environments. They're instrumental in cloud computing, where services like AWS, Azure, and Google Cloud Platform rely on environment variables to manage configurations for scalable, distributed applications. Environment variables enable the secure management and injection of sensitive information, such as database credentials (DB_USERNAME, DB_PASSWORD) and API keys (API_KEY), into applications without hardcoding them into source code, ensuring applications remain both agile and secure.

Expanding the Toolbox: Tips and Strategies:

Now, let's pivot to how we can apply this understanding to enhance our current work with environment variables, bridging historical insights with practical strategies for today's development challenges.

Security:
Leverage environment variables to store sensitive data like DB_PASSWORD outside your codebase, enhancing security. Integrating with secret management tools like HashiCorp Vault or AWS Secrets Manager adds an extra layer of security, ensuring that secrets are encrypted and accessible only to authorized services.

Documentation:
Maintaining a comprehensive, up-to-date list of environment variables and their purposes aids in onboarding new developers and ensuring operational clarity. For instance, documenting the use and expected values for DATABASE_URL or REDIS_CACHE_ADDRESS can prevent configuration errors and streamline development processes.

Portability:
Environment variables like DATABASE_URL enable you to abstract environment-specific configurations from your application code, making your projects portable and adaptable across different environments. This practice allows for seamless transitions from development through to production without the need for code changes.

Simplify Local Development:
Utilize environment variables to manage different configurations for local development, testing, and production. Tools like Docker and docker-compose can use .env files to set up containers with the appropriate environment variables, making it easy to replicate settings locally or switch between different configurations without changing the application code.

Dynamic Configuration for Scalability:
In cloud-native architectures, environment variables can dynamically adjust application behavior based on the deployment context. For example, MAX_WORKERS or CACHE_TTL can be tweaked to optimize resource utilization and response times as the load varies. This dynamic configurability is essential for applications that need to scale up or down efficiently in response to real-time demand.

Environment-Specific Feature Flags:
Use environment variables as feature flags to enable or disable features in different environments. This approach allows for safe testing of new features in development or staging environments before they are enabled in production. For instance, setting FEATURE_X_ENABLED=true in a development environment variable can activate a new feature for testing, while the same feature remains inactive in production until ready for release.

Consistency Across Services:
In microservices architectures, maintaining consistency in environment variable naming and usage across services can greatly simplify configuration management and debugging. Establishing a convention for naming environment variables (e.g., prefixing with SERVICE_NAME_) ensures clarity and reduces the risk of conflicts or confusion when multiple services are configured together.

Fallback Strategies:
Design your applications to use fallback values for critical environment variables, ensuring that your application can function even if some configurations are missing. For example, defaulting to a standard logging level if LOG_LEVEL is not set.

Auditing and Rotation:
Regular audits of your environment variables help identify obsolete or unused variables, reducing clutter and potential security risks. Rotating sensitive values periodically minimizes the impact of potential exposure.

Your Turn:
Discovered a unique approach or handy tip with environment variables? The comments are your stage. Share your insights to inspire and contribute to our collective knowledge pool. Together, we can uncover new perspectives and elevate our practices. Join the conversation and let's grow smarter, together.

Conclusion

The evolution of environment variables from a nascent concept in the Multics operating system to a fundamental part of modern software infrastructure is a testament to their versatility and enduring importance. They remind us that sometimes, the most influential tools in software development are those that quietly work behind the scenes, shaping our digital world without fanfare. By incorporating the strategies and insights shared, developers can fully leverage environment variables to build applications that are secure, portable, and maintainable. As we look to the future, the role of environment variables is sure to evolve, but their legacy as a cornerstone of software configuration is undeniably secure.

Exploring Pkl: Apple's Fresh Approach to Configuration Languages

Ran Cohen — Thu, 29 Feb 2024 14:20:00 +0000

In a digital epoch where the only constant is change, Apple introduces Pkl—pronounced "Pickle"-a new entrant in the dynamic landscape of software development. With an eye towards addressing some of the longstanding issues in configuration management, Pkl trying to bring forward concepts of programmability, scalability, and safety. But beyond the initial buzz, what does Pkl truly offer to the modern developer?

Pkl at a Glance

Born from the need to transcend the limitations of static configuration files, Pkl stands as Apple’s innovative foray into programmable configuration management. It's not just another language; it's a paradigm shift towards configuration-as-code (CaC).

Configuration-as-Code: Automating Application Configuration

Ran Cohen ・ Jun 14 '23

#devops #programming #opensource #configuration

Key Features

Programmability: Pkl introduces conditions, loops, and functions within configuration files, transforming them from static documents into dynamic scripts.
Scalability: Tailored for projects of any size, Pkl's design ensures configurations remain manageable, regardless of the project's complexity.
Enhanced IDE Support: With auto-complete, error highlighting, and inline documentation, Pkl is designed to make configuration management a more integrated and less error-prone part of the development process.

A Simple Pkl Example

Let's dive into a simple yet illustrative example of Pkl in action. Imagine you're setting up the configuration for a web application. With Pkl, you can easily define your application's settings, including environment-specific variables and even incorporate logic to dynamically adjust settings based on the deployment context.



// Define a basic web application configuration
module WebAppConfig {
  hostname: String,
  port: Int,
  environment: "development" | "staging" | "production",

  // Dynamically adjust debug mode based on the environment
  debugMode: Bool = (environment == "development")
}

// Application instance for development
instance devConfig: WebAppConfig {
  hostname = "localhost",
  port = 8080,
  environment = "development"
}

This snippet demonstrates Pkl’s capability to elegantly tailor configurations to different environments, a testament to its programmable nature and practical utility.

Community Reception and Comparisons

Since its unveiling, Pkl has stirred a mix of excitement and skepticism within the developer community. Platforms like Hacker News and Reddit have become arenas of debate, weighing Pkl’s potential against the backdrop of existing solutions. While some applaud Pkl for its innovative approach, others question the necessity of introducing yet another player into the configuration language game. This discourse highlights the diverse needs and preferences within the software development community, underscoring the importance of choice in tools and methodologies.

Looking Forward

Pkl's debut is not just about a new tool; it's a conversation starter on the future of configuration management. Its adoption and the community's feedback will shape the role it plays in how we manage and deploy software in an increasingly complex world.

Useful Resources

Configu: Unleashing the Power of Configuration-as-Code

Ran Cohen — Wed, 28 Jun 2023 19:17:54 +0000

Welcome back, fellow tech enthusiasts! In this thrilling sequel, we embark on a captivating exploration of Configu—an open-source implementation of Configuration as Code (CaC) for application configuration management. Prepare yourselves for a wild ride as we uncover how Configu tackles the challenges of handling complex configurations, revolutionizes development workflows, and becomes the superhero of configuration management. So hold on tight and brace yourself for another exciting tech adventure!

Background: Unveiling Application Configurations

Before we delve into the magic of Configu, let's take a moment to revisit the importance of application configurations. In the software development lifecycle, configurations—those precious bits of metadata—are extracted from code and managed separately to customize application behavior across different deployments and environments. Configurations are the secret sauce that adds flavor and adaptability to our applications. Configu recognizes their significance and sets out to streamline their management and collaboration, putting the power back in the hands of developers.

The Motivation behind Configu: Addressing the Configuration Struggle

As our codebase grows, so does the complexity of our configuration needs. Yet, managing configurations in today's reality can be a formidable challenge. Configuration data is scattered across various systems, files, databases, and even third-party systems, often resembling a confounding labyrinth. This fragmented approach leads to bottlenecks, manual errors, and missed opportunities for automation and collaboration. Outdated or incorrect configurations can result in bugs, downtime, and even worse—an angry horde of users storming your virtual gates! Configu rides in on a mission to simplify configuration management, ensuring stability, reliability, and empowering development teams to conquer the configuration chaos.

Configu Main Concepts: Unraveling the Configuration Flow

Now, let's unravel the inner workings of Configu and shed light on its key concepts, seamlessly intertwined to optimize the configuration flow:

Config: The Genetic Blueprint:

At the core of Configu lies the Config—a fundamental data model that defines the essence of your application's behavior. It's like the DNA strand that shapes and guides your software. Each Config carries a unique identifier (key), a context (set), and the actual configuration value. Think of it as your application's genetic blueprint, ready to be brought to life.

ConfigStore: Versatile Storage Engine:

Configs find their home in the ConfigStore—a dynamic storage engine that adapts to different systems and services. Whether it's a database, file, secret manager, or even a custom-made solution, the ConfigStore stands tall as the fortress that guards your configurations. It offers flexibility, scalability, and enough magic to make even the most eccentric configurations feel right at home.

ConfigSchema: Declaring the Genetic Code:

The ConfigSchema plays a crucial role in Configu's configuration orchestration. This special file, adorned with the .cfgu extension, declares the genetic code of your application. It establishes the bindings between Config keys and their respective Cfgu declarations, specifying characteristics, and constraints for each value. With ConfigSchema, your configurations adhere to a predefined structure, ensuring harmony and consistency in the configuration universe.

.cfgu.json

{
  "LOG_LEVEL": {
    "type": "RegEx",
    "pattern": "^(debug|info|warn|error)$",
    "default": "info",
    "required": true,
    "description": "Represents the severity of the log events passed to the stdout"
  }
}

Configu Interfaces: Orchestrating the Configuration Symphony:

Configu interfaces are the maestros orchestrating the symphony of configurations. Through unified APIs like the Configu SDKs and the Configu CLI, developers wield the power to effortlessly interact with configuration data. Fetch, set, transform, and inject configurations with the grace of a seasoned conductor. These interfaces bridge the gap between developers and the ConfigStore, granting them full command over their application's behavior.

Storing Configs: Activating the ConfigStore:

When Configs are stored, the Configu interfaces connect to the ConfigStore and unleash its mighty set method. Configs find solace in the ConfigStore's memory, snug and secure, awaiting their call to action. This meticulous process ensures your configurations are safely stored, ready to rise when the time is right.

configu upsert --store 'configu' --set 'development' --schema './.cfgu.json' -c 'LOG_LEVEL=debug'
configu upsert --store 'configu' --set 'production' --schema './.cfgu.json' -c 'LOG_LEVEL=error'
configu upsert --store 'configu' --set 'production/tenant' --schema './.cfgu.json' -c 'LOG_LEVEL=warn'

Fetching Configs: Unleashing the Configuration Magic:

During build, deployment, or runtime, Configu interfaces spring into action, summoning configurations from the ConfigStore. By activating the get method, relevant Configs are retrieved based on context and schema, like a magical conjuration tailored to your needs. These configurations can then be transformed into powerful artifacts or injected seamlessly into your software's processes. Behold the true magic of Configu—the ability to harness the perfect configuration spells at the right time and place.

configu eval --store 'configu' --set 'development' --schema './.cfgu.json' | configu export --run 'node server.js'
configu eval --store 'configu' --set 'production' --schema './.cfgu.json' | configu export --format 'HelmValues' > ./values.yaml

Bulk Fetching: Streamlining the Configuration Flow:

To streamline the process and make configuration retrieval a breeze, Configu offers the option of bulk fetching. It's like summoning an army of configurations to fulfill your application's needs, all with a flick of a wand. No more painstakingly fetching one configuration at a time. With bulk fetching, unleash the full power of Configu and let it rain configurations upon your software.

configu eval ... | configu eval ... | configu export ...

Configu's "How": Implementing the Features and Principles of CaC

Now that we've untangled the configuration flow, let's witness how Configu embraces the core features and principles of Configuration as Code:

Consistency and Reproducibility:

Configu ensures consistency and reproducibility by treating configurations as code. Configs, residing in the ConfigStore, retain their context and value, guaranteeing consistent behavior across different environments. Say goodbye to unpredictable configurations and hello to harmony and stability!

Versioning and Tracking:

With Configu, bid farewell to configuration confusion. By including the ConfigSchema in your version control system alongside your code, changes to configurations are recorded and tracked. This historical record empowers you to roll back to previous versions with ease. It's like having a time machine for your configurations, traveling back to any desired state of your application.

Automation and Efficiency:

Configu embraces automation and efficiency with open arms. Configu interfaces allow developers to automate configuration processes. Whether you're using the Configu CLI during build and deployment or the Configu SDK at runtime, fetching, transforming, and injecting configurations becomes a breeze. Say goodbye to manual effort and hello to automated brilliance. Let Configu handle the heavy lifting while you sit back and enjoy the show!

Scalability and Agility:

Configu's flexible architecture and hierarchical ConfigSets empower you to scale and adapt effortlessly. No limits, no boundaries—organize your .cfgu files to suit your needs and workflow. Spread them across different codebases, subfolders, or even repositories. Fine-tune configurations for various contexts, environments, and customer needs. Configu bends and flexes like a nimble gymnast, adapting to your software's demands effortlessly.

Collaboration and Teamwork:

Configu fosters collaboration among developers, DevOps, and stakeholders. Through unified Configu APIs and ConfigSchema, everyone gains a shared understanding of configurations. Effective communication and teamwork become second nature. Configu becomes the conductor that brings harmony to your team's configuration symphony. Let the collaborative magic unfold!

Auditability and Security:

Configu ensures auditability and security, standing guard over your configurations. Versioned records of configuration changes provide accountability and a sense of security. ConfigSchema acts as a vigilant guardian, enforcing compliance and maintaining the integrity of your configurations. Sleep well at night, knowing Configu has your back.

Simplifies Configuration Management:

Configu simplifies configuration management by centralizing configurations in the ConfigStore. Modify, organize, and track configurations with ease, reducing complexity and improving maintainability. Configu is the organizational wizard that tames the wild configuration chaos, restoring order to the realm.

Eliminates Manual Errors:

With Configu treating configurations as code, the risk of manual errors diminishes. Configuration values are specified and managed consistently, drastically reducing the chances of misconfigurations and boosting reliability. Say goodbye to those pesky manual errors and greet a world of stable configurations.

Enhances Efficiency and Productivity:

Configu's automation capabilities unlock a new level of efficiency and productivity. By streamlining configuration processes, Configu frees up developers' time to focus on their true superpower—coding! Configu becomes your trusted sidekick on the path to success, empowering you to achieve more in less time.

configu / configu

Open-source ConfigOps infrastructure ⚙️

Open-source ConfigOps infrastructure that empowers engineering teams to seamlessly manage and collaborate on software configuration data—such as environment variables, secrets, and feature flags—across any system or environment. ⚙️✨

Get Started 👋 ・ Explore Docs 📚 ・ Join Chat 💬

⭐️ Configu?

Configu¹ is a simple², modern³, and secure⁴ standard for managing and collaborating on software configuration data. It is built to provide a common⁵, flexible⁶, and extensible⁷ process for storing, mutating, and orchestrating configuration data across any system or environment.

Configu is designed to be easy to integrate into existing systems and workflows, and to be adaptable to the diverse needs of teams and projects. It can be used from the command line, integrated in a build pipeline, or embedded in a program. Configu scales from small to large, simple to complex, ad-hoc to repetitive configuration…

View on GitHub

Conclusion: Configu's Mission to Simplify Configuration Management

As we bid farewell to this enthralling sequel of our Configu adventure, we hope you've witnessed the true power of Configuration as Code through Configu's lens. Configu, with its unwavering focus on configuration consistency, automation, collaboration, and security, emerges as the game-changer in the realm of configuration management. Our mission at Configu is crystal clear—to revolutionize the way software configurations are managed and collaborated upon. By empowering developers to focus on coding rather than configuration struggles, Configu paves the way for a future where configurations are effortless, reliable, and secure.

Thank you for joining us on this enlightening tech adventure. Stay tuned for more exciting journeys with Configu, and until next time, happy coding!

Configuration-as-Code: Automating Application Configuration

Ran Cohen — Wed, 14 Jun 2023 14:42:40 +0000

Welcome to the world of Configuration-as-Code (CaC), where automation reigns supreme, and even the most tedious tasks can be transformed into a coding extravaganza. In this post, we'll dive into the fascinating realm of "as-code" buzzwords, take a brief journey through the evolution of automation, and explore how CaC revolutionizes application configuration. So buckle up and get ready for a tech adventure!

The Birth of "As-Code" Buzzwords

Picture this: developers and IT operations/sysadmins joining forces, uniting their powers to conquer the land of automation. It all started with the DevOps movement and the emergence of "infrastructure-as-code". Devs and sysadmins realized that reusable code and version control could automate environment modifications. While developers had already been automating their software delivery processes, sysadmins were catching up. It was like a race to the automation finish line, where developers cheered on the sysadmins to join the party and make everyone's lives easier.

Everything-as-Code: Mapping the Automation Frontier

Now let's take a moment to marvel at the wonders of automation in the software development lifecycle (SDLC). From development to production, there's a parade of "*-as-code" automation marching through the stages. It's like a grand carnival, where automation floats dazzle the onlookers with their charm and efficiency. But amidst this spectacle, we wonder: are there areas left unexplored? Are there opportunities for novel automation? Let's unravel the secrets and find out where "everything-as-code" resides.

For further exploration, I highly recommend diving into a fascinating blog post by Patrick Debois, the visionary who coined the term "DevOps." In his blog, he delves into an in-depth analysis of trends across over 50 different "as code" concepts. Prepare to expand your knowledge and uncover valuable insights that will fuel your automation journey.

Unveiling Application Configuration

Ah, application configuration! It's the chameleon of software deployment, adapting to various environments like a master of disguise. An app's configuration encompasses everything that varies between deployments: resource handles, credentials to external services, and per-deploy values. It's a crucial part of the software puzzle, ensuring the application behaves consistently across different settings.

Enter Configuration-as-Code (CaC)

Ladies and gentlemen, meet the concept of Configuration-as-Code (CaC), the star of our adventure! Building upon the success of Infrastructure-as-Code (IaC), CaC takes the stage, treating application configuration as code. But what does CaC bring to the table, you ask? Well, hold on tight, because we're about to embark on a whirlwind tour of its captivating features:

Consistency and Reproducibility:

With CaC, application configurations are declared and managed using code, ensuring consistency and reproducibility across different environments.

Versioning and Tracking:

CaC allows us to track changes made to application configurations over time. Just like in IaC, it can version, track, and even perform rollbacks if needed.

Automation and Efficiency:

Say goodbye to manual configuration headaches! CaC automates the process by replacing manual steps with code-based configurations. Faster and more efficient deployments are within reach!

Scalability and Agility:

Like a master acrobat, CaC gracefully handles changing requirements. It offers the flexibility to adjust and scale configurations, ensuring the application keeps up with the ever-evolving demands of the audience.

Collaboration and Teamwork:

CaC fosters a harmonious symphony of collaboration among developers, operations teams, and stakeholders. With the power of code, they can work together, share knowledge, and align their efforts to create a breathtaking software masterpiece.

Auditability and Security:

Just like IaC, CaC keeps security buffs satisfied. Changes made to configuration code can be reviewed, audited, and enforced. Compliance and security practices can rest easy knowing that the configurations are in good hands.

Simplifies Configuration Management:

CaC brings order to the configuration chaos by treating configurations as code. It provides a structured approach, making it easier to understand, modify, and evolve configurations over time. Goodbye, tangled mess of manual configurations!

Eliminates Manual Errors:

CaC is the superhero that swoops in to rescue us from the clutches of human error. Code-based configurations reduce manual mistakes, resulting in more reliable and consistent application setups.

Enhances Efficiency and Productivity:

With CaC in your toolkit, efficiency and productivity reach new heights. It automates configuration processes, reduces manual efforts, and frees up valuable time for teams to focus on delivering top-notch applications and achieving business goals.

Introducing Configu: an OSS Implementation of CaC for Application Configuration

Based on the incredible benefits of CaC, there's an open-source project that makes the implementation even smoother. Allow us to introduce Configu! Configu is a simple, modern, and generic standard for managing and collaborating software configurations. It is built to provide a common, flexible, and extensible process for storing, mutating, and orchestrating configuration data across different environments and systems. With Configu, you can leverage the power of CaC and ensure streamlined and efficient configuration management for your applications. You can see how our team implemented it in this GitHub repo.

configu / configu

Open-source ConfigOps infrastructure ⚙️

Get Started 👋 ・ Explore Docs 📚 ・ Join Chat 💬

⭐️ Configu?

View on GitHub

Conclusion

And there you have it, folks! Configuration-as-Code (CaC) takes the stage, uniting the powers of automation and code to revolutionize application configuration. It brings consistency, scalability, collaboration, and security to the world of configurations. Say goodbye to manual headaches, configuration chaos, and tedious setups. Embrace CaC, along with the Configu Orchestrator, and join the grand symphony of automation in the software development journey.

So let the code be with you, and may your configurations always be harmonious and error-free!

Thank you for joining us on this tech adventure. Until next time, happy coding!

MongoDB composed

Ran Cohen — Tue, 13 Jun 2023 00:53:11 +0000

Introduction:

This post provides a ready-to-use Docker Compose based configuration that allows you to effortlessly set up a MongoDB single node replica set with a reliable healthcheck and a Web GUI of mongo-express.

Prerequisites

Configuration

`docker-compose.yml`

version: '3.9'

services:
  mongo:
    container_name: mongo
    image: mongo:${MONGO_VERSION} # https://hub.docker.com/_/mongo
    entrypoint:
      - bash
      - -c
      - |
        if [ ! -f ${MONGO_REPLICA_SET_KEY_FILE} ]; then
          echo "generating replica set key file ..."
          openssl rand -base64 756 > ${MONGO_REPLICA_SET_KEY_FILE}
          chmod 600 ${MONGO_REPLICA_SET_KEY_FILE}
          chown 999:999 ${MONGO_REPLICA_SET_KEY_FILE}
        fi
        exec docker-entrypoint.sh $$@
    command: "--bind_ip_all --replSet ${MONGO_REPLICA_SET_NAME} --keyFile ${MONGO_REPLICA_SET_KEY_FILE}"
    environment:
      MONGO_INITDB_ROOT_USERNAME: ${MONGO_USERNAME}
      MONGO_INITDB_ROOT_PASSWORD: ${MONGO_PASSWORD}
    ports:
      - 27017:27017
    healthcheck:
      test: test $$(echo "rs.initiate().ok || rs.status().ok" | mongo -u $${MONGO_INITDB_ROOT_USERNAME} -p $${MONGO_INITDB_ROOT_PASSWORD} --quiet) -eq 1
      start_period: 30s
    restart: unless-stopped

  mongo-gui:
    container_name: mongo-gui
    image: mongo-express:${MONGO_GUI_VERSION} # https://hub.docker.com/_/mongo-express
    environment:
      ME_CONFIG_MONGODB_ADMINUSERNAME: ${MONGO_USERNAME}
      ME_CONFIG_MONGODB_ADMINPASSWORD: ${MONGO_PASSWORD}
      ME_CONFIG_MONGODB_SERVER: mongo
      ME_CONFIG_MONGODB_ENABLE_ADMIN: true
    ports:
      - 8081:8081
    depends_on:
      mongo:
        condition: service_healthy
    restart: unless-stopped

`.env`

MONGO_VERSION=5.0.18 # pick your version from https://hub.docker.com/_/mongo/tags
MONGO_USERNAME=root
MONGO_PASSWORD=root
MONGO_REPLICA_SET_NAME=rs0
MONGO_REPLICA_SET_KEY_FILE=/opt/rs.key
MONGO_GUI_VERSION=0.54.0 # change to "latest" after a stable release, currently on 1.0.0-alpha.X

connection string

# directConnection=true is a must for the local topology - see https://www.mongodb.com/docs/drivers/node/current/fundamentals/connection/connect/#direct-connection, https://jira.mongodb.org/browse/NODE-2966
MONGODB_URL=mongodb://root:root@localhost:27017?replicaSet=rs0&directConnection=true

commands

# start
docker compose --project-name composed-mongodb up --detach

# stop
docker compose --project-name composed-mongodb stop

# clean
docker compose --project-name composed-mongodb down --volumes --remove-orphans

verify

Open the browser and navigate to http://localhost:8081/, you should see the following:

Resources:

macOS awesome terminal

Ran Cohen — Fri, 28 Jan 2022 16:20:14 +0000

Hello developers 🧑‍💻, thanks for reading my post 🙏.

A shell is a user interface for access to an operating system’s services. Most often the user interacts with the shell using a command-line interface (CLI). The terminal 💻 is a program that opens a graphical window and lets you interact with the shell.

As developers, one of the programs we use most is our terminal, and as such it should allow us to be as productive and fast as possible 🚀 as well as to show off its features and pretty look 😎.

So let's embark on a journey 🧭 to make our terminal in macOS no less awesome 🙌.

iTerm2

Initially, we will replace the preinstalled terminal for iTerm2, which has tons of very useful features.

Go to iTerm2 download page and downloaded the stable release.
After the download is complete, drag the file to the Applications folder.
Bonus: Place iTerm2 in a convenient place in the dock (replace the original terminal if you already have it there).

iTerm2 has a standalone update mechanism so I prefer to install it manually rather than through a brew cask.

Oh My Zsh

Next, let's unleash our terminal by installing Oh My Zsh.

Oh My Zsh is an open source, community-driven framework for managing your Zsh configuration.

Oh My Zsh comes bundled with plugins that'll make your life as a developer easier and leave you feeling like a superhero.

To enable a plugin add its name to the plugins array in your .zshrc file (found in the $HOME directory).
Here are some that I highly recommend:

plugins=(alias-finder brew common-aliases copypath copyfile docker encode64 git history jsontools macos web-search z)

When installing Zsh's theme and plugins, opt for oh-my-zsh installation (... which is actually a git clone of the plugin's repo into the $ZSH_CUSTOM folder and a setting change in .zshrc). It will play best with the other settings in .zshrc, and the autoupdate plugin will keep them up-to-date.

Zsh Theme

My choice of theme for Zsh is Powerlevel10k. Beyond being fast and minimal, it has maintained its popularity for many years and continues to improve with each new version. Also, I'm totally in favor of installing the recommended font that comes with it.

Zsh plugins

autoupdate-zsh-plugin: Auto update git-repositories in $ZSH_CUSTOM folder. This plugin is crucial because it keeps the rest of the plugins up-to-date.
zsh-syntax-highlighting: It enables highlighting of commands while they’re typed. This helps in reviewing commands before running them, particularly in catching syntax errors.
zsh-autosuggestions: It suggests commands as you type based on your history and completions.
zsh-history-substring-search: Type in any part of any command from your history, and then press chosen keys, such as the UP and DOWN arrows, to cycle through matches.
zsh-nvm: Zsh plugin for installing, updating and loading nvm. nvm installation is relevant for developers working with node.js. This plugin is recommended since nvm via brew isn't supported.

iTerm2 Elevation

To open the preferences at any stage, Open iTerm2 → Press (command) ⌘ + , (comma).

Shell Integration

Enabling the shell integration feature enables many useful features by itself.
Follow the official documentation instructions.
In practice the steps should look something like this:

Key Mappings

Setting these key mappings allows us to use common keyboard shortcuts in our terminal.

Follow the instructions in the gist I made for this purpose, and if you found it helpful, I'd appreciate you giving it a star as well.

Hotkey

Setting a hotkey allows us to show or hide iterm window even if another application is active.

Preferences → Keys → Hotkey → ☑️ Show/hide all windows with a system-wide hotkey
Click on the rectangle that appears right to "Hotkey:" → Set your preferred key sequence

Status Bar

Setting up the status bar allows us to get up-to-date information about the environment we're working in.

Preferences → Profiles → Session → ☑️ Status bar enabled
Click Configure Status Bar button to begin setting up your status bar configuration. My active components are: CPU Utilization, Memory Utilization and Network Throughput

Remove the `last login: ...` line on top of each new window

cd ~; touch .hushlogin

Allow iTerm to access administrative files

System Preferences → Security & Privacy → Full Disk Access → ☑️ iTerm.app

Use TouchID for sudo execution

Open /etc/pam.d/sudo with a text editor as admin

sudo vim /etc/pam.d/sudo

add auth sufficient pam_tid.so line on top and save
Preferences → Advanced → Allow sessions to survive logging out and back → Select "No"

Set some "nice to have" preferences

General → Closing → ☑️ Quit when all windows are closed
General → Selection → ☑️ Double-click performs smart selection
General → Window → ☑️ Smart window placement
Appearance → General → Theme → Select "Minimal"
Appearance → Tabs → ☑️ Show tab bar even when there is only one tab
Profiles → General → Icon (Basics) → Select "Built-in Icon for Current App"
Profiles → General → Working Directory → 🔘 Advanced Configuration → Edit... → Working Directory for New Split Panes → 🔘 Reuse previous session's directory
Profiles → Text → Cursor → 🔘 Vertical bar + ☑️ Blinking cursor
Profiles → Text → Font → Increase size (I use 16 as in my IDE)
Profiles → Terminal → Shell Integration → ☐ Show mark indicators
Keys → Navigation Shortcuts → Shortcut to choose a split pane → Select your preferred option (I use ⌥ Number)
Pointer → General → ☑️ Focus follows mouse
Advanced → Mouse → Scroll wheel sends arrow keys when in alternate screen mode → Select "Yes"

Add a screensaver

CMatrix shows text flying in and out in a terminal like as seen in "The Matrix" movie.

brew install cmatrix
cmatrix

pipes.sh shows animated pipes.

brew install pipes-sh
pipes-sh

Install useful command-line tools

agarrharr / awesome-cli-apps

🖥 📊 🕹 🛠 A curated list of command line apps

macOS developer

Ran Cohen ・ Jan 28 '22

#tutorial #productivity #macos #developer

The Result ✨

configuration files (.zshrc, iterm-profile.json etc..) as a gist

macOS developer

Ran Cohen — Fri, 28 Jan 2022 16:18:01 +0000

Hello developers 🧑‍💻, thanks for reading my post 🙏.

Recently I got to spend quite a bit of time finding settings, tools and apps that boost my productivity 🚀 and make my development routine easier ✨.

Note that all of the following are just my personal warm recommendations, and I'd like to hear about any additional tools I may have missed in the discussion section 🎤.

The following preferences ⚙️ allow me to utilize macOS more effectively

System Preferences 💻

Dock & Menu Bar → ☑️ Automatically hide and show the Dock
Dock & Menu Bar → Other Modules → Battery → ☑️ Show Percentage
Accessibility → Pointer Control → Mouse & Trackpad → Trackpad Options… → ️️☑️ Enable dragging
Trackpad → Point & Click → ☑️ Tap to click
Trackpad → Point & Click → Increase Tracking speed
Battery → Power Adapter → ☐ Wake for network access

Finder Preferences 🗂

Preferences → Advanced → ☑️ Show all filename extensions
View → Show Path Bar
View → Show Status Bar
Press shift ⇧ + command ⌘ + . (Show hidden files)

The following tools and applications are free 🆓 and allow me to significantly shorten daily operations

Command-line Tools 🛠

Xcode Command Line Tools is a small self-contained package available for download separately from Xcode and that allows you to do command line development in macOS.
Some of the tools below depend on it, so it is important to install it first.
Press command ⌘ + Space → Type terminal → Run xcode-select --install → Wait few minutes
Homebrew is an open-source software package management system that simplifies the installation of software on macOS.
Open Homebrew page in a new tab → Copy the snippet → Paste it in the terminal → Follow the instructions that the script will display

By installing the rest of the command-line tools via brew, you will be able to keep them up-to-date by running brew upgrade once in a while.

brew install git curl wget jq htop speedtest tldr thefuck fzf bat

git version control system
curl transferring data with URLs
wget retrieving files
jq JSON processor
htop process viewer
speedtest internet connection measurement
tldr commands cheatsheets
thefuck corrects errors in previous commands
fzf fuzzy finder
bat cat with syntax highlighting and Git integration

Applications 📱

Since most apps these days can update themselves automatically, I prefer to install them via their native installer (download then drag to application directory, and if needed remove with AppCleaner) rather than via brew casks, but it is entirely up to you. (see this issue).

iTerm2 terminal emulator

macOS awesome terminal

Ran Cohen ・ Jan 28 '22

#tutorial #productivity #macos #terminal
Visual Studio Code code editor.

To launch VS Code from the terminal, Open VS Code → Press ⌘ + ⇧ + P → Type shell command → Select Install code command in path.

Maccy clipboard manager
Rectangle keyboard windows manager
Xnip screenshot app
Kap screen recorder
KeepingYouAwake prevents sleep
Ngrok public URL to your localhost
Bitwarden password manager
AppCleaner thoroughly uninstall apps

Extra 🛸

Happy coding day 🍻