DEV Community: Jeremy Stretch

Git Remotes Cheatsheet: Mastering Remote Repository Management

Jeremy Stretch — Thu, 07 Nov 2024 21:18:15 +0000

Who is this for?

Developers who need to:

Work with GitHub, GitLab, or other remote Git servers
Manage remote repositories
Collaborate with team members

What is covered?

This cheat sheet provides a comprehensive overview of Git remotes, including:

Essential remote operations and commands
Common terminology and concepts
Repository structures and relationships
Best practices for remote collaboration
Common workflows and scenarios
Troubleshooting tips

Whether you're working on personal projects or collaborating in a team, this guide will help you understand and effectively manage remote repositories.

What are Git Remotes?

Git remotes are versions of your repository hosted on the Internet or network. They enable collaboration and code sharing between team members.

Remote Commands

# List all remotes and their URLs
git remote -v

# Show remote branch details
git ls-remote <remote_name> <branch_name>

# Extract remote URL from Git config
git config --get remote.origin.url
# Alternative: grep "url =" .git/config

# Update remote URL
git remote set-url origin <new_url>

# Add new remote
git remote add <remote_name> <remote_url>

# Remove remote
git remote remove <remote_name>

# Rename remote
git remote rename <old_name> <new_name>

# Prune deleted remote branches
git remote prune <remote_name>

# Fetch from remote
git fetch <remote_name>

# Fetch specific branch from remote
git fetch <remote_name> <branch_name>

# Fetch all remote branches and tags
git fetch --all --tags

# Pull from remote (fetch + merge)
git pull <remote_name> <branch_name>

# Push to remote
git push <remote_name> <branch_name>

# Push to specific remote and branch
git push <remote_name> <local_branch>:<remote_branch>

# Force push (use with caution!)
git push --force-with-lease  # Safer alternative to --force
                             # Acts as a safety mechanism by checking if the remote
                             # branch has been modified since your last fetch.
                             # This helps prevent accidentally overwriting team
                             # members' work, which can happen with a regular
                             # --force push.

# Check remote tracking branches
git branch -vv

# Remove stale remote-tracking branches
git remote prune origin --dry-run  # Preview what will be pruned
git remote prune origin            # Actually prune
git fetch --prune                  # Fetch and prune in one command

# Mirror a repository (including all refs)
git clone --mirror <repository_url>

# Verify remote connections
git remote verify <remote_name>

# Sync fork with upstream
git fetch upstream
git checkout main
git merge upstream/main

Terminology

# Repository Concepts
repository  - A directory containing your project and its version history
local       - Your copy of the repository on your machine
remote      - A repository hosted on the internet or network (GitHub, GitLab, etc.)
fork        - A copy of someone else's repository under your account
clone       - A local copy of a remote repository

# Common Remote Names
origin      - The default name Git gives to the primary remote repository you cloned from
upstream    - Commonly used name for the original repository you forked from

# Branches and References
main/master - The default primary branch name (main is the new standard, master was historical)
HEAD        - A pointer to the current branch/commit you're working on
tracking branch - A local branch that tracks a remote branch (e.g., main tracks origin/main)

# Data Transfer Operations
fetch       - Download changes from remote without integrating them
pull        - Download changes from remote and integrate them (fetch + merge)
push        - Upload your local changes to a remote repository

# Merging and Integration
merge       - Combine changes from one branch into another
fast-forward - A merge where no new commit is created (target branch just moves forward)
merge commit - A new commit created when combining changes from two branches
conflict    - When changes in two branches modify the same code and cannot auto-merge
rebase      - Alternative to merge: replays your changes on top of target branch
cherry-pick - Apply a specific commit from one branch to another

Repository Structure Example

Your Local Repo (local)
    │
    ├── origin (your fork on GitHub)
    │     └── main branch
    │
    └── upstream (original repository)
          └── main branch

Example:
- upstream: https://github.com/original-author/project
- origin: https://github.com/your-username/project
- local: /home/your-username/project

Remote Setups Example

Simple Setup (Direct):
local ↔ origin (primary remote repository)

Fork Setup (Contributing to others):
local ↔ origin (your fork) ↔ upstream (original repository)

Multiple Remotes:
local ↔ origin (primary)
     ↔ staging (testing server)
     ↔ production (live server)

Best Practices

Always verify the remote URL before pushing sensitive code
Use SSH over HTTPS when possible for better security
Set up SSH keys with passphrase for secure authentication
Use --force-with-lease instead of --force to prevent overwriting others' work
Regularly sync with upstream repositories when working with forks
Set up branch protection rules on remote repositories for critical branches
Use meaningful names for remotes (e.g., 'upstream' for original repo, 'origin' for fork)
Configure remote-tracking branches explicitly for better control
Implement backup strategies using multiple remotes
Use .gitignore properly to avoid pushing sensitive or unnecessary files
Review remote changes before merging using git fetch + git diff
Document your repository structure and Git workflows in a README.md file

Key Points

Working with remotes is a fundamental part of Git and collaborative development. Remember these key points:

Always synchronise with your remote before starting new work
Regularly commit and push your changes to avoid large, complex merges
Use meaningful commit messages to help your colleagues understand your changes
Keep your local repository organised and regularly prune old branches
Document your team's Git workflow to maintain consistency
Consider using Git aliases for frequently used remote commands
Back up your work regularly by pushing to remote repositories
When in doubt, create a new branch rather than working directly on main

For more detailed information, consult:

Git official documentation: https://git-scm.com/doc
GitHub Guides: https://guides.github.com/
Your team's internal Git guidelines

Ansible Vault Cheatsheet: Mastering Secure Configuration Management

Jeremy Stretch — Sat, 26 Oct 2024 15:35:05 +0000

What is Ansible Vault?

Ansible Vault is a feature in Ansible that allows you to encrypt and protect sensitive data (such as passwords, API keys, etc.) that needs to be included in your Ansible projects.

Why Use a Vault Password?

Ansible Vault uses a password to encrypt and decrypt files. This is useful for keeping sensitive information secure, as it allows you to store encrypted content in version control systems without exposing sensitive data.

Benefits of Using a Vault Password:

Security: Keeps sensitive data encrypted and safe.
Version Control: Encrypted files can be safely pushed to repositories like Git.
Ease of Use: Once configured, using vault-encrypted files in playbooks is simple.

Example Use Case: Storing Database Credentials Securely

Let's assume we want to store database credentials (username and password) in a secure file for use in an Ansible playbook.

1. Create a New Vault File

To create a new encrypted vault file that stores database credentials:

ansible-vault create db_credentials.yml

This command will:

Prompt for a password (which will be used for encryption).
Open the default text editor (e.g., Vim) where you can add your sensitive data.

Add the following content to db_credentials.yml (sample database credentials):

db_user: admin
db_password: secure_password123

Once you save and exit, the file will be encrypted.

2. Edit an Existing Vault File

To edit the vault file (e.g., if you need to update the database password), use the ansible-vault edit command:

ansible-vault edit db_credentials.yml

This will:

Prompt for the vault password to decrypt the file.
Open the file in your default editor, allowing you to make changes.

Update the password (or any other value), then save and exit to re-encrypt the file automatically.

3. Encrypting Individual Strings

Sometimes you may want to encrypt just a single string rather than an entire file. This is useful for storing sensitive data in otherwise unencrypted files.

ansible-vault encrypt_string 'secret_password' --name 'db_password'

This will output an encrypted version of 'secret_password' that you can paste into a YAML file:

db_password: !vault |
          $ANSIBLE_VAULT;1.1;AES256
          66386439653236336462626566653063336164663966303231363934653561363132333162393533
          3661643066663534383564343537343334633431346664310a316465383138636532343463633236
          37623064636339623565626265353466613262366165396233396465636135353863376136393132
          3938626664623838350a653839636539636465626565316130383833623733326132366265376461
          6233

4. Using Multiple Vault Passwords

When working with different environments (e.g., development and production), you might want to use different vault passwords for each.

1. Create vault-encrypted files with different IDs:

ansible-vault create --vault-id dev@prompt secret_dev.yml

ansible-vault create --vault-id prod@prompt secret_prod.yml

2. Use these vault IDs in your playbook:

ansible-playbook site.yml --vault-id dev@prompt --vault-id prod@prompt

This allows you to use different passwords for different environments, enhancing security.

5. View the Contents of a Vault File

To view the encrypted file without editing it, use the ansible-vault view command:

ansible-vault view db_credentials.yml

This command will:

Prompt for the vault password.
Display the decrypted content of the vault file.

This is useful when you only need to see the values without making changes.

6. Encrypt an Existing File

If you already have a plain-text file and want to encrypt it using Ansible Vault, you can use:

ansible-vault encrypt plain_file.yml

This will:

Encrypt the contents of plain_file.yml and overwrite the file with its encrypted version.

7. Decrypt an Encrypted Vault File

If you need to decrypt a vault file and revert it to plain text, you can use:

ansible-vault decrypt db_credentials.yml

This will:

Prompt for the vault password.
Decrypt the file, leaving it as plain text.

8. Running a Playbook with Encrypted Variables

If your playbook includes variables from a vault-encrypted file (like db_credentials.yml), you can run the playbook by providing the vault password:

ansible-playbook site.yml --ask-vault-pass

Alternatively, you can specify a password file for automation purposes:

ansible-playbook site.yml --vault-password-file /path/to/vault_pass.txt

Adding and Using Vault-Encrypted Variables in a Playbook

To use the encrypted variables from db_credentials.yml in your playbook:

1. Include the vault file in your playbook:

Add the following line at the beginning of your playbook:

   - name: Include database credentials
     include_vars:
       file: db_credentials.yml

2. Use the decrypted variables in your tasks:

Once included, you can use the variables like any other Ansible variable:

   - name: Configure database connection
     mysql_user:
       name: "{{ db_user }}"
       password: "{{ db_password }}"
       priv: "*.*:ALL"
       host: "localhost"
       state: present

3. Full playbook example:

   ---
   - hosts: database_servers
     vars_files:
       - db_credentials.yml

     tasks:
       - name: Configure database connection
         mysql_user:
           name: "{{ db_user }}"
           password: "{{ db_password }}"
           priv: "*.*:ALL"
           host: "localhost"
           state: present

In this example, we use vars_files to include the encrypted file directly in the play.

Remember to run this playbook with either --ask-vault-pass or --vault-password-file as mentioned earlier.

9. Specifying Vault Password in ansible.cfg

To avoid entering the vault password every time, you can specify the vault password file in your ansible.cfg:

[defaults]
vault_password_file = /path/to/vault_pass.txt

This is especially useful for automation purposes, but be cautious about the security implications of storing your vault password on disk.

10. Changing the Encryption Key (Rekeying)

If you need to change the encryption key of a vault-encrypted file:

ansible-vault rekey secret.yml

This will prompt you for the current vault password and then ask for a new password. It's a good practice to periodically rekey your vault-encrypted files for security reasons.

Best Practices for Ansible Vault

Use a Strong Vault Password: Always choose a strong and unique password to protect your encrypted files.
Version Control Safety: It is safe to store vault-encrypted files in version control, but never commit the vault password or password file to version control.
Separate Vault Files: If possible, separate sensitive data into dedicated vault files to minimise the exposure of credentials across different environments or teams.
Use --vault-password-file for Automation: When automating playbook runs in CI/CD pipelines (e.g., GitLab CI), use the --vault-password-file option to avoid manual password entry.
Restrict File Access: Ensure that only authorised users and systems have access to the vault password file and the encrypted files.
Environment-Specific Vaults: Use separate vaults for different environments (development, staging, production) to ensure proper security segregation.

Summary of Commands

Action	Command
Create a new encrypted file	`ansible-vault create secret.yml`
Edit an existing encrypted file	`ansible-vault edit secret.yml`
View an encrypted file without editing	`ansible-vault view secret.yml`
Encrypt an existing file	`ansible-vault encrypt existing_file.yml`
Decrypt an encrypted file	`ansible-vault decrypt secret.yml`
Encrypt a string	`ansible-vault encrypt_string 'secret_password' --name 'db_password'`
Create encrypted files with different vault IDs	`ansible-vault create --vault-id dev@prompt secret_dev.yml` `ansible-vault create --vault-id prod@prompt secret_prod.yml`
Run a playbook with vault-encrypted files	`ansible-playbook site.yml --ask-vault-pass` `ansible-playbook site.yml --vault-password-file /path/to/vault_pass.txt` `ansible-playbook site.yml --vault-id dev@prompt --vault-id prod@prompt`
Change the encryption key of a vault-encrypted file	`ansible-vault rekey secret.yml`

Remember: When using ansible-vault, you'll be prompted for the vault password unless you specify a password file.

DEV Community: Jeremy Stretch

Git Remotes Cheatsheet: Mastering Remote Repository Management

Who is this for?

What is covered?

What are Git Remotes?

Remote Commands

Terminology

Repository Structure Example

Remote Setups Example

Best Practices

Key Points

For more detailed information, consult:

Ansible Vault Cheatsheet: Mastering Secure Configuration Management

What is Ansible Vault?

Why Use a Vault Password?

Benefits of Using a Vault Password:

Example Use Case: Storing Database Credentials Securely

1. Create a New Vault File

2. Edit an Existing Vault File

3. Encrypting Individual Strings

4. Using Multiple Vault Passwords

5. View the Contents of a Vault File

6. Encrypt an Existing File

7. Decrypt an Encrypted Vault File

8. Running a Playbook with Encrypted Variables

Adding and Using Vault-Encrypted Variables in a Playbook

9. Specifying Vault Password in ansible.cfg

10. Changing the Encryption Key (Rekeying)

Best Practices for Ansible Vault

Summary of Commands

Jeremy Stretch 241024