DEV Community: lbonanomi

Github "Dimmed" Dark-Mode Images

lbonanomi — Sun, 19 Jun 2022 16:17:43 +0000

Hey gang,

Github supports gh-light-mode-only and gh-dark-mode-only for images, does anyone know if there's support for targeting an image to the Dark Dimmed theme?

TIA!

Lecturing About curl-bash Pipelines

lbonanomi — Tue, 26 Oct 2021 23:11:49 +0000

Grab some popcorn 'cuz we're here to talk about curl-bash pipelines on Linux. Personally I don't like them for maintenance reasons (how do you remove things you installed like this?), and I want to nag my fellows to not casually run a wall-of-script from the Internet.

Interactive vs non-interactive shells

When a bash shell is started at login it's considered interactive and expects to be attached to a TTY. When we run curl https://example.com/script.sh | bash the bash process started is non-interactive and expects no human input. We can confirm a bash process' interactivity either by looking at the process table (run ps) or by checking the builtin variable $- (the shell's starting arguments) for an 'i' character.

PIDs, Parents and "Siblings"

Part of the classic Unix process model is that a process has a parent that created it and may have child processes that it created. For the purpose of this exercise let's call processes that were created by the same immediate parent process "siblings".

We can find the current PID's parent with ps -p $$ -o ppid -h. Running a wide-ranging ps and grepping for a common parent PID will show us all processes started by the same parent PID as our shell.

Seeing connections in procfs

Now that we have a list of all a PID's siblings we can cat-out procfs to see if any of those processes are connected to a known HTTP port.

$ cat /proc/21456/net/tcp
  sl  local_address rem_address   st tx_queue rx_queue tr tm->when retrnsmt   uid  timeout inode                                                     
   0: 00000000:2328 00000000:0000 0A 00000000:00000000 00:00000000 00000000     0        0 264000 1 0000000000000000 100 0 0 10 0                    
   1: 00000000:0016 00000000:0000 0A 00000000:00000000 00:00000000 00000000     0        0 264692 1 0000000000000000 100 0 0 10 0                    
   2: 0100007F:0019 00000000:0000 0A 00000000:00000000 00:00000000 00000000     0        0 48214 1 0000000000000000 100 0 0 10 0                     
   3: F71CFE0A:0016 224EFBCF:FBC3 01 00000034:00000000 01:00000015 00000000     0        0 67266130 3 0000000000000000 21 4 17 10 54                 
   4: F71CFE0A:9EE0 856EC7B9:01BB 08 00000000:00000001 02:000012E3 00000000 150001        1 67376561 2 0000000000000000 20 4 0 10 -1                 
   5: F71CFE0A:88D8 0200FE0A:0035 06 00000000:00000000 03:0000028A 00000000     0        0 0 3 0000000000000000                                      
   6: F71CFE0A:89C8 0200FE0A:0035 06 00000000:00000000 03:00001073 00000000     0        0 0 3 0000000000000000                                      
   7: F71CFE0A:960E 98A27032:01BB 01 00000000:00000000 02:000008BC 00000000     0        0 67384497 2 0000000000000000 26 4 6 10 -1                  
   8: F71CFE0A:DC02 71529D36:270D 01 00000000:00000000 00:00000000 00000000     0        0 67343092 1 0000000000000000 20 4 1 10 -1

This looks pretty-intense but we're only interested in the last 4 characters (TCP port) of the third column (remote host). These values are all in sigh hex, so it will be fastest to just grep for a few well-known ports like 0050 (80), 01BB (443), 1F90 (8080 and 20FB (8443)

Putting it all together

Now that we can detect non-interactive shell's children connecting to a well-known web port and suspend their process how do we get this functionality deployed? Let's save our final shell script as /usr/local/bin/blocker.sh:

if [[ $(grep "$(cat /proc/$$/cmdline | tr '\000' "\n" | tail -1)$" /etc/shells) ]]
then
    echo $- | grep -qv i && ps awwwx -ocmd,pid,ppid | grep "$(ps -p $$ -o ppid -h)$" |  while read p
    do
        sib=$(echo "$p" | awk '{ print $(NF-1) }')
        egrep -q ":0050|:01BB|:1F90|:20FB"  /proc/$sib/net/tcp 2>/dev/null && kill -SIGSTOP $sib 2>/dev/null &&\
        echo "If you 𝙧𝙚𝙖𝙡𝙡𝙮 want to execute some rando script from the Internet type ctrl-Z and then 𝗳𝗴"
    done | uniq
fi

and set $BASH_ENV=/usr/local/bin/blocker.sh to execute our script at startup of every non-interactive bash shell.

As an example, here's this script pausing an install of Omnitruck

; curl -s https://omnitruck.chef.io/install.sh | bash
If you 𝙧𝙚𝙖𝙡𝙡𝙮 want to execute some rando script from the Internet type ctrl-Z and then 𝗳𝗴

[1]+  Stopped                 curl -s https://omnitruck.chef.io/install.sh | bash

What was I doing in this shell?

lbonanomi — Sat, 14 Nov 2020 14:09:18 +0000

Do you frequently open another SSH window for another task or to connect to another machine? I know I do. Do you ever return to those windows a day or two later and wonder what you were doing there to begin-with? I know I do.

Put this into your .profile to to run the lulwat function any time you hit enter three times in a row.

# source me.

export PROMPT_COMMAND=btnMash

function btnMash {
  # Init a counter for prompts-displayed if its missing
  [[ -f /tmp/promptsDisplayed ]] || history | tail -1 | awk '{ print $1 }' > /tmp/promptsDisplayed;

  # Init a counter for commands
  [[ -f /tmp/commandsRun ]] || (history | tail -1 | awk '{ print $1 }' > /tmp/commandsRun)

  COMMANDS=$(cat /tmp/commandsRun)

  # Increment prompts-displayed counter
  echo $(($(cat /tmp/promptsDisplayed)+1 )) > /tmp/promptsDisplayed;

  history | tail -1 | awk '{ print $1 }' > /tmp/commandsRun

  if [[ $COMMANDS -lt $(cat /tmp/commandsRun) ]]
  then
    cat /tmp/commandsRun > /tmp/promptsDisplayed
  fi

  PROMPTS=$(cat /tmp/promptsDisplayed)

  if [[ $PROMPTS -ge $(($COMMANDS+3)) ]]
  then
    lulwat; rm /tmp/promptsDisplayed; return
  fi
}

function lulwat {
    # If I am su-ed into this account, tell me.
    #
    ID=$(id | awk '{gsub(/[\(-\)]/," ")} { print $2 }');
    WHO=$(who -m | awk '{ print $1 }')

    [[ "$ID" == "$WHO" ]] && echo "You are: $ID"
    [[ "$ID" == "$WHO" ]] || echo "You are: $ID (su-ed from $WHO)"

    # Hostname and path
    #
    printf "On: $(hostname):$(pwd)\n"

    # If this a git repo, remind me of the remote
    #
    git status &>/dev/null && (
        printf "$(git remote -vv | awk '{ print $2 }' | head -1) ($(git branch | awk '{ print $2 }'))\n"
    )
}

Cheap sudo tricks

lbonanomi — Sat, 01 Aug 2020 14:15:40 +0000

It's easier to ask forgiveness than it is to get permission.
-- Grace Hopper

I've done ops work under various titles for various employers for the last 15 years. If memory serves every one of them has given the ops-geeks sudo rules that allowed sudo chmod and sudo chown to run as root, but would choke on a request for sudo -i. Here's why that's a stupid policy:

Create an SSH key on your local box if you don’t have one already.
Connect to a target box, and cd to ~roleaccount.
If ~roleaccount/.ssh exists, change permissions on it to allow you to read and write. Otherwise, create ~roleaccount/.ssh
cd to ~roleaccount/.ssh and sudo chmod o+wx authorized_keys.
Edit the authorized_keys file to append your public key.
Restore permissions on authorized_keys
Go up to ~roleaccount and restore permissions on .ssh
From your local box: ssh roleaccount@target_host

If you reset permissions correctly, you have just connected directly as roleaccount.

This isn't a stealthy method of switching to another user, so please be ready to explain yourself if there's an audit.

Yet another Github Profile

lbonanomi — Sat, 11 Jul 2020 19:32:31 +0000

I can't find anything to say about Github profiles that hasn't been posted already so I'll ask a question instead: does anyone know the recommended daily allowance of Perl?

https://github.com/lbonanomi/

Hints For Engineers During Outages

lbonanomi — Fri, 22 May 2020 10:43:03 +0000

Don't make jokes in a logged channel. Every written record will be read by someone with more clout and less sense of humor than you.
Don't feel like you have to offer an opinion if you weren't asked for one. Its okay to look and listen before saying anything.
Be high tea with the Queen level courteous. Angry people make bad decisions, and people can stay mad for a long time.
Excuse yourself from small outage calls if you need to attend to something. Excuse yourself to your direct manager for large outage calls.
Management has an agenda you don't know about, avoid getting into territory battles above your paygrade.

Your Own Gravatar.com

lbonanomi — Fri, 01 May 2020 11:06:40 +0000

The Problem

After a a few years of running Jira as a pool of federated servers, management decided to let us consolidate into a Jira Datacenter cluster. We provisioned extremely beefy VMs, allocated huge disks and got a dedicated Postgres cluster. Projects were imported, attachments migrated and we were pretty pleased with ourselves until the user feedback started: we hadn't imported the user's custom avatars correctly and rapid boards looked like hot garbage.

The vendor's instructions are geared toward moving whole instances from one platform to another, which didn't fit our model of merging multiple Jiras together. After some deliberation we decided to try hosting avatars on an independent server.

Hosting your own Gravatar server

After experiments with the API, and a discussion about making changes directly in the database we decided to try using a custom avatar server. After a surprisingly frustrating day trying to get Libravatar on RHEL7, we settled on the comparatively simpler Go project intravatar. Intravatar offers binaries that work beautifully right out of the box, but we made a few simple changes to accommodate our MitM SSL and Internet proxying. Now that we have a place to serve them from, let's get avatars!

Grabbing Jira avatars

Jira stores avatar files under $JIRA_HOME/data/avatars in the format of $ID_$name, so we used the REST API to associate usernames and avatar files. The below bash script will get a list of users from the Jira database, parse details from the user endpoint and scrape the instance's avatar URL to get a file.

#!/usr/bin/bash

# Database connection string is intentionally vague.
psql -tc "select lower_user_name from cwd_user;" | while read USER
do
    curl -snk "https://$OLD-JIRA/rest/api/2/user?username=$USER" | jq .avatarUrls | grep 48 | grep -q $OLD-JIRA && (
        # Get old avatar
        FILE=$(curl -snk "https://$OLD-JIRA/rest/api/2/user?username=$USER" | jq ."emailAddress" | tr -d '"' | tr -d "\n" | md5sum - | awk '{ print $1 }')
        URL=$(curl -snk  "https://$OLD-JIRA/rest/api/2/user?username=$USER" | jq .avatarUrls | grep 48 | awk -F"\"" '{ print $4 }')

        echo "$URL" | grep gravatar && (
            curl -snk $URL > $FILE
        )

        echo "$URL" | grep gravatar || (
            curl -snk > $FILE
        )
    )
done

My team <3s Bert Baron and intravatar

Quick Checks for Log Repetition

lbonanomi — Sun, 26 Apr 2020 11:21:35 +0000

Got paged at 06:30, second disk alert on an Apache proxy in 24 hours.

$ tail -500 access_log | awk '{ print $11 }' | while read l ; 
do echo "$l" | cksum; done | sort | uniq -c | sort -rnk1
    254 646511054 1131
    241 4112021984 1128
      1 785623522 964
      1 3691076460 4
      1 3287357281 899
      1 2484884658 1131
      1 2437517015 72

Looks like 99% of requests are the same 2 things. Eyeballing demonstrates my hosts are getting polled hard by a user. They are suspended until they explain what they are doing to generate ~15 gigs of logging all by themselves every 20 hours.

Hints For Managers During Outages

lbonanomi — Wed, 12 Feb 2020 00:49:01 +0000

If you talk to another engineer privately about our problem while I am on-task I will consider the problem assigned to someone else and find another chore. If you talk to another engineer about me privately, expect both of us to waste valuable time arguing with each other instead of fixing something.
If you say "can we X?" when you mean "Do X." I will start considering everything you say to be a conversation starter instead of an instruction.
If you drop off our outage call to attend to another regularly scheduled meeting "our" problem will lose urgency.
If you schedule a group status call and then fail to join it because you were informed out-of-band I will assume law of the jungle and feel free to update your manager about my progress directly.
If you miss the group status call without explanation and then ask for a status, please expect little.
Your decision did not retroactively become our decision because something broke.

What's New at The Book of Secret Knowledge

lbonanomi — Mon, 02 Dec 2019 12:57:37 +0000

Do you follow The Book of Secret Knowledge on Github?

Here's a full-content RSS feed of recent changes.

Provider Shells

lbonanomi — Fri, 08 Nov 2019 01:03:53 +0000

There are a ton of Linux users on dev.to, do any of you use remote Linux shell services or is everyone operating off of their PCs and laptops?

Am I trapped in 1995?

SSH Shibboleths

lbonanomi — Tue, 05 Nov 2019 12:40:13 +0000

I'm a bad meeting attendee; 5 minutes into any slide deck I retreat into paranoid fantasies of network espionage. Last budget session fantasy-me needed a subtle back channel to indicate that I was operating from a compromised position, so regular me hacked-together a working prototype with bash init scripts, SSH key features, and a little-used SSH config file.

Commands in SSH keys

It's probably common knowledge that key values in the SSH authorized_keys file can be locked to a single command, so I'll just say that the regularly used SSH key for this system was altered to start with command="exec bash" to start an interactive shell.

sshrc

Hosts using the OpenSSH daemon will process a user's $HOME/.ssh/rc file before sourcing the user's shell init scripts. By default sshd will not allow environment variables to be exported, but will happily execute shell scripts, so a simple instruction to run touch $HOME/.ssh.lck was added to create a lockfile. This has the helpful side-affect of not messing with console logins.

bashrc

Last stop is to modify .bashrc to look for the ~/ssh/rc lockfile, set $PROMPT_COMMAND to execute a function for password-auth vs. ssh key-auth, and remove the lockfile.

if [[ -f .ssh.lck ]]
then
        if [[ $(ps axwww | awk '$1 == '"$$"' { print $NF }' | grep "bash") ]]
        then
                function passworded() {
                        echo "User is password-authenticated"
                        unset PROMPT_COMMAND
                }
                export PROMPT_COMMAND="passworded"
        fi

        rm .ssh.lck 2>/dev/null
else
        function keyed() {
                echo "User is SSH key-authenticated"
                unset PROMPT_COMMAND
        }
        export PROMPT_COMMAND="keyed"

        rm .ssh.lck 2>/dev/null
fi

The passworded and keyed functions should of course be tailored for your needs.