DEV Community: Scotticles

Jenkins for running scripts!

Scotticles — Mon, 09 Jan 2023 19:09:37 +0000

I have over 30 scripts that are used to pull data and push data between different services. Its critical to know when they fail. Sure you could do this with cron but then I looked around and saw that Jenkins could run scripts and have a console output log, and email me on fails.

After you have set up jenkins and set up the email settings.

Create a freestyle project
1. Build Triggers -> set your time so it runs whenever you want.
2. Build Steps -> Add Build Step -> Execute Shell perl <my/path/to/script.pl
3. Post Build -> Email Notification and make sure Send Email for every unstable build is checked.

This will get your scripts running and have email alerts on failed scripts. Its super helpful and its nice to have a decent web ui to automate and organize all these jobs I have to run.

If anyone has a better system / suggestion please let me know, I like what I have but I have been looking for something not so heavy on Java.

Jumphost - Pull Switch Configs with Perl

Scotticles — Fri, 09 Dec 2022 21:20:17 +0000

I have a few jumphosts that let I use to ssh into and then access network switches. I used to use rancid and have like 6 rancid instanes...no more. I wrote my own switch backup script. I wont go into the details of my script BUT i thought this ssh trick was worth a look with Perl.

Net::OpenSSH <--Awesome

This snippet of code does a ping check using a Jump Host. This is fairly easy to do using just ssh without Perl. But if I wanted to use Expect, I need Net::OpenSSH instance to pass into it.

The Modules I use for Expect are:
Mnet::Expect::Cli
Mnet::Expect::Cli::Ios
I actually wrap these in a MCE::Loop to do 10 or so switches at a time. MCE::Loop is always my go to for my loops that need to be sped up. Im pulling 30 switches in under a minute.

I am going to break up this snippet and explain whats going on.

Identify your two ips.

use Net::OpenSSH;
use Modern::Perl;

my $ip = '192.168.1.2'; #switch ip
my $proxy_ip = '10.90.3.2'; #jump host ip

sub pingCheck ( $ip, $proxy_ip) {

This example uses a ssh key pair, you can change key_path for password and put the ssh password in.

    my %opts = (host => $proxy_ip, user => 'myuser', key_path => '/home/myuser/.ssh/id_rsa');
    my $ssh_proxy = Net::OpenSSH->new(%opts);
    $ssh_proxy->error and die "Couldn't establish SSH connection: ". $ssh_proxy->error;

Now I run a ping command on the jump host.

    my @ping = $ssh_proxy->capture("ping -c 1 ".$ip);

    foreach(@ping) {
        if( $_ =~ m/^1\spackets transmitted/ && $_ =~ m/\s0\% packet loss/ ) {
            return 1;
        }
        elsif($_ =~ m/^1\spackets transmitted/) {
            return 0;
        }
    }
}
if(pingCheck()) {
    warn "YES";
}

If i needed to access the $ip using a different port instead of ping, lets say it was telnet.

$ssh_proxy->system({ssh_opts => ['-O','forward','-L127.0.0.1:8000:'.$ip.':23']});

I can then do telnet localhost 8000 and it will connect to the $ip.

This might be confusing for some, its technical and geared for network engineers that can do programming. If you are interested in how i pull switches, you can ask and i'll hint at how to put something together.

Snipe-IT + Perl

Scotticles — Mon, 05 Dec 2022 16:57:41 +0000

I have a new module that is a work in progress, added to as needed. It's on github so you can add more api calls to it and submit a pull request to me. Its co-developed by me and my co-worker.

https://metacpan.org/pod/WWW::SnipeIT

What is Snipe-IT?

Snipe-IT is an open source asset management, to enable IT departments to track who has which laptop, when it was purchased, which software licenses and accessories are available, and so on. It can be self-hosted making it a great option for system administrators.

Purpose

We needed a way to interact with Snipe-IT from Perl for a Koha Plugin he is making.

At first, I set it up to use Moo for the object class but then with Perl Devs working towards Corinna with Object::Pad, I moved it to Object::Pad, the migration was super easy, but I am not using a ton of its features.

How to use

Firstly, you need a access token and the url to the Snipe-IT instance, if its https that would be preferred. This example its http.

use WWW::SnipeIT;

my $snipeIT = SnipeIT->new( endpoint => 'http://<mysnipeip>/api/v1/', accessToken => 'mylongapikey');

Here is a basic example of pulling information on a asset tag.

https://snipe-it.readme.io/reference/api-overview

When creating this i tried to follow their structure with $snipe->{''} to make it easy to follow their guide.

my $snipe = $snipeIT->snipe();
my $result = $snipe->{'hardware'}->getHardwareIDByAssetTag(123);
say $result;

And more examples on how to use it.

my $updateBody = '{"name":"Fruit Computer","notes":"Apple Computer"}';
my $searchBody = '{"name":"name","element":"text","field_values":"3Com"}';
my $assetTag = 123;
my $hardwareID = 321;
my $user = "Scott";
my $fieldName = "";
my $fieldValue = "3Com";
my $assetName = "3Com";
my $searchString = "ChromeBook";
my $categoryID = 3;
my $serialNumber = "abc1234";

warn Dumper($snipe->{'hardware'}->getHardwareIDByAssetTag($assetTag));
warn Dumper($snipe->{'hardware'}->getHardwareByAssetTag($assetTag));
warn Dumper($snipe->{'hardware'}->getAssetTagByHardwareID($hardwareID));
warn Dumper($snipe->{'hardware'}->updateAssetByHardwareID($hardwareID, $updateBody));
warn Dumper($snipe->{'hardware'}->updateAssetByAssetTag($assetTag, $updateBody));
warn Dumper($snipe->{'hardware'}->getHardwareByCustomField($fieldName, $user));
warn Dumper($snipe->{'hardware'}->getHardwareBySerialNumber($serialNumber));


warn Dumper($snipe->{'hardware'}->getHistoryByHardwareID($hardwareID));
warn Dumper($snipe->{'hardware'}->getHistoryByAssetTag($assetTag));
warn Dumper($snipe->{'hardware'}->searchHardware($searchString));

warn Dumper($snipe->{'hardware'}->getHardwareByCategory($categoryID));

Its not fully featured but its a good start.

BookstackApp css fixes

Scotticles — Fri, 29 Apr 2022 20:25:41 +0000

Frustrated with their dark theme, and dark reader wasnt doing it. I moved over some of my dokuwiki theme i was using over and moved some other settings around like the opacity hovers were annoying me, the banner on top was annoying, the small content area was annoying with the two side bars way out there (large screen), this seems tweaks those things. Enjoy.

`
 html.dark-mode { background: #000 !important; color: #eee; } html.dark-mode body { color: #eee; } html.dark-mode .card { border: 1px solid #444; box-shadow: 0 0 0.5em #ddd; border-radius: 2px; } html.dark-mode header { filter: none !important; background-color: #000 !important; border-bottom: 1px solid #f90; } html.dark-mode h1, html.dark-mode h2, html.dark-mode h3, html.dark-mode h4, html.dark-mode h5, html.dark-mode h6 { color: #eee; } .breadcrumbs { opacity: 1; } @media screen and (min-width: 1000px) html.dark-mode .tri-layout-left-contents>, html.dark-mode .tri-layout-right-contents> { opacity: 1; } html.dark-mode .tri-layout-left-contents > *, html.dark-mode .tri-layout-right-contents > * { opacity: 1; } .cm-s-darcula.CodeMirror { background: #000; } .table > table > tbody { background-color: #000; } .page-content { max-width: 100%; } .tri-layout-middle-contents { max-width: 100%; } .tri-layout-container { max-width: 80%; margin: 0 auto; } `

Zabbix Proxy and Low Disk Space

Scotticles — Fri, 29 Apr 2022 15:44:17 +0000

I recently had one of my proxy servers run at 99% disk space. The proxy server is using mysql and because its proxy its pulling metrics -> the metrics get sucked off from the main server and then those records on the proxy get removed. My mysql had binary logs turned on... so the .bin files found in /var/lib/mysql was filling up all the space, so I edited the /etc/mysql/mysql.conf.d/mysqld.conf file to remove bin logging, it is not needed for this database and then purged the old bin files.

PURGE BINARY LOGS BEFORE '2022-04-29 00:00:00';

I am now only using 22% of my disk space. Which is mostly the OS. :D

Organizing Mojolicious Routes

Scotticles — Fri, 08 Apr 2022 16:07:20 +0000

Mojolicious is an awesome web framework and I have been developing an app in it for 2 years now.

The basic structure is (taken off the docs - Growing)

myapp                      # Application directory
|- script                  # Script directory
|  +- my_app               # Application script
|- lib                     # Library directory
|  |- MyApp.pm             # Application class
|  +- MyApp                # Application namespace
|     +- Controller        # Controller namespace
|        +- Example.pm     # Controller class
|- my_app.yml              # Configuration file
|- t                       # Test directory
|  +- basic.t              # Random test
|- log                     # Log directory
|  +- development.log      # Development mode log file
|- public                  # Static file directory (served automatically)
|  +- index.html           # Static HTML file
+- templates               # Template directory
   |- layouts              # Template directory for layouts
   |  +- default.html.ep   # Layout template
   +- example              # Template directory for "Example" controller
      +- welcome.html.ep   # Template for "welcome" action

the MyApp.pm is the core file that defines routes, app settings and plugins. As I was growing my app, that pm file became very large (200+ lines) with routes and I needed to do something. I remember reading from the devs that its best to use plugins for routes when your project becomes big.

I made a folder and a file under lib called Route > user.pm

I generally follow this pattern so if my url is /user/lookup
i'll have a route file called Route/user.pm that will handle all my /user routes.

Here is what the plugin looks like

under lib/MyApp/route/user.pm

package MyApp::Route::User;
use Mojo::Base 'Mojolicious::Plugin', -signatures;

sub register ($self, $app, $r) {
  $r->get('/user/lookup')->to('user#lookup');
}

1;

Now to add it into the MyApp.pm under lib.

my $r = $self->routes; #this should be somewhere already in the pm file.
$self->plugin('MyApp::Route::User', $r);

This removes tons of lines in the MyApp.pm and makes it easy to find the routes when needed to be worked on.

Fixing EFI on linux

Scotticles — Wed, 27 Jan 2021 20:14:41 +0000

I use kde neon and dual boot into windows every now and then, well my laptop receives firmware updates and I decided to get those updated on windows. I haven't had good luck on the linux fwupdate... Well it hosed my efi, windows boot still worked but my linux side was no longer booting. Thanks windows updates for forcing me to do them.

I scoured the internet for a solution. I tried boot-repair, no go. I downloaded KDE Neon live iso, put it on a usb and then finally found a solution.

source

I mounted the necessary folders and then chrooted into the mnt. I noticed my /boot was readonly..thats odd., unmounted it and ran

sudo dosfsck -t -a -w /dev/sda1 <--change /dev/sdx to your /boot partition disk

Then do this:

/dev/sda8 in this example is the / partition
/dev/sda1 in this example is the efi partition

sudo mount /dev/sda8 /mnt
sudo mount /dev/sda1 /mnt/boot/efi
for i in /dev /dev/pts /proc /sys /run; do sudo mount -B $i /mnt$i; done
sudo chroot /mnt
grub-install /dev/sda
update-grub

This might help someone, it's definitely a bookmark for me.

dynamic dns and ufw

Scotticles — Wed, 16 Dec 2020 22:21:53 +0000

Github Gist Link: Gist

Own a vps? I scored over black friday and now have a ton of them for some projects. I ended up setting up wireguard on all of them and linking them all together. I have a raspberry pi in my home for my private container registry and hooked it up to the wireguard network.

I did not want to open up my wireguard port and any ssh port to the public.

I found a similar script like this on a google search, had to modify it for ufw to work and to do wireguard and ssh. My home router does a dynamic dns update and keeps that updated. If your router cannot do that, you can setup a pi or a script to run off your computer so when it boots up it will try to keep it up to date.

UFW cannot accept a dns name, unfortunately. This script will require the host command, if your vps does not have that command, it is usually in bind-utils on ubuntu or dns-utils. Add in your hostname and ssh port, wireguard port. Run it manually first to verify it works and then cron it.

sudo chmod +x mydnsscript.sh
sudo ./mydnsscript.sh
sudo ufw status

You should see your home network ip allowed for ssh and wireguard.

Here is the script:

#!/bin/bash

#SET THE FOLLOWING

HOSTNAME=mydyndns.com
SSH_PORT=22
WIREGUARD_PORT=5246

#IF IT DOES NOT WORK, AT LEAST ON UBUNTU INSTALL, bind-utils to get the host command

#Create a cron /15 * * * * root bash /path/to/dynamicdnsupdater.sh

if [[ $EUID -ne 0 ]]; then
   echo "This script must be run as root"
   exit 1
fi
new_ip=$(host $HOSTNAME | head -n1 | cut -f4 -d ' ')
old_ip=$(/usr/sbin/ufw status | grep $HOSTNAME | head -n1 | tr -s ' ' | cut -f3 -d ' ')
if [ "$new_ip" = "$old_ip" ] ; then
    echo IP address has not changed
else
    if [ -n "$old_ip" ] ; then
        /usr/sbin/ufw delete allow from $old_ip to any port $SSH_PORT
        /user/sbin/ufw delete allow from $old_ip to any port $WIREGUARD_PORT
    fi
    /usr/sbin/ufw allow from $new_ip to any port $SSH_PORT comment $HOSTNAME
    /usr/sbin/ufw allow from $new_ip to any port $WIREGUARD_PORT comment $HOSTNAME
    echo UFW have been updated
fi

I use this so now my home network can reach my vps network via wireguard or ip and I can vpn into my home network to jump. Very cool!

Perl tips I wish I knew as a beginner

Scotticles — Wed, 09 Dec 2020 23:10:35 +0000

I think the hard part with Perl and other languages, is setting up your environment. As you do more Perl you realize there are better ways to do things that aren't straight forward when you first began the journey.

Disclaimer: I use Linux and this will be on a Linux environment, BUT with WSL on windows and with homebrew on mac.

System Perl and User Perl

On Linux, we have a system Perl. It's great to have but we don't want to mess with it. Lets keep the system Perl for the system. We want a user environment and you really must do this, its way better. We have two tools we can pick from that make it easy to get started.

Plenv and Perlbrew. I started on Perlbrew and moved over to Plenv.

Plenv allows you to make it so each project you work on has a .plenv file and you can specify the Perl version you want.

Look at the two links, pick one and set it up, pick a perl version and install it. Verify it works with perl -v.

You want to install cpanminus.
plenv install-cpanm <--plenv
cpan App-cpanminus

You are now off system Perl and on your user Perl, its awesome. Cpanminus is installed and you can now install Perl modules for your project.

The cpanfile

In your project you want to make a cpanfile. Cpanfiles contain a list of Perl modules.

As your project grows, add in the modules to the cpanfile. When you need to distribute it, you can use the below command to install the dependencies. Makes life easier for everyone.
cpanm --installdeps .

Carton

"Perl module dependency manager (aka Bundler for Perl)"
Carton is awesome, with plenv its even more awesome.

This is stolen from metacpan pod:

# On your development environment
> cat cpanfile
requires 'Plack', '0.9980';
requires 'Starman', '0.2000';

> carton install
> git add cpanfile cpanfile.snapshot
> git commit -m "add Plack and Starman"

# Other developer's machine, or on a deployment box
> carton install
> carton exec starman -p 8080 myapp.psgi

VSCode

Before vscode came out, I was living in PHPStorm with a Perl plugin, it worked but it was only okay. Code is awesome with all its plugins, once you get code up, browse the perl plugins. One you will want to have is perltidy and you'll need a .perltidyrc in your project, you can find examples on the web, but here is Mojolicoius perltidyrc. I use that one.

This should get you started with Perl.

Perl is dead simple with CBC Block Encryption

Scotticles — Tue, 08 Dec 2020 22:25:38 +0000

INTRO

I hope this blog post will help someone with CryptX.

My disclaimer: I am not an expert in cryptography.

I first started using Crypt::CBC but I liked how CryptX had more modules, better maintained and it was faster. I moved my system over to CryptX. In Crypt, the CBC piece was done more for you, where Crypt::Mode::CBC requires a little more from the developer to configure it.

WHY?

I am designing a system and needed a way to take files uploaded by users and encrypt them before storing them until needed. S3 has it built it, I know but I didn't want to rely on that. I decided to use CBC block as my encryption method. I also had to encrypt text and store them in the database, I can use the same encryption methods for both tasks. If you plan on storing in a database, remember these are bytes and needs to be a blob type.

HOW?

Here is the Crypt::Mode::CBC snippet

use Crypt::Mode::CBC;
my $m = Crypt::Mode::CBC->new('AES');

#(en|de)crypt at once
my $ciphertext = $m->encrypt($plaintext, $key, $iv);

Crypt::Mode::CBC

It uses AES, requires a $key and a $iv. AES is one of the many Ciphers to use. The padding by default is PKCS5 padding.

vulnerabilities-cbc-mode - Extra Reading

What the heck is the IV?

The IV is 'initialization vector'. To make each message unique, an initialization vector must be used in the first block. IV Resource

I generate a IV for each encrypted object and store it in the database for when decryption is needed.
IV Resource

Here is a full example.
I use Mojo::File for file manipulation, it makes things easy.

Just to note, rand() is not cryptographically secure. Please see the doc for picking the right module, if you need a secure implementation. In this example I am using Math::Random::Secure that replaced the native rand()
https://perldoc.perl.org/functions/rand

Requires Modules:

Crypt::Mode::CBC
Mojo::File
Modern::Perl
Crypt::Digest::SHA512_256
Math::Random::Secure

use Crypt::Mode::CBC;
use Mojo::File;
use Modern::Perl;
use warnings;
use strict;
use Crypt::Digest::SHA512_256 qw( sha512_256_hex );
use Math::Random::Secure qw(rand);
my $key =  sha512_256_hex(rand(1000));
my $iv =  sha512_256_hex(rand(1000));

# Lets print out the IV and KEY for fun.
say $key;
say substr($iv, 0, 32);

my $cbc = Crypt::Mode::CBC->new('AES');

# Encrypt
my $file = Mojo::File->new('plainfile.txt');

my $ciphertext = $cbc->encrypt($file->slurp, pack("H*", $key), pack("H*", substr($iv, 0, 32)));

my $encFile = Mojo::File->new('plainfile.txt.crypted');
$encFile->spurt($ciphertext);

# Move the old file
$file->move_to('plainfile.txt.backup');
$file->new('plainfile.txt');

# Decrypt

my $plaintext = $cbc->decrypt($encFile->slurp, pack("H*", $key), pack("H*", substr($iv, 0, 32)));

$file->spurt($plaintext);
# this should print out the data and if you cat the file plainfile.txt, it should print out the data.
say $plaintext;

If you run this, you will have a few files to look at.