DEV Community: muckitymuck

Simple Express server

muckitymuck — Tue, 17 Jan 2023 17:40:53 +0000

const express = require('express') // including the express library
const app = express() // creating an express application instance
const port = 3123 // setting the port for the app to run on

const cluster = require('cluster') // including the cluster library
const numCPUs = require('os').cpus().length; // getting the number of CPU available on the machine

if (cluster.isMaster) {
  console.log(`Master ${process.pid} is running`);

  // Fork workers.
  for (let i = 0; i < numCPUs; i++) {
    cluster.fork();
  }

  cluster.on('exit', (worker, code, signal) => {
    console.log(`worker ${worker.process.pid} died`);
  });
} else {
  // this is where the code will trigger
  const clientid = ''
  if (cluster.isWorker){
    console.log(`Client ID: ${cluster.worker.id} running`)
    const clientid = cluster.worker.id
  }
}

const bodyParser = require("body-parser"); // including the body-parser library
app.use(bodyParser.json()); // using json body parser for the app

const timestamp = new Date().toISOString(); // getting the timestamp for each request

app.get('/', (req, res) => {
    console.log(`OK: ${timestamp}`) // logging the timestamp of the request

    console.log(`data received: ${JSON.stringify(req.body)}`) // logging the data received in the request
    console.log(`${clientid}:${timestamp}:${req.body}`) // logging the clientid, timestamp and data received
    res.send(`OK: ${timestamp}`) // sending a response to the client

})
app.use((req, res, next) => {
    res.header("Access-Control-Allow-Origin", "*"); // allowing all origin to access the server
    res.header("Access-Control-Allow-Headers", "Origin, X-Requested-With, Content-Type, Accept"); // allowing certain headers
    next();
});
app.post('/', ( req, res ) => {
    console.log(req.body) // logging the data received in post request
    res.json({
        message: `OK ${new Date().toISOString()}`, // sending back a message with the current timestamp
        data: req.body // sending back the data received in the request
    });
})

app.listen(port, () => {
    console.log(`example app listening on port ${port}`) // logging that the server is listening on a specific port
})

AI Generated:

In this code, we are using the cluster module in Node.js to create a cluster of worker processes that listen on the same port and handle incoming requests. The cluster module allows for easy creation of child processes that share server ports, which can greatly improve the performance of a Node.js application.

The first thing we do is require the cluster and os modules. Then, we use os.cpus().length to get the number of CPU cores on the machine, which we will use to determine how many worker processes to create.

Next, we check if the current process is the master process using cluster.isMaster. If it is, we log that the master process is running and then use a for loop to create worker processes equal to the number of CPU cores. We also set up an event listener for when a worker process exits, which logs the worker's process ID.

If the current process is not the master process, we then set a variable clientid to be an empty string. Then we check if the process is a worker with cluster.isWorker. If it is, we log that the worker process with the corresponding ID is running and set clientid to be the worker's ID.

After that, we use the body-parser middleware to parse the incoming request body as JSON. Then, we set up routes for handling GET and POST requests to the server. For GET requests, we log the timestamp and any data received in the request body. For POST requests, we log the request body and send a JSON response containing a message with the current timestamp and the data received in the request.

Finally, we have the server listen on the specified port.

In short, this code is using the Node.js cluster module to create worker processes and listening on the same port to handle incoming requests. This helps improve performance by allowing multiple requests to be handled simultaneously. The code also logs the worker ID and any data received in the request body. This can be useful for debugging and keeping track of which worker is handling which requests.

ChatGPT pair programming

muckitymuck — Mon, 09 Jan 2023 20:04:13 +0000

github
live demo
I know it's been a bit (too long) since I wrote a blog on what I was doing. Recently, I was hired for a OpenAI project and started researching to prepare myself. The best way to learn is to do and so I started building my own OpenAI project.
This project is a small scope generative AI text service to put out an opening setting to a crime mystery novel in the chosen style of famous authors and constrained to locations entered by the user. For example, if you enter a city like NYC it will set the crime to a typical setting like a back alley, subway station, or Manhattan penthouse. A setting for temperature will also affect the output by how random the setting is. The author style setting affects the location but also affects the characters that appear in the setting.

As a bonus to the project, I worked along side ChatGPT to fill out the app basic code. I would say the Chat got it about 80% at any given output. It did understand how the backend works with the frontend with naming conventions but random misused syntax would pop in.

In this conversation, we worked on a project that is a web application that uses OpenAI's GPT-3 API to generate mystery scenes in different writing styles. The project was built using a combination of Next.js, the OpenAI API, React, Node.js, and Express.

We first added a range slider to the web page that allows the user to change the temperature of the GPT-3 model. We also added a display on the webpage to show the current temperature.

Next, we modified the generate.js file to include the value set on the range slider as the temperature in the request to the OpenAI API.

We also added a dropdown menu to the frontend to allow the user to switch the writing style in the prompt. To do this, we added a state variable to store the value of the selected style and passed it to the value prop of the select element.

Finally, we wrote a README.md file for the project, which provides an overview of the technology used and instructions for running the project locally.

Overall, this conversation demonstrated how to use OpenAI's GPT-3 API to generate text in different styles and how to incorporate user input in the form of a range slider and dropdown menu in a web application.

generate.js

import { Configuration, OpenAIApi } from "openai";

const configuration = new Configuration({
  apiKey: process.env.OPENAI_API_KEY,
});
const openai = new OpenAIApi(configuration);

export default async function (req, res) {
  if (!configuration.apiKey) {
    res.status(500).json({
      error: {
        message: "OpenAI API key not configured, please follow instructions in README.md",
      }
    });
    return;
  }

  const scene = req.body.scene || '';
  if (scene.trim().length === 0) {
    res.status(400).json({
      error: {
        message: "Please enter a valid location",
      }
    });
    return;
  }

  try {
    const temperature = Number(req.body.temperature) || 0.5;
    const completion = await openai.createCompletion({
      model: "text-davinci-003",
      prompt: generatePrompt(scene),
      max_tokens: 200,
      //temperature: 0.9,
      temperature: temperature,
    });
    res.status(200).json({ result: completion.data.choices[0].text });
  } catch(error) {
    // Consider adjusting the error handling logic for your use case
    if (error.response) {
      console.error(error.response.status, error.response.data);
      res.status(error.response.status).json(error.response.data);
    } else {
      console.error(`Error with OpenAI API request: ${error.message}`);
      res.status(500).json({
        error: {
          message: 'An error occurred during your request.',
        }
      });
    }
  }
}

function generatePrompt(scene, style, character) {

  let prompt = `Suggest a setting and initial characters with names the cops meet at a crime scene in the 
  style of famous authors`

  if (style === "Agatha Christie"){
    prompt += 
    `
    Setting: ${scene}
    Character: ${character}
    style: ${style}
    `;
  } else if (style === "Raymond Chandler"){
    prompt += 
    `
    Setting: ${scene}
    Character: ${character}
    style: ${style}
    `;
  }else if (style === "P.D. James"){
    prompt += 
    `
    Setting: ${scene}
    Character: ${character}
    style: ${style}
    `;
  } else if (style === "Dashiell Hammet"){
    prompt += 
    `
    Setting: ${scene}
    Character: ${character}
    style: ${style}
    `;
  } else {
    prompt += 
    `
    Setting: ${scene}
    Character: ${character}
    style: ${style}
    `;
  }
  return prompt;
}

The generate.js file is a server-side file that exports a function that handles an HTTP POST request to the /api/generate endpoint. When this function is called, it does the following:

It first checks whether the OpenAI API key is configured. If it is not configured, it sends a JSON response with an error message to the client.
If the API key is configured, it retrieves the scene value from the request body. If the scene value is not provided or is an empty string, it sends a JSON response with an error message to the client.
If the scene value is provided, it creates a completion using the OpenAI API, passing in the model, prompt, max_tokens, and temperature values as parameters.
If the completion is created successfully, it sends a JSON response with the result of the completion to the client.
If there is an error during the request to the OpenAI API, it sends a JSON response with an error message to the client.
This function is used to generate a mystery scene based on the scene input provided by the client and send the generated text back to the client as a response.
In the generate.js file, prompt is a parameter passed to the createCompletion method of the OpenAI API. It represents the text that the API should use as a starting point for generating text.

In the code provided, prompt is generated by calling the generatePrompt function and passing it the scene value as a parameter. The generatePrompt function returns a string that prompts the GPT-3 model to generate a mystery scene with a setting and initial characters with names that the police meet at a crime scene in the style of Arthur Canon Doyle, Agatha Christie, H.P. Lovecraft, or Raymond Chandler.

The prompt value is used by the OpenAI API as the starting point for generating text, which is then returned to the client as a response.

index.js

import Head from "next/head";
import { useState } from "react";
import styles from "./index.module.css";


export default function Home() {
  const [temperature, setTemperature] = useState(0.5);
  const [sceneInput, setSceneInput] = useState("");
  const [style, setStyle] = useState("");
  const [result, setResult] = useState();
  console.log(result);

  async function onSubmit(event) {
    event.preventDefault();
    try {
      const response = await fetch("/api/generate", {
        method: "POST",
        headers: {
          "Content-Type": "application/json",
        },
        body: JSON.stringify({ 
          scene: sceneInput,
          temperature: temperature, 
        }),
      });

      const data = await response.json();
      if (response.status !== 200) {
        throw data.error || new Error(`Request failed with status ${response.status}`);
      }

      setResult(data.result);
      setSceneInput("");
    } catch(error) {
      // Consider implementing your own error handling logic here
      console.error(error);
      alert(error.message);
    }
  }

  return (
    <div>
      <Head>
        <title>OpenAI Mystery Generator</title>
        <link rel="icon" href="/magglass.png" />
      </Head>

      <main className={styles.main}>
        <img src="/magglass.png" className={styles.icon} />
        <h3>Set a Mystery</h3>
        <form onSubmit={onSubmit}>
          <input
            type="text"
            name="scene"
            placeholder="Enter an location"
            value={sceneInput}
            onChange={(e) => setSceneInput(e.target.value)}
          />
          {/* Add the range slider */}
          <label htmlFor="temperature">Temperature:</label>
          <input
            type="range"
            min="0"
            max="1"
            step="0.1"
            id="temperature"
            value={temperature}
            onChange={(e) => setTemperature(e.target.value)}
          />
          <span>{temperature}</span>

          {/* Add a Styles dropdown menu */}
          <label htmlFor="style">Style:</label>
          <select id="style" value="{style}" onChange={(e) => setStyle(e.target.value)}>
            <option value="Arthur Canon Doyle">Arthur Canon Doyle</option>
            <option value="Agatha Christie">Agatha Christie</option>
            <option value="Raymond Chandler">Raymond Chandler</option>
            <option value="Dashiell Hammet">Dashiell Hammet</option>
            <option value="P.D. James">P.D. James</option>
          </select>
          <span id="selectStyle">{style}</span>

          <input type="submit" value="Generate scene" />

        </form>
        <div className={styles.result}>{result}</div>



      </main>
    </div>
  );
}

The index.js file exports a functional component that represents the home page of the web application. The component has the following components:

Head: This is a Next.js component that allows you to add elements to the head of the document. In this case, it is used to add a title and a link to a favicon to the head of the document.

useState: This is a React hook that allows you to add state to functional components. In this case, it is used to add two state variables: sceneInput and result. sceneInput is used to store the value of the text input field, and result is used to store the result of the API request.

onSubmit: This is an async function that is called when the form is submitted. It prevents the default form submission behavior, makes a POST request to the /api/generate endpoint with the sceneInput value, and stores the response in the result state variable. If there is an error during the request, it logs the error and displays an alert.

The main layout of the page consists of an image, a form with a text input field and a submit button, and a div element to display the result of the API request. The form has an onSubmit event handler that calls the onSubmit function, and the text input field has an onChange event handler that updates the sceneInput state variable with the current value of the input field.

Overall, this component allows the user to enter a location in the text input field, submit the form, and display the result of the API request on the page.

The above wasn't just code generated by ChatGPT, the descriptions were also its output. One of the best features of working with Chat is its ability to explain code and answer questions.

It also feels very personal. The interface is seductive to use. I even find myself giving it the nickname "Chat", as though it has a name.

I encourage you to give it a try when doing a project if only to see the novelty. Below you can find my project details.

github
live demo

Prometheus and Grafana, Part 2

muckitymuck — Mon, 22 Nov 2021 20:13:46 +0000

The Prometheus.yml:

# my global config
global:
  scrape_interval:     15s # Set the scrape interval to every 15 seconds. Default is every 1 minute.
  evaluation_interval: 15s # Evaluate rules every 15 seconds. The default is every 1 minute.

scrape_configs:
  # The job name is added as a label `job=<job_name>` to any timeseries scraped from this config.
  - job_name: 'prometheus'

    # metrics_path defaults to '/metrics'
    # scheme defaults to 'http'.

    static_configs:
      - targets: ['localhost:9090']

  - job_name: 'node-exporter'

    # Override the global default and scrape targets from this job every 5 seconds.
    scrape_interval: 5s
    metrics_path: /metrics

    static_configs:
      - targets:
        - <PUBLIC IP>:9100

This is a good starting place for a monitoring system for Linux boxes

Running the services in docker is also advised as it simplifies the rollout.

Docker node exporter:

sudo docker run --restart always -d --net="host" --pid="host"  -v "/:/host:ro,rslave"  prom/node-exporter:latest  --path.rootfs=/host

Docker prometheus:

sudo docker service create --replicas 1 --name dockerprometheus     --mount type=bind,source=/home/ubuntu/prometheus/prometheus.yml,destination=/etc/prometheus/prometheus.yml     --publish published=9090,target=9090,protocol=tcp     prom/prometheus

When you get around to making panels in Grafana, these queries are very useful:

100 - ((node_filesystem_avail_bytes{mountpoint="/",fstype!="rootfs"} * 100) /            node_filesystem_size_bytes{mountpoint="/",fstype!="rootfs"})

node_filesystem_avail_bytes

OR you could just setup a template for Node Exporter:
To import a new dashboard for Node Exporter, go to Dashboards -> Manage

Enter 405 for the Grafana ID.

Pick up the name and choose a Data Source for the dashboard.

To make an alert, start a NEW dashboard and NEW panel. Template dashboards like above will not work.

Enter the query for the item you want to track and press the Alert tab.
That should do it for a basic monitoring service.

Boto3: Connecting your AWS with Python, EC2 edition

muckitymuck — Mon, 22 Nov 2021 16:24:10 +0000

One way to automate your AWS administration is using python3 and its Boto3 library. It can be more secure if you make a IAM role that has limited programmatic access with temporary credentials to limit session time.
It is possible to pull information out of the EC2 instances regarding services, configs, or settings. We will go through the complete script below to explain the different parts.

import boto3

sts_client = boto3.client('sts')
assumed_role_object=sts_client.assume_role(RoleArn="arn:aws:iam::<ACCOUNTNUMBER>:role/AuthorizedRole",RoleSessionName="AssumeRoleSession1")
credentials=assumed_role_object['Credentials']
client = boto3.client('servicediscovery')

ec2_resource=boto3.resource('ec2',
        aws_access_key_id=credentials['AccessKeyId'],
        aws_secret_access_key=credentials['SecretAccessKey'],
        aws_session_token=credentials['SessionToken'],
)
productionbox = []
intbox = []
for instance in ec2_resource.instances.all():

        print(instance.id)

        for box in instance.tags:
                if box['Key'] == 'environment' and box['Value'] == 'production':
                        productionbox.append(instance.id+" : "+ instance.public_ip_address)
                elif box['Key'] =='environment' and box['Value'] != 'production':
                        intbox.append(instance.id+" : "+ instance.public_ip_address)
print("Critical: Production")
print(productionbox)
print("Other Boxes")
print(intbox)

So the first chunk deals with the initial connection settings.

sts_client = boto3.client('sts')
assumed_role_object=sts_client.assume_role(RoleArn="arn:aws:iam::<ACCOUNTNUMBER>:role/AuthorizedRole",RoleSessionName="AssumeRoleSession1")
credentials=assumed_role_object['Credentials']
client = boto3.client('servicediscovery')

The STS is AWS Security Token Service (STS). It gives the temporary, limited credentials for the service.

Using STS, you can use the assume_role to take the role of an authorized role and assign it a RoleSessionName.
From there, you can stores the Credentials of the Assumed Role and store them for use later.

ec2_resource=boto3.resource('ec2',
        aws_access_key_id=credentials['AccessKeyId'],
        aws_secret_access_key=credentials['SecretAccessKey'],
        aws_session_token=credentials['SessionToken'],
)

This stores the necessary credentials for use later. These keys and tokens will change every time you connect to preserve security.

productionbox = []
intbox = []
for instance in ec2_resource.instances.all():

        print(instance.id)

        for box in instance.tags:
                if box['Key'] == 'environment' and box['Value'] == 'production':
                        productionbox.append(instance.id+" : "+ instance.public_ip_address)
                elif box['Key'] =='environment' and box['Value'] != 'production':
                        intbox.append(instance.id+" : "+ instance.public_ip_address)
print("Critical: Production")
print(productionbox)
print("Other Boxes")
print(intbox)

This part is pretty straight forward. The ec2_resource.instances.all allows you to go through all the EC2 instances and filter what you need. The rest of the script goes through the tags on the instances and presents the instances in separate arrays that are Production environment and the other environments. You can go further and separate them down to different types or settings.

Hope this helps someone out there.

Github Actions: Build+Test in GitubVM, Deploy on Cloud

muckitymuck — Tue, 16 Nov 2021 20:07:28 +0000

An interesting new way to Build and Test in the Github environment before deploying is possible using Artifacts.

It involves breaking up 2 jobs in your workflow yml.

You will use the

actions/upload-artifact@v2
actions/download-artifact@v2

as a means to pass the artifact between the jobs.
Here, I am using NodeJS as the build, so I all I need to move is the Dist folder without Markdown.

- name: Archive production artifacts
        uses: actions/upload-artifact@v2
        with:
          name: dist-without-markdown
          path: |
            dist
            !dist/**/*.md

You also need to remember to name the artifact in case there are multiple artifacts produced in the workflow.

Finished workflow is below.

name: CI

# Controls when the workflow will run
on: 
  push:
    branches: [prod]

jobs:
  build:
    runs-on: ubuntu-20.04      
    steps:
      ### new workflow artifact
      - name: Make envfile
        uses: SpicyPizza/create-envfile@v1
        with:
          envkey_DEBUG: false
          envkey_SECRET_KEY: ${{ secrets.ENV_FILE }}
          file_name: .env
      - name: Checkout respository
        uses: actions/checkout@v2
      - name: npm install, build and test
        run: |
          npm install
          npm run build
          npm test 

      - name: Archive production artifacts
        uses: actions/upload-artifact@v2
        with:
          name: dist-without-markdown
          path: |
            dist
            !dist/**/*.md
  deploy:
    #needs will wait for Build to finish
    needs: build
    runs-on: self-hosted 
    steps:
      - name: Runs code deploy
        uses: actions/checkout@v2
        with:
          host: ${{ secrets.HOST }}
          username: ${{ secrets.USERNAME }}
          password: ${{ secrets.PASSWORD }}
          port: ${{ secrets.PORT }}
      - run: cd /path/to/deploy/
      - name: Make envfile
        uses: SpicyPizza/create-envfile@v1
        with:
          envkey_DEBUG: false
          envkey_SECRET_KEY: ${{ secrets.ENV_FILE }}
          file_name: .env

      - name: Archive production artifacts
        uses: actions/download-artifact@v2
        with:
          name: dist-without-markdown
          path: |
            /path/to/deploy/dist
      - run: npm ci
      - run: pm2 restart PROCESSNAME

Github Actions, env file creation, docker silliness

muckitymuck — Fri, 12 Nov 2021 16:20:32 +0000

To push on a branch you want to target, make a workflow yml file and include:

on: 
  push:
    branches: [prod]

You can add an env file to your build when you deploy via Github Actions and still protect the data in your Secrets.

First take your .env file and put in secrets with any name(here it is ENV_FILE).
Then add this to your workflow.yml in the build area:

      - name: Make envfile
        uses: SpicyPizza/create-envfile@v1
        with:
          envkey_DEBUG: false
          envkey_SECRET_KEY: ${{ secrets.ENV_FILE }}
          file_name: .env

If in the course you get this error:

Got permission denied while trying to connect to the Docker daemon socket at unix:///var/run/docker.sock: Post http://%2Fvar%2Frun%2Fdocker.sock/v1.24/build?

Try these:

sudo groupadd docker
sudo usermod -aG docker ${USER}
sudo chmod 666 /var/run/docker.sock

Prometheus Additional Configs

muckitymuck — Tue, 19 Oct 2021 21:55:30 +0000

This is for the prometheus.service created

[Unit]
Description=Prometheus
Wants=network-online.target
After=network-online.target

[Service]
User=ubuntu
Group=ubuntu
Type=simple
ExecStart=/usr/bin/prometheus \
    --config.file /etc/prometheus/prometheus.yml \
    --storage.tsdb.path /var/lib/prometheus/ \
    --storage.tsdb.retention=2d
    --web.console.templates=/etc/prometheus/consoles \
    --web.console.libraries=/etc/prometheus/console_libraries

[Install]
WantedBy=multi-user.target

--storage.tsdb.path is for where logs will be stored
--storage.tsdb.retention is for how long will the logs be stored

Other settings prometheus.yml

global:
  scrape_interval: 15s
  evaluation_interval: 15s
# Alertmanager configuration
alerting:
  alertmanagers:
  - static_configs:
    - targets:
      - alertmanager:9093

# Load rules once and periodically evaluate them according to the global 'evaluation_interval'.
rule_files:
  # - "first_rules.yml"
  # - "second_rules.yml"
    - "/etc/prometheus/mem_over.yml # A Rule for Alert noted below

scrape_configs:
  - job_name: 'prometheus'
    # scrape frequency
    scrape_interval: 1m
    static_configs:
      - targets: ['localhost:9090']
  - job_name: 'node'
    # scrape frequency
    scrape_interval: 1m
    static_configs:
      - targets: ['localhost:9100']

A Rule that can send alerts under conditions, mem_over.yml:

groups:
- name: Memory alarm rules
  rules:
  - alert: Memory usage alarm
    expr: (node_memory_MemTotal_bytes - (node_memory_MemFree_bytes+node_memory_Buffers_bytes+node_memory_Cached_bytes )) / node_memory_MemTotal_bytes * 100 > 80
    for: 1m
    labels:
      user: prometheus
      severity: warning
    annotations:
      description: "The server: Memory usage over 80%!(Current value: {{ $value }}%)"

AWS S3, Cloudfront, Certificates, and GoDaddy?

muckitymuck — Fri, 15 Oct 2021 19:36:28 +0000

So we are creating a static website with the following features:
-Hosted on AWS S3 bucket
-routed through Cloudfront
-SSL certs through ACM

and as a real kicker:
-Domain acquired on GoDaddy(not Route53)

First and foremost, let's fix the GoDaddy issue as managing DNS on 2 separate services is unwieldly.

Log into GoDaddy, go to Domains, and Manage All.

Find your domain and go into it. Scroll down to Additional Settings -> Manage DNS.

Now in a separate tab, go to your AWS account and into Route53. Create a Hosted Zone by the same name as the Domain on GoDaddy.
Once you finish that wizard step, make note of the NS Type records it makes for you:

Go back to your GoDaddy account and scroll down to the Nameservers section. Click on Change:

Change over to I'll Use My own nameservers and add in at least 2 server FQDN from AWS Route53:

After it goes through you should see this. Do Not Panic.

This just means that your DNS is being managed at AWS and because the resources are hosted there as well, this simplifies things.

Now we need to have content for the website.
Helpful:
https://medium.com/tensult/aws-hosting-static-website-on-s3-using-a-custom-domain-cd2782758b2c
Make two S3 buckets. One will be named after the naked domain name and the other will be the www.

The configs for this are to TURN OFF the Blocks for Public Access and to add the Bucket Policy similar to:

{
  "Version":"2012-10-17",
  "Statement":[{
    "Sid":"PublicReadGetObject",
    "Effect":"Allow",
    "Principal": "*",
    "Action":["s3:GetObject"],
    "Resource":["arn:aws:s3:::example.com/*"]
  }]
}

Also, Enable Static Website Hosting. On the naked domain S3 bucket, change the Hosting Type to Redirect requests for an object.

Add the Host Name to the www web address:

BUCKETNAME.s3-website-us-east-1.amazonaws.com

That should make them public facing and routing to the wwwbucket.
For 403 errors:
https://stackoverflow.com/questions/56244709/static-web-hosting-on-aws-s3-giving-me-403-permission-denied

Next, we need to make Cloudfront Distribution to help serve up the SSL.

This time around, I made two Distributions for the two S3 buckets. Make the Origin Names to the same as the buckets, respectively to the Origin Domains of the buckets. And add the Alternative domain Names under General to the naked domain and wwwdomain as well.
If you open the Distribution link (dxxxxxx.cloudfront.net), you should get an SSL protected site. If you get errors, sometimes you have to wait or fix the configs.
The Just Wait error:
https://stackoverflow.com/questions/42251745/aws-cloudfront-access-denied-to-s3-bucket/42285049
Now that those are running we need to make some certs.
Head over to ACM and request 1 cert for each distribution. You can probably do this with a single cert but it worked with 1 each so give it a try with a single if you care to.

You will need to make a CNAME record in Route53 for the certs you made. It will be everything before the domain name on the Name field and the everything for the value. The Certificates should turn to Successful after this is completed.

Head back to Cloudfront and find your respective Distributions. Go under Settings and update Custom SSL certificates to the certificates you made previously.
If all goes well, this should be working.

A very convoluted way to get a website up and running. Honestly an instance or a docker image is easier to setup. But the cost is near Zero except for the cost of the domain. And because you purchased that on GoDaddy the price should be down a ways.

Happy Building.

Prometheus and Grafana

muckitymuck — Mon, 04 Oct 2021 19:27:28 +0000

Monitoring services is a big part of Cloud Infrastructure Management. Here we will go through one way to install Prometheus on Ubuntu and make pretty dashboards with Grafana on a separate machine.

Go grab this download and extract to start:

wget https://github.com/prometheus/prometheus/releases/download/v2.30.3/prometheus-2.30.3.linux-amd64.tar.gz

tar -xvzf prometheus-2.11.1.linux-amd64.tar.gz

cd prometheus/

./prometheus

Should be great success:

Open a web browser and head over to the http://{IP}:9090/graph

So far, So Good
You can see the raw metrics by going to http://{IP}:9090/metrics

You can check what is up by going to /targets

Let's run it as a service, create this file:

/etc/systemd/system/prometheus.service

And give it this basic text settings:

[Unit]
Description=Prometheus Server
Documentation=https://prometheus.io/docs/introduction/overview/
After=network-online.target

[Service]
User=root
Restart=on-failure

#Change this line if you download the 
#Prometheus on different path user

ExecStart=~/prometheus/prometheus --storage.tsdb.path=/var/lib/prometheus/data/ --web.external-url=http://myurl.com:9090

[Install]
WantedBy=multi-user.target

Make sure to reload it and start it up:

sudo systemctl daemon-reload
sudo systemctl start prometheus

There are Exporters you can add to Prometheus to increase the utility. Node Exporter is one.

wget https://github.com/prometheus/node_exporter/releases/download/v0.18.1/node_exporter-0.18.1.linux-amd64.tar.gz
tar -xvzf node_exporter-0.18.1.linux-amd64.tar.gz
mv node_exporter-0.18.1.linux-amd64 node_exporter
cd node_exporter
./node_exporter

You can open the service in a browser at http://{IP}:9100

You can add the node exporter to Prometheus in /etc/prometheus/prometheus.yml

Let's make it into a service. Create a file:

sudo vi /etc/systemd/system/node_exporter.service

[Unit]
Description=Node Exporter
After=network.target
[Service]
User={user}
Group={user}
Type=simple
ExecStart=/usr/local/bin/node_exporter
[Install]
WantedBy=multi-user.target

Let's go ahead and get this going:

sudo systemctl daemon-reload
sudo systemctl start node_exporter
sudo systemctl status node_exporter

You can see the two services on /targets

If all is running well, enable this service to start at boot.

sudo systemctl enable node_exporter

That's enough for today. I will continue this in a second part for Grafana.

PM2, node, NextJS, Express

muckitymuck — Fri, 03 Sep 2021 19:02:48 +0000

A quick collection of building and monitoring commands in Linux.

To get ExpressJS started under pm2:

/home/ec2-user/expressjs/bin

pm2 start --name expressjs www

PM2
List the processes:

pm2 list

NPM build, start process

npm ci
npm test --if-present
npm run build
npm run start
pm2 start npm -- start
pm2 start --name AppName npm -- start

Restart the processes:

pm2 restart all
pm2 restart <app name>

GITHUB! Jenkins, pipeline

muckitymuck — Tue, 31 Aug 2021 19:31:41 +0000

As a record of how to set up a new CI/CD pipeline from Github to Jenkins:

A good first step is to install Jenkins on a separate machine or VM to get this started.

https://www.jenkins.io/doc/book/installing/
You can pick Linux, Docker, Kubernetes. What ever is your preference as long as you can get the webserver running.

We will start by assuming you have a github repo and you set up the credentials such as tokens correctly.

Open a new pipeline and choose Github.

Enter your github token to access it initially for the wizard.

If you are associated with multiple accounts or organizations, pick the one of your choosing and then the repo from the list provided.

The fun part starts, you can start adding steps and actions. It even comes with a choice for Git.

One you are done you can run it, but it probably won't work if the repo is private. So we will need to add the login info.

You will find this under Configure of your Pipeline.

Because of Github security you will need your Token once more. It will be protected much like Github Secrets.

We will add the target machine. Go to the main page of Jenkins.
And click on Manage Jenkins.

Give it a new name. Go back to Main Dashboard and Manage Nodes. Click on the new node.

Go under Configure and give it a name.

Add the IP address and credentials for the SSH.

This should do it to pull from github and apply to the target machine.

Terraform Cheat Sheet

muckitymuck — Fri, 13 Aug 2021 11:06:01 +0000

Will add more later:
As Main.tf

Alternatively, You can break it up into separate files:

Terraform AWS resource list

Commands:
terraform init
initialize a working directory containing Terraform configuration files

terraform fmt
formats the files to be Terraform friendly.

terraform validate
checks config files but does not check remote services

terraform plan
creates execution plan

terraform apply
executes the actions proposed in a Terraform plan

terraform destroy
tears down terraform infrastructure

For the Terraform Import method:
https://spacelift.io/blog/importing-exisiting-infrastructure-into-terraform