Hunting space gobblers in Ubuntu OS.

Aneesh Anil — Mon, 27 Jun 2022 03:21:51 +0000

Introduction

The other day one of my slack channels started buzzing. It said our DGraph database was down.This seems to happen sometimes, usually restarting works. But here it wasn't!
2 seconds after a restart, it goes down again.

No Space

One thing I noticed while I was logged into the droplet where Dgraph was hosted, was the OS wasn't able to autocomplete my commands on pressing tab key. So there was no space.
I was stumped, what was happening ?!?

So I ran

df -h which showed

Filesystem      Size  Used Avail Use% Mounted on
.
.
/dev/vda1        49G  45G   45G   100% /
.
.

Whoa! What is /dev/vda1 and why is it's usage at 100% ?

The hunt

So I used du -h --max-depth=1 from root /, and I noticed /var had a huge usage.
Moved into /var and ran the same, now log had a huge usage.
Turns out it was syslog that was using up the space.

Solution

Went ahead and truncated syslog now, /dev/vda1 is at 9% usage.
We also cleaned up /tmp.
And set the logs to rotate.

Recommendation

We didn't see a need to backup the syslog but it's recommended that anybody clearing it up do backup for later analysis on why it fills up, if that happens regularly.
In our case the server has been running for quite some time, some it was just a natural accumulation of logs.

Another potentially useful command

du -sh [/path]

References

How to deleted syslog safely. [https://stackoverflow.com/questions/35638219/ubuntu-large-syslog-and-kern-log-files]

Using Cloud Firestore Emulator for local development

Aneesh Anil — Sun, 02 Jan 2022 13:55:52 +0000

I want to build a small project, I would like to host it as well. So I did some digging and found out Google Cloud offers free tier upto a limit. This is awesome, all I need is a laptop and an internet connection and I can create a small startup from home!

My basic concern is where to store data? Cloud Firestore is a NoSQL document database.

Project Setup

I have never worked with NoSQL databases, so I wanted to try it. The setup is pretty straightforward, go to Cloud Console and Click Add Project and follow the flow to setup a project. Once that is complete navigate to the Cloud Firestore section of the Firebase console. You'll be prompted to select an existing Firebase project. Follow the database creation workflow. Google has super docs on how to setup project and firestore.

Now we don't want to use this database for development, we would like to keep that for prod use only, that is where emulator comes in.

Installing Emulator

To install the emulator Firebase CLI needs to installed already. Check if it is installed by running the command firebase --version if that showed a version you are good otherwise proceed to install it. For Linux/MacOS running the auto-install script (curl -sL https://firebase.tools | bash) is the simplest way to do that. For more info and on how to install in Windows OS check out this docs.
Once the script completes successfully check if the installation is successful or not by using the command firebase --version. Then navigate to your project and init firebase firebase init. Then initialise emulators firebase init emulators and select Firebase from options.

To start the emulator locally use cmd firebase emulators:start and in the browser open the Emulator UI.

Now to connect to the Firestore emulator using Go, download the dependency go get firebase.google.com/go.

// imports
...
"cloud.google.com/go/firestore"
firebase "firebase.google.com/go"
"google.golang.org/api/option"
...

// Use the application default credentials
ctx := context.Background()
conf := &firebase.Config{ProjectID: os.Getenv("GCP_PROJECT_ID")}
app, err := firebase.NewApp(ctx, conf)
if err != nil {
    log.Fatalln(err)
}

client, err := app.Firestore(ctx)
if err != nil {
    log.Fatalln(err)
}
return client

Now use that client to talk to the database

_, _, err := client.Collection("users").Add(ctx, map[string]interface{}{
    "first":  "Alan",
    "middle": "Mathison",
    "last":   "Turing",
    "born":   1912,
})
if err != nil {
    log.Fatalf("Failed adding aturing: %v", err)
}

You should see this document in the Emulator UI.

Persisting the data

When we exit the emulator we loose the data.
To persist the data across multiple sessions use firebase emulators:start --import <export-directory> --export-on-exit will import the data on startup and save the data on exit.

A more detailed explanation can be found here.

DEV Community: Aneesh Anil