DEV Community: Winni Neessen

User permission checking in Go

Winni Neessen — Sat, 12 Nov 2022 21:14:30 +0000

This article was originally published on my personal blog.

The problem

While working on my go-pf module, I was faced with the problem to make sure that the /dev/pf device is readable and writable. While other languages like the Shell or Perl offer test functions to check if the current user/process has read/write/exec permissions on the specific file, in Go this task is not trivial.

Here is how we can do it with Perl:

#!/usr/bin/env perl

my $file = "test.txt";
if(!-r $file || !-w $file) {
    print $file . " is not read- and/or writable to the process\n";
    exit(1);
}
print "We're all good!\n"

If we run a quick one-liner in the Shell then, we can see how the script works:

$ for i in 100 200 400 500 600 700; do echo "chmod: $i"; \
    chmod $i test.txt; perl test.pl; done
chmod: 100
test.txt is not read- and/or writable to the process
chmod: 200
test.txt is not read- and/or writable to the process
chmod: 400
test.txt is not read- and/or writable to the process
chmod: 500
test.txt is not read- and/or writable to the process
chmod: 600
We're all good!
chmod: 700
We're all good!

In Go we would need to rely on os.Lstat which would then provide us with the Mode().Perm() method, which we could use to get the file mode of the file. But that is only half the rent. Just because the file might return a file mode of 700, which basically would mean that read and write permissions are given, the file might not be owned by us. It gets even more complicated when we need to take in account that we also have group and "other" file permissions.

The solution

Since I couldn't find a ready-made solution, I made it my task to solve the problem - preferably in an equally simple way, such as in Perl.

As mentioned, we can utilize os.Lstat as our starting point. With the file permissions and some bit shifting, we can easily figure out if a file is read/write/executable for "other", group and user. With this information at hand, the next step is to figure out the owner and the group of the file in question. For UNIX-like operating systems, Go provides us with the Sys() method on our FileInfo that is returned by os.Lstat. We can try to type-cast the result of Sys() into a *syscall.Stat_t pointer. If this succeeds, we will get the information that we need with the Uid and Gid properties.

Problem number one is solved, we can see who owns the file and check if our current user is indeed the owner. The os/user.Current method can help us with that. Our next task is group ownership - again this is not a trivial task, since a user can be part of several groups. Again the Go standard library provides us with the tools we need in form of the User.GroupIds method, which will give us a list of groups that our user is part of. With this list, it is fairly easy to figure out if our user is part of the group of the file. The second problem is solved as well. For the "other" permission we can again work off with some bit-shifting.

go-fileperm

You probably guessed already, that I built something out of all of this. go-fileperm uses all the information I discussed before into one simple-to-use Go module. Let's have a look.

First of all, we use the New() method to get an instance of the type UserPerm. New takes a string as an argument, which is the path to the file in question. If this succeeds, the module will provide you with a couple of methods to check the file access permissions (all of which return a boolean value):

UserExecutable(): returns true if the current user/process has execution permission on the file in question.
UserWritable(): returns true if the current user/process has write permission on the file in question.
UserReadable(): returns true if the current user/process has read permission on the file in question.
UserWriteExecutable(): returns true if the current user/process has write and execution permission on the file in question.
UserReadExecutable(): returns true if the current user/process has read and execution permission on the file in question.
UserWriteReadExecutable(): returns true if the current user/process has read, write and execution permission on the file in question.

Performance

Since go-fileperm mostly makes use of bit-shifting, the performance of the module is pretty fast. Also, we work allocation-free, which is always a plus :)

goos: darwin
goarch: arm64
pkg: github.com/wneessen/go-fileperm
BenchmarkPermUser_UserReadable
BenchmarkPermUser_UserReadable-8                 7364846               143.6 ns/op             0 B/op          0 allocs/op
BenchmarkPermUser_UserWritable
BenchmarkPermUser_UserWritable-8                 7803267               154.9 ns/op             0 B/op          0 allocs/op
BenchmarkPermUser_UserExecutable
BenchmarkPermUser_UserExecutable-8               7922624               149.2 ns/op             0 B/op          0 allocs/op
BenchmarkPermUser_UserWriteReadable
BenchmarkPermUser_UserWriteReadable-8            6494815               186.1 ns/op             0 B/op          0 allocs/op
BenchmarkPermUser_UserWriteExecutable
BenchmarkPermUser_UserWriteExecutable-8          6590229               181.0 ns/op             0 B/op          0 allocs/op
BenchmarkPermUser_UserReadExecutable
BenchmarkPermUser_UserReadExecutable-8           6190532               184.7 ns/op             0 B/op          0 allocs/op
BenchmarkPermUser_UserWriteReadExecutable
BenchmarkPermUser_UserWriteReadExecutable-8      5728713               208.8 ns/op             0 B/op          0 allocs/op
PASS

Full code example

Let's check out a full code example and apply our one-liner that we used in the beginning.

package main

import (
    "fmt"
    "log"

    "github.com/wneessen/go-fileperm"
)

func main() {
    p, err := fileperm.New("test.txt")
    if err != nil {
        log.Fatalf("failed to create UserPerm instance: %s", err)
    }
    fmt.Printf("test.txt is user-readable:              %t\n", p.UserReadable())
    fmt.Printf("test.txt is user-writable:              %t\n", p.UserWritable())
    fmt.Printf("test.txt is user-executable:            %t\n", p.UserExecutable())
    fmt.Printf("test.txt is user-read/writable:         %t\n", p.UserWriteReadable())
    fmt.Printf("test.txt is user-read/executable:       %t\n", p.UserReadExecutable())
    fmt.Printf("test.txt is user-write/executable:      %t\n", p.UserWriteExecutable())
    fmt.Printf("test.txt is user-read/write/executable: %t\n", p.UserWriteReadExecutable())
}

$ for i in 100 200 400 500 600 700; do echo "chmod: $i"; \
    chmod $i test.txt; go run main.go; done

chmod: 100
test.txt is user-readable:              false
test.txt is user-writable:              false
test.txt is user-executable:            true
test.txt is user-read/writable:         false
test.txt is user-read/executable:       false
test.txt is user-write/executable:      false
test.txt is user-read/write/executable: false
chmod: 200
test.txt is user-readable:              false
test.txt is user-writable:              true
test.txt is user-executable:            false
test.txt is user-read/writable:         false
test.txt is user-read/executable:       false
test.txt is user-write/executable:      false
test.txt is user-read/write/executable: false
chmod: 400
test.txt is user-readable:              true
test.txt is user-writable:              false
test.txt is user-executable:            false
test.txt is user-read/writable:         false
test.txt is user-read/executable:       false
test.txt is user-write/executable:      false
test.txt is user-read/write/executable: false
chmod: 500
test.txt is user-readable:              true
test.txt is user-writable:              false
test.txt is user-executable:            true
test.txt is user-read/writable:         false
test.txt is user-read/executable:       true
test.txt is user-write/executable:      false
test.txt is user-read/write/executable: false
chmod: 600
test.txt is user-readable:              true
test.txt is user-writable:              true
test.txt is user-executable:            false
test.txt is user-read/writable:         true
test.txt is user-read/executable:       false
test.txt is user-write/executable:      false
test.txt is user-read/write/executable: false
chmod: 700
test.txt is user-readable:              true
test.txt is user-writable:              true
test.txt is user-executable:            true
test.txt is user-read/writable:         true
test.txt is user-read/executable:       true
test.txt is user-write/executable:      true
test.txt is user-read/write/executable: true

Conclusion

As you can see, the module is pretty easy to use and hopefully will be a helpful tool for some of you. As usual, I've published it on GitHub under the MIT license, so feel free to give it a try or even contribute if you think there is things that need improvements.

Sending mails in Go the easy way

Winni Neessen — Sat, 08 Oct 2022 11:36:03 +0000

I recently read this article on dev.to about "How to Send Emails in Go". It seems to be a copy from a blog post from mailtrap.io.

While the article is certainly not wrong on what is written there, it seems like its main intention is to make the point that it's easier to use a 3rd party mail service - which is perfectly fine... yet, I found the conclusion a bit baffleing. In the article it says:

This wraps up our rather quick guide to sending emails in Golang. As you can see, the choice comes down to either utilizing basic in-built functionalities of Go or connecting to 3rd parties to send emails on your behalf.

I have to disagree with this, since there are lots of good Go mail libaries that really make your job of sending mail so much easier.

go-mail makes it easy for you

As backbone for my form mailing microservice js-mailer, I've created go-mail as a state of the art Go mail libarary that is comprehensive, yet easy to use.

go-mail consists of two main components. The Msg representing the mail message and everything related to it and the Client which is handling the mail server communication and the delivery of the Msg.

Since the original article first mentions smtp.SendMail, let's have a look at how we would archive the same with go-mail.

The original example creates a simple mail from bill@gates.com delivered to bill@gates.com with a simple subject why are you not using Mailtrap yet? and a mail body saying Here’s the space for our great sales pitch. The whole thing is delivered via SMTP server smtp.mailtrap.io:25 and using SMTP Plain Authentication.

With go-mail it would look like this:

package main

import (
    "github.com/wneessen/go-mail"
    "log"
)

func main() {
    // First we create a mail message
    m := mail.NewMsg()
    if err := m.From("bill@gates.com"); err != nil {
        log.Fatalf("failed to set From address: %s", err)
    }
    if err := m.To("bill@gates.com"); err != nil {
        log.Fatalf("failed to set To address: %s", err)
    }
    m.Subject("Why are you not using go-mail yet?")
    m.SetBodyString(mail.TypeTextPlain, "You won't need a sales pitch. It's FOSS.")

    // Secondly the mail client
    c, err := mail.NewClient("smtp.mailtrap.io", 
        mail.WithPort(25), mail.WithSMTPAuth(mail.SMTPAuthPlain), 
        mail.WithUsername("piotr@mailtrap.io"), mail.WithPassword("extremely_secret_pass"))
    if err != nil {
        log.Fatalf("failed to create mail client: %s", err)
    }

    // Finally let's send out the mail
    if err := c.DialAndSend(m); err != nil {
        log.Fatalf("failed to send mail: %s", err)
    }
}

At first glance this might look like much more code, but while with net/smtp and smtp.SendMail you need to make sure that all parameters (like mail addresses) are syntactically right and line breaks are put in correctly, go-mail will take care of all of this for you. Additionally the syntax is self-explainatory and easy to understand.

Authentication

The original article talks about the authentication methods that can be used. We make use of the smtp.Auth interface, which allows the user to implement their own authentication methods and use them with go-mail without issues.

This means, analogous to net/smtp, go-mail supports SMTP PLAIN and CRAM-MD5 authentication. Additionally I've also added support for SMTP LOGIN (which is similar to SMTP PLAIN but not the same).

TLS

Nowadays, you want to make sure that your authentication is secure. SMTP PLAIN and LOGIN basically send your username and password in plaintext over the wire. Therfore we need to make use of transport encryption using TLS.

I am a firm believer in strong encryption by default and that nowadays, with all the easy tools like Let's Encrypt and co., there is basically no excuse to not use TLS for any connection anymore. For that reason, go-mail defaults to mandatory TLS in the client - which means the client will fail if the server does not support STARTTLS for the connection. Still you have the option to set the client to opportunistic TLS (use TLS if available, otherwise don't) or disable TLS completely.

To do so, all you need to do is giving the NewClient() method an additional option.

Re-using the code from our initial example, it would look like this:

c, err := mail.NewClient("smtp.mailtrap.io", 
    mail.WithPort(25), mail.WithSMTPAuth(mail.SMTPAuthPlain), 
    mail.WithUsername("piotr@mailtrap.io"), 
    mail.WithPassword("extremely_secret_pass"),
    mail.WithTLSPolicy(mail.TLSOpportunistic)))

Multiple recipients and (B)CC

Next the article mentions that things get a bit more sophisticated when it comes to mutliple recipients. Not for go-mail though. The Msg.To() method allows to provide multiple recipients. So for our initial example all we would need to do is the following:

if err := m.To("bill@gates.com", "stevie@microsoft.com"); err != nil {
    log.Fatalf("failed to set To address: %s", err)
}

Easy, isn't it?

The original article now wants to seperate the 2nd address from the To-Address list. No problem! We have Msg.Cc() in go-mail. Which makes sure that envelope address header and mail address header are setup correctly for this operation. Here is what our code would look now:

if err := m.To("bill@gates.com"); err != nil {
    log.Fatalf("failed to set To address: %s", err)
}
if err := m.Cc("stevie@microsoft.com"); err != nil {
    log.Fatalf("failed to set CC address: %s", err)
}

We still get the benefit from go-mails syntax checks and we don't need to care about any newlines or formating.

Finally the article doesn’t want Bill to know that Steve knows. So they separate envelope recipients from the mail header recipients - which makes the code look really confusing in my opinion. go-mail has a much more comfortable solution: Msg.Bcc(). Same as you would set a blind carbon copy recipient in your mail client, go-mail takes care of all the bells and whistles.

Here is our new code snippet:

if err := m.To("bill@gates.com"); err != nil {
    log.Fatalf("failed to set To address: %s", err)
}
if err := m.Bcc("stevie@microsoft.com"); err != nil {
    log.Fatalf("failed to set BCC address: %s", err)
}

Yes, it's really as easy as that.

Quality of Life

When I initially created go-mail one of my main goals was to make go-mail act similar to a normal mail user agent (MUA). Hence go-mail has a lot of - what I would call - "Quality of Life" methods. Things that you would likly be able to do yourself without go-mail providing these methods but it's so much easier to not having to think about it. Let's for example look at mail importance.

Mails can have importance flags. Unforuntately not every mail client handles those in the same way. It's all depending on the correct mail headers and the values you provide them. To make the user's life easier, go-mail provides a Msg.SetImportance() method. All you need to do is provide the importance you like to set for your message and go-mail will take care of the rest.

Example:

m.SetImportance(mail.ImportanceHigh)

We can do much more

In the final chapter of the article it switches from Go's own tools to using 3rd party APIs for sending mails, giving you access to i. e. "beautiful HTML mails" or "message previews". Let's address those two points in terms of go-mail.

HTML mails

go-mail is built with full MIME and multipart support. This means it takes care of proper MIME encoding as well as being able to handle multiple mail parts like mail body + attachment. Not only this, we have built in support for Go's text/template and html/template systems.

To send a HTML mail, all we would need to do is change one line in our initial example code:

Replace:

m.SetBodyString(mail.TypeTextPlain, "You won't need a sales pitch. It's FOSS.")

with:

m.SetBodyString(mail.TypeTextHTML, "<h1>You won't need a sales pitch. It's FOSS.</h1>")

Now let's assume we live in a modern world, where mails not only consist of plain text or HTML, but we let the client decide what to show instead. For this we can make use of alternative mail body parts. Let's have a look how this would look in code:

m.SetBodyString(mail.TypeTextHTML, "<h1>You won't need a sales pitch. It's FOSS and HTML!</h1>")
m.AddAlternativeString(mail.TypeTextPlain, "You won't need a sales pitch. It's FOSS and plain text!")

A more advanced example would be the use of HTML templates. Let's have a quick look on how this would work.

import "html/template"
[...]
type MyStruct struct {
    Placeholder string
}
data := MyStruct{Placeholder: "Teststring"}
tpl, err := template.New("test").Parse("This is a {{.Placeholder}}")
if err != nil {
    log.Fatalf("failed to parse HTML template: %s", err)
}
if err := m.SetBodyHTMLTemplate(tpl, data); err != nil {
    log.Fatalf("failed to set HTML template mail body: %s", err)
}

Here we create a new example struct that would represent our template data. We create a new html/template template and add some example text with placeholders that would fit our template data. All that's left is to use Msg.SetBodyHTMLTemplate() with our template and data and go-mail will take care of the rest.

In fact go-mail has support for many different kinds of attachments. Be it simple files from the local file system, from an io.Reader interface, from an embed.FS or the already mentioned template packages.

Message preview

I already mentioned that go-mail consists of two main components. The Client and the Msg. The beauty of this is, that we have implemented a io.WriteTo interface into the Msg. This means, we get access to our mail message without having to pump it through our Client. Not only that, we also have methods on the Msg which allows us to store the mail directly into a file or push it to a io.Writer interface. Let's have a closer look...

If you work on a unix like OS (Linux, *BSD, MacOS) you probably know that there a special devices for standard input and output. One of them is /dev/stdout which is your standard output on a terminal console. In Go this interface is represented by the os.Stdout file pointer. Since it satisfies the io.Writer interface, it makes it super easy for us to preview a mail.

Check this out:

if _, err := m.WriteTo(os.Stdout); err != nil {
    log.Fatalf("failed to write mail to STDOUT: %s", err)
}

We tell go-mail to write our mail message to STDOUT and it will perform all it's magic first and just pump the whole output into the STDOUT file. The output would look similar to this in my linux console:

$ go run main.go

Date: Sat, 08 Oct 2022 12:47:50 +0200
MIME-Version: 1.0
Message-ID: <116004.6075668864260473870.1665226070@arch-vm.local.host>
Subject: Why are you not using go-mail yet?
User-Agent: go-mail v0.2.9 // https://github.com/wneessen/go-mail
X-Mailer: go-mail v0.2.9 // https://github.com/wneessen/go-mail
From: <bill@gates.com>
To: <bill@gates.com>
Content-Type: multipart/alternative;
 boundary=9c5a427bc18e45ff56e0b8f7053cddceaca54a5c985a425213544ab7977f

--9c5a427bc18e45ff56e0b8f7053cddceaca54a5c985a425213544ab7977f
Content-Transfer-Encoding: quoted-printable
Content-Type: text/html; charset=UTF-8

<h1>You won't need a sales pitch. It's FOSS and HTML!</h1>
--9c5a427bc18e45ff56e0b8f7053cddceaca54a5c985a425213544ab7977f
Content-Transfer-Encoding: quoted-printable
Content-Type: text/plain; charset=UTF-8

You won't need a sales pitch. It's FOSS and plain text!
--9c5a427bc18e45ff56e0b8f7053cddceaca54a5c985a425213544ab7977f--

See how go-mail internally took care of the encoding, the formating, the different mail parts? That's the beauty of this libary. You, the user, only needs to take care of what is important to you and go-mail makes sure it complies to the corresponding mail RFCs.

Local mail server

Sometimes you don't have an external mail service to sent you mails from and all you want to do is using a local sendmail client to deliver the mail. go-mail has you covered here as well. We provide access to local sendmail programs using its own Msg.WriteToSendmail() method (or Msg.WriteToSendmailWithCommand() if your sendmail binary is not found in /usr/sbin/sendmail).

We can basically use the same code syntax as in our previous example, just with the new method:

if _, err := m.WriteToSendmail(); err != nil {
    log.Fatalf("failed to write mail to local sendmail: %s", err)
}

Middleware

One of the principles that I introduced when I wrote go-mail was that I want to only rely on the Go standard library and not introduce any 3rd party code. This we go-mail can concentrate on it's main functionality: sending mails.

With the v0.2.8 release, we received a PR for an awesome feature though: Msg.Middleware. The middleware interface allows the user to implement their own methods on the Msg without having the restriction of only the Go standard library. The interface is super easy:

type Middleware interface {
    Handle(*Msg) *Msg
}

All your code needs to do is provide a Handle method that takes a Msg pointer and in the end returns the Msg pointer again. What happens with the mail message in between is totally up to you.

Inspired by this cool feature, I started an additional GH repository called go-mail-middleware, which is supposed to be a collection of useful middlwares that interact with go-mail. For now I've only added a simple subject capitalization middleware but I am hoping that other users will provide cool stuff there as well in the future. If you have a cool idea for a mail middleware, a PR is more than welcome.

Summary

As you can hopefully see, go-mail is a powerful tool in your Go toolbox. It not only makes sure that your mails comply with the mail RFCs it also has many small "Quality of Life" features which will make your work with mails in Go so much easier. Check out the full documentation and find out how go-mail can make your life easier.

The project might be relatively new but we already have a healthy community and according to the Github stats, a couple of projects already rely on go-mail for their code.

We also have a discord channel on the official Gopher Discord server. Feel free to stop by and say hello!

Sending (form-)mails from static websites with Go

Winni Neessen — Thu, 25 Aug 2022 15:59:37 +0000

From HTML to Wordpress to Hugo

For over 7 years, I've been maintaining the website of a non-profit organisation. When I took over this task from the previous webmaster, the page was a collection of manually generated and maintained HTML pages. Since I didn’t want to manually write HTML on each page for every change that needed to be done, I decided to rebuild the whole site in Wordpress.

Even though this worked pretty well, over the years I started to dislike Wordpress. Also with all the needed plugins, the site became horribly slow (4-5 seconds per requests - I’ve never really figured out which plugin caused this slowness). So eventually I decided to replace the Wordpress with a static page generator: Hugo.

Sending mails from HTML forms

Hugo is great and fast and I really enjoy working with it. It was the right decision. Now, the page is responsive, fast and easy to maintain.

Yet, one thing I couldn’t easily accomplish with Hugo was sending mails from contact forms - basically a form mailer. With Wordpress, since it’s PHP under the hood on the server side, this is pretty easy. But with Hugo, the pages are static HTML.

So I had to use JavaScript and some kind of form mailer to send contact form requests. There are 3rd party services like https://formspree.io/ that are specialized on this, but since I don’t wanna invest money just to send forms and I also don’t trust the data privacy of such services (GDPR is a big thing in the EU you know 😉), I decided to write my own little web service for it.

Introducing js-mailer

js-mailer is a very simple, yet powerful, web service writting in Go that allows static websites to send forms via JavaScript’s fetch() or XMLHttpRequest APIs.

It follows a two-step workflow. First your JavaScript requests a token from the API using the /api/v1/token endpoint. If the request is valid and website is authorized to request a token, the API will respond with a TokenResponseJson. This holds some data, which needs to be included into your form as hidden inputs. It will also provide a submission URL endpoint /api/v1/send/<formid>/<token> that can be used as action in your form. Once the form is submitted, the API will then validate that all submitted data is correct and submit the form data to the configured recipients.

What started as a quick and dirty “I want to send HTML forms as mail” service, quickly transformed into a fully flegged web service. At the time of this post, the service supports the following features:

Single-binary webservice
Multi-form support
Multiple recipients per form
Only display form-fields that are configured in for the form in the resulting mail
Check for required form fields
Anti-SPAM functionality via built-in, auto-expiring and single-use security token feature
Anti-SPAM functionality via honeypot fields
Limit form access to specific domains
Per-form mail server configuration
hCaptcha support
reCaptcha v2 support
Form field type validation (text, email, number, bool)
Confirmation mail to poster
Custom Reply-To header based on sending mail address

It is FOSS and can be found on Github. Maybe it can help you with a similar problem.

Sending mails with Go

In the first iterations of js-mailer I used the popular go-gomail/gomail package for handling the mail sending part. Unfortunately the project isn't maintained anymore and uses some pretty outdated opinions on how to handle the mail sending.

Due to the above mentioned reasons and the lack of proper alternatives decided to write my own Go mail library. On a long weekend I started working on wneessen/go-mail and in the night of Sunday I had a kind of working prototype and I implemented it with js-mailer.

Meanwhile go-mail has grown to what I would call a pretty comprehensive library for all kinds of "sending mails with Go" tasks and the Github issues page shows that people start using it and request things they'd like to see as part of go-mail. The feature list of go-mail currently holds these pinpoints:

Only Standard Library dependant
Modern, idiomatic Go
Sane and secure defaults
Explicit SSL/TLS support
Implicit StartTLS support with different policies
Makes use of contexts for a better control flow and timeout/cancelation handling
SMTP Auth support (LOGIN, PLAIN, CRAM-MD)
RFC5322 compliant mail address validation
Support for common mail header field generation (Message-ID, Date, Bulk-Precedence, Priority, etc.)
Reusing the same SMTP connection to send multiple mails
Support for attachments and inline embeds (from file system, io.Reader or embed.FS)
Support for different encodings
Support sending mails via a local sendmail command
Message object satisfies io.WriteTo and io.Reader interfaces
Support for Go's html/template and text/template (as message body, alternative part or attachment/emebed)
Output to file support which allows storing mail messages as e. g. .eml files to disk to open them in a MUA

So if you need to send mails in Go, I recommend giving it a quick look and see if it can help you accomlish your "mail tasks" easier. Feedback, bug reports, fixed and PRs are of course always welcome.