DEV Community

loading...
Cover image for How to securely store passwords?

How to securely store passwords?

techschoolguru profile image TECH SCHOOL ・13 min read

Hello everyone, welcome back to the backend master class!

In this lecture, we’re gonna learn how to securely store users’ password in the database.

Here's:

How to store password

As you already know, we should never ever store naked passwords! So the idea is to hash it first, and only store that hash value.

Basically, the password will be hashed using brypt hashing function to produce a hash value.

Besides the input password, bcrypt requires a cost parameter, which will decide the number of key expansion rounds or iterations of the algorithm.

Bcrypt also generates a random salt to be used in those iterations, which will help protect against the rainbow table attack. Because of this random salt, the algorithm will give you a completely different output hash value even if the same input password is provided.

The cost and salt will also be added to the hash to produce the final hash string, which looks something like this:

Alt Text

In this hash string, there are 4 components:

  • The first part is the hash algorithm identifier. 2A is the identifier of the bcrypt algorithm.
  • The second part is the cost. In this case, the cost is 10, which means there will be 2^10 = 1024 rounds of key expansion.
  • The third part is the salt of length 16 bytes, or 128 bits. It is encoded using base64 format, which will generate a string of 22 characters.
  • Finally, the last part is the 24 bytes hash value, encoded as 31 characters.

All of these 4 parts are concatenated together into a single hash string, and it is the string that we will store in the database.

Alt Text

So that’s the process of hashing users’ password!

But when users login, how can we verify that the password that they entered is correct or not?

Well, first we have to find the hashed_password stored in the DB by username.

Then we use the cost and salt of that hashed_password as the arguments to hash the naked_password users just entered with bcrypt. The output of this will be another hash value.

Then all we have to do is to compare the 2 hash values. If they’re the same, then the password is correct.

Alt Text

Alright, now let’s see how to implement these logics in Golang.

Implement functions to hash and compare passwords

In the previous lecture, we have generated the code to create a new user in the database. And hashed_password is one of the input parameters of the CreateUser() function.

type CreateUserParams struct {
    Username       string `json:"username"`
    HashedPassword string `json:"hashed_password"`
    FullName       string `json:"full_name"`
    Email          string `json:"email"`
}

func (q *Queries) CreateUser(ctx context.Context, arg CreateUserParams) (User, error) {
    row := q.db.QueryRowContext(ctx, createUser,
        arg.Username,
        arg.HashedPassword,
        arg.FullName,
        arg.Email,
    )
    var i User
    err := row.Scan(
        &i.Username,
        &i.HashedPassword,
        &i.FullName,
        &i.Email,
        &i.PasswordChangedAt,
        &i.CreatedAt,
    )
    return i, err
}
Enter fullscreen mode Exit fullscreen mode

Also, in this createRandomUser() function of the unit test in db/sqlc/user_test.go, we’re using a simple "secret" string for the hash_password field, which doesn’t reflect the real correct values this field should hold.

func createRandomUser(t *testing.T) User {
    arg := CreateUserParams{
        Username:       util.RandomOwner(),
        HashedPassword: "secret",
        FullName:       util.RandomOwner(),
        Email:          util.RandomEmail(),
    }

    ...
}
Enter fullscreen mode Exit fullscreen mode

So today we’re gonna update it to use a real hash string.

Hash password function

First, let’s create a new file password.go inside the util package. In this file, I’m gonna define a new function: HashPassword().

It will take a password string as input, and will return a string or an error. This function will compute the bcrypt hash string of the input password.

// HashPassword returns the bcrypt hash of the password
func HashPassword(password string) (string, error) {
    hashedPassword, err := bcrypt.GenerateFromPassword([]byte(password), bcrypt.DefaultCost)
    if err != nil {
        return "", fmt.Errorf("failed to hash password: %w", err)
    }
    return string(hashedPassword), nil
}
Enter fullscreen mode Exit fullscreen mode

In this function, we call bcrypt.GenerateFromPassword(). It requires 2 input parameters: the password of type []byte slice, and a cost of type int.

So we have to convert the input password from string to []byte slice.

For cost, I use the bcrypt.DefaultCost value, which is 10.

The output of this function will be the hashedPassword and an error. If the error is not nil, then we just return an empty hashed string, and wrap the error with a message saying: "failed to hash password".

Otherwise, we convert the hashedPassword from []byte slice to string, and return it with a nil error.

Compare passwords function

Next, we will write another function to check if a password is correct or not: CheckPassword().

This function will take 2 input arguments: a password to check, and the hashedPassword to compare. It will return an error as output.

Basically, this function will check if the input password is correct when comparing to the provided hashedPassword or not.

As the standard bcrypt package has already implemented this feature, all we have to do is to call bcrypt.CompareHashAndPassword() function, and pass in the hashedPassword and naked password, after converting them from string to []byte slices.

// CheckPassword checks if the provided password is correct or not
func CheckPassword(password string, hashedPassword string) error {
    return bcrypt.CompareHashAndPassword([]byte(hashedPassword), []byte(password))
}
Enter fullscreen mode Exit fullscreen mode

And that’s it. We’re done!

Write unit test for HashPassword and CheckPassword functions

Now let’s write some unit tests to make sure these 2 functions work as expected.

I’m gonna create a new file password_test.go inside the util package. Then let’s define function TestPassword() with a testing.T object as input.

First I will generate a new password as a random string of 6 characters. Then we get the hashedPassword by calling HashPassword() function with the generated password.

We require no errors to be returned, and the hashedPassword string should be not empty.

func TestPassword(t *testing.T) {
    password := RandomString(6)

    hashedPassword, err := HashPassword(password)
    require.NoError(t, err)
    require.NotEmpty(t, hashedPassword)

    err = CheckPassword(password, hashedPassword1)
    require.NoError(t, err)
}
Enter fullscreen mode Exit fullscreen mode

Next we call CheckPassword() function with the password and hashedPassword parameters.

As this is the same password we used to create the hashedPassword, this function should return no errors, which means correct password.

Let’s also test the case where an incorrect password is provided!

I will generate a new random wrongPassword string, and call CheckPassword() again with this wrongPassword argument. This time, we expect an error to be returned, since the provided password is incorrect.

func TestPassword(t *testing.T) {
    password := RandomString(6)

    hashedPassword, err := HashPassword(password)
    require.NoError(t, err)
    require.NotEmpty(t, hashedPassword)

    wrongPassword := RandomString(6)
    err = CheckPassword(wrongPassword, hashedPassword)
    require.EqualError(t, err, bcrypt.ErrMismatchedHashAndPassword.Error())
}
Enter fullscreen mode Exit fullscreen mode

To be exact, we use require.EqualError() to compare the output error. It must be equal to the bcrypt.ErrMismatchedHashAndPassword error.

OK, the test is now completed. Let’s run it!

Alt Text

It passed! Awesome!

Update the existing code to use HashPassword function

So the HashPassword() function is working properly. Let’s go back to the user_test.go file and use it in the createRandomUser() function.

Here I’m gonna create a new hashedPassword value by calling util.HashPassword() function with a random string of 6 characters.

We require no errors, then change the "secret" constant to hashedPassword instead:

func createRandomUser(t *testing.T) User {
    hashedPassword, err := util.HashPassword(util.RandomString(6))
    require.NoError(t, err)

    arg := CreateUserParams{
        Username:       util.RandomOwner(),
        HashedPassword: hashedPassword,
        FullName:       util.RandomOwner(),
        Email:          util.RandomEmail(),
    }

    ...
}
Enter fullscreen mode Exit fullscreen mode

Alright, let’s run the whole db package test!

Alt Text

All passed!

Now if we open the database in Table Plus and check the users table, we can see that the hashed_password column is now containing the correct bcrypt hashed string.

Alt Text

It looks just like the example that I shown you in the beginning of this video.

Make sure all hashed passwords are different

One thing we want to make sure of is: if the same password is hashed twice, 2 different hash values should be produced.

So let’s go back to the TestPassword() function. I’m gonna change the hashPassword variable’s name to hashedPassword1.

Then let’s duplicate the hash password code block, and change the variable’s name to hashedPassword2.

func TestPassword(t *testing.T) {
    password := RandomString(6)

    hashedPassword1, err := HashPassword(password)
    require.NoError(t, err)
    require.NotEmpty(t, hashedPassword1)

    err = CheckPassword(password, hashedPassword1)
    require.NoError(t, err)

    wrongPassword := RandomString(6)
    err = CheckPassword(wrongPassword, hashedPassword1)
    require.EqualError(t, err, bcrypt.ErrMismatchedHashAndPassword.Error())

    hashedPassword2, err := HashPassword(password)
    require.NoError(t, err)
    require.NotEmpty(t, hashedPassword2)
    require.NotEqual(t, hashedPassword1, hashedPassword2)
}
Enter fullscreen mode Exit fullscreen mode

What we expect to see is: the value of hashedPassword2 should be different from the value of hashedPassword1. So here I use require.NotEqual() to check this condition.

OK, let’s rerun the test.

Alt Text

It passed! Excellent!

To really understand why it passed, we have to open the implementation of the bcrypt.GenerateFromPassword() function.

func GenerateFromPassword(password []byte, cost int) ([]byte, error) {
    p, err := newFromPassword(password, cost)
    if err != nil {
        return nil, err
    }
    return p.Hash(), nil
}

func newFromPassword(password []byte, cost int) (*hashed, error) {
    if cost < MinCost {
        cost = DefaultCost
    }
    p := new(hashed)
    p.major = majorVersion
    p.minor = minorVersion

    err := checkCost(cost)
    if err != nil {
        return nil, err
    }
    p.cost = cost

    unencodedSalt := make([]byte, maxSaltSize)
    _, err = io.ReadFull(rand.Reader, unencodedSalt)
    if err != nil {
        return nil, err
    }

    p.salt = base64Encode(unencodedSalt)
    hash, err := bcrypt(password, p.cost, p.salt)
    if err != nil {
        return nil, err
    }
    p.hash = hash
    return p, err
}
Enter fullscreen mode Exit fullscreen mode

As you can see here, in the newFromPassword() function, a random salt value is generated, and it is used in the bcrypt() function to generate the hash.

So now you know, because of this random salt, the generated hash value will be different everytime.

Implement the create user API

Next step, I’m gonna use the HashPassword() function that we’ve written to implement the create user API for our simple bank.

Let’s create a new file user.go inside the api package.

This API will be very much alike the create account API that we’ve implemented before, so I’m just gonna copy it from the api/account.go file.

Then let’s change this struct to createUserRequest.

The first parameter is username. It is a required field.

And let’s say we don’t allow it to contain any kind of special characters, so here I’m gonna use the alphanum tag, which is already provided by the validator package. It basically means that this field should contain ASCII alphanumeric characters only.

The second field is password. It is also required. And normally we don’t want the password to be too short because it would be very easy to hack. So here let’s use the min tag to say that the length of the password should be at least 6 characters.

type createUserRequest struct {
    Username string `json:"username" binding:"required,alphanum"`
    Password string `json:"password" binding:"required,min=6"`
    FullName string `json:"full_name" binding:"required"`
    Email    string `json:"email" binding:"required,email"`
}
Enter fullscreen mode Exit fullscreen mode

The third field is full_name of the user. There’s no specific requirements for this field, except that it is required.

Then, the last field is email, which is very important because it would be the main communication channel between the users and our system. We can use the email tag provided by validator package to make sure that the value of this field is a correct email address.

There are many other useful built-in tags that were already implemented by the validator package, you can check them out in its documentation or github page.

Now let’s go back to the code to complete this createUser() function.

func (server *Server) createUser(ctx *gin.Context) {
    var req createUserRequest
    if err := ctx.ShouldBindJSON(&req); err != nil {
        ctx.JSON(http.StatusBadRequest, errorResponse(err))
        return
    }

    hashedPassword, err := util.HashPassword(req.Password)
    if err != nil {
        ctx.JSON(http.StatusInternalServerError, errorResponse(err))
        return
    }

    arg := db.CreateUserParams{
        Username:       req.Username,
        HashedPassword: hashedPassword,
        FullName:       req.FullName,
        Email:          req.Email,
    }

    ...   
}
Enter fullscreen mode Exit fullscreen mode

Here we use the ctx.ShouldBindJSON() function to bind the input parameters from the context into the createUserRequest object.

If any of the parameters are invalid, we just return 400 Bad Request status to the client. Otherwise, we will use them build the db.CreateUserParams object.

There are 4 fields that need to be set: Username, HashedPassword, Fullname, and Email.

So first, we compute the hashedPassword by calling util.HashPassword() function and pass in the input request.Password value.

If this function returns a not nil error, then we just return a status 500 Internal Server Error to the client.

Else, we will build the CreateUserParams object, where Username is request.Username, HashedPassword is the computed hashedPassword, FullName is request.FullName, and Email is request.Email.

func (server *Server) createUser(ctx *gin.Context) {
    ...

    user, err := server.store.CreateUser(ctx, arg)
    if err != nil {
        if pqErr, ok := err.(*pq.Error); ok {
            switch pqErr.Code.Name() {
            case "unique_violation":
                ctx.JSON(http.StatusForbidden, errorResponse(err))
                return
            }
        }
        ctx.JSON(http.StatusInternalServerError, errorResponse(err))
        return
    }

    ctx.JSON(http.StatusOK, user)
}
Enter fullscreen mode Exit fullscreen mode

Then we call server.store.CreateUser() with this input argument. It will return the created user object or an error.

Just like in the create account API, if error is not nil, then there are some possible scenarios. Keep in mind that, in the users table, we have 2 unique constraints:

  • One is for the primary key username,
  • And the other is for the email column.

We don’t have a foreign key in this table, so here we only need to keep the unique_violation code name to return status 403 Forbidden in case an user with the same username or email already exists.

Finally, if no errors occur, we just return status 200 OK with the created user to the client.

OK, so now the createUser API handler is completed. The last step we must do is to register a route for it in the api/server.go file.

Here, in this NewServer() function, I’m gonna add a new route with method POST. Its path should be /users, and its handler function is server.createUser

// NewServer creates a new HTTP server and set up routing.
func NewServer(store db.Store) *Server {
    server := &Server{store: store}
    router := gin.Default()

    if v, ok := binding.Validator.Engine().(*validator.Validate); ok {
        v.RegisterValidation("currency", validCurrency)
    }

    router.POST("/users", server.createUser)

    router.POST("/accounts", server.createAccount)
    router.GET("/accounts/:id", server.getAccount)
    router.GET("/accounts", server.listAccounts)

    router.POST("/transfers", server.createTransfer)

    server.router = router
    return server
}
Enter fullscreen mode Exit fullscreen mode

And that’s it! We’re done!

Test the create user API

Let’s open the terminal and run make server to start the server.

I’m gonna use Postman to test the new API.

Let’s select method POST and fill in the URL: http://localhost:8080/users

For the request body, let’s choose raw, and select JSON format. I'm gonna use this JSON data:

{
    "username": "quang1",
    "full_name": "Quang Pham",
    "email": "quang@email.com",
    "password": "secret"
}
Enter fullscreen mode Exit fullscreen mode

OK, let’s send this request!

Alt Text

It’s successful! We’ve got the created user object here with all correct field values.

Let’s open the database to find this user.

Alt Text

Here it is! So our API is working well in the happy case.

Now let’s see what happens if I send this same request the second time.

Alt Text

We’ve got a 403 Forbidden status code. And the reason is that the unique constraint for username is violated.

We’re trying to create another user with the same username, So clearly it should not be allowed!

Now let’s try changing the username to quang2, but keep the email value the same, and send the request again.

Alt Text

We still got 403 Forbidden. But this time, the error is because the email unique constraint is violated. Exactly what we expected!

If I change the email to quang2@email.com, then the request will be successful, since this email doesn’t belong to any other users.

Alt Text

OK, now let’s try an invalid username, such as quang#2:

Alt Text

This time, the status code is 400 Bad Request. And the reason is: the field validation for username failed on the alphanum tag. There’s a special character # in the username, which is not alphanumeric.

Next, let’s try an invalid email. I’m gonna change the username to quang3, and email to quang3email.com, without the @ character.

Alt Text

We’ve got 400 Bad Request status again. And the error is: field validation for email failed on the email tag, which is exactly what we want.

OK now let’s fix the email address, and change the password to a very short value, such as "123". Then send the request one more time.

Alt Text

This time, we’ve got an error because the password field validation failed on the min tag. It doesn’t satisfy the minimum length constraint of 6 characters.

API should not expose hashed password

Before we finish, there’s one more thing I want to tell you. Let’s fix the password value and send the request again.

Alt Text

Now it’s successful. But you can notice that the hashed_password value is also returned, which doesn’t seem right, because the client will never need to use this value for anything.

And it might raise some security concerns, as this piece of sensitive information is being transmitted in the public.

It would be better to remove this field from the response body.

To do that, I’m gonna declare a new createUserResponse struct in the api/user.go file. It will contain almost all fields of the db.User struct, except for the HashedPassword field that should be removed.

type createUserResponse struct {
    Username          string    `json:"username"`
    FullName          string    `json:"full_name"`
    Email             string    `json:"email"`
    PasswordChangedAt time.Time `json:"password_changed_at"`
    CreatedAt         time.Time `json:"created_at"`
}
Enter fullscreen mode Exit fullscreen mode

Then here, at the end of the createUser() handler function, we build a new createUserResponse object, where Username is user.Username, FullName is user.FullName, Email is user.Email, PasswordChangedAt is user.PasswordChangedAt, and CreatedAt is user.CreatedAt.

func (server *Server) createUser(ctx *gin.Context) {
    ...

    user, err := server.store.CreateUser(ctx, arg)
    if err != nil {
        if pqErr, ok := err.(*pq.Error); ok {
            switch pqErr.Code.Name() {
            case "unique_violation":
                ctx.JSON(http.StatusForbidden, errorResponse(err))
                return
            }
        }
        ctx.JSON(http.StatusInternalServerError, errorResponse(err))
        return
    }

    rsp := createUserResponse{
        Username:          user.Username,
        FullName:          user.FullName,
        Email:             user.Email,
        PasswordChangedAt: user.PasswordChangedAt,
        CreatedAt:         user.CreatedAt,
    }
    ctx.JSON(http.StatusOK, rsp)
}
Enter fullscreen mode Exit fullscreen mode

Finally, we return the response object instead of user. And we’re done!

Let’s restart the server. Then go back to Postman, update the username and email to new values, and send the request.

Alt Text

It’s successful. And now there’s no hashed_password field in the response body anymore. Perfect!

So that brings us to the end of this lecture. I hope you have learned something useful.

Thank you for reading, and see you in the next one!


If you like the article, please subscribe to our Youtube channel and follow us on Twitter or Facebook for more tutorials in the future.

Discussion (9)

pic
Editor guide
Collapse
jhelberg profile image
Joost Helberg

No, no, do not store the password. Your database already has a solution for that. Use it and implement row-level security along the way. I like your exposé, it is clear and thorough, but the starting point is wrong. Don't ever store a password. Use other services. Your lawyer will appreciate this.

Collapse
techschoolguru profile image
TECH SCHOOL Author

Hey Joost, I guess you didn't read the article and only look at the title. ^^
Actually, we don't store the password but only hash it using bcrypt, and only store the hashed value.
Besides, I don't know what you meant by "row-level security", or "use other services".

Collapse
jhelberg profile image
Joost Helberg

I did read the article. Its very thorough. And yes, you do store the password. You try to hide some of it, but many people may be able to, and will, still see a password after some trying. If you want to know more about RLS and database login roles, please click on it to Google it. There are some nice articles on it. Both mssql and postgresql have extensive support for it. Avoiding storing passwords is a by-product of it.

Thread Thread
techschoolguru profile image
TECH SCHOOL Author

I think you must have misunderstood the content. There's no password stored in the DB. Only the hash-value of the password (not encrypted value). So the only way you can "see" the naked password is to brute force all possible values, hash them, then compare them with the stored hashed value.

RLS/db login roles only limit access of the company's employees, but it doesn't solve the purpose of how the web application server can authenticate users (login API). In order to authenticate user, the web app needs to check if the user's provided password (when login) matches with their actual password (when register) or not.

Thread Thread
jhelberg profile image
Joost Helberg

Storing a hash is almost similar to storing the password. Unles you can keep up with the hackers. Believe me, small private developers cannot. A hash doesn't protect you. Now of course, a bit, until the hash algorithm is proven useless in the next few years.
Login roles are not limited to inbound users. Use of login roles for everyone is a great security improvement and should be implemented a lot more often. It avoids lots of risky application solutions. Lots of critical security bugs are in applications doing the authorization in code.

Thread Thread
techschoolguru profile image
TECH SCHOOL Author

How do you know whether the password user provides at login matches with his real password when he registers if you don't store anything in the DB? My point is, even if you somehow use existing technology of the DB, that technology still needs to store something about the user's password.
Or maybe you have a better explanation of how it works?

Thread Thread
jhelberg profile image
Joost Helberg

Leaving authentication to an underlying product does mean it is stored somewhere, you are right about that. My experience is that no application builder understands the importance of authentication as good as the server-designers of database and other authentication providers. So, my opinion is that you'd better leave authentication to those parties and keep away from authentication as far as possible.
RLS allows you to use the user-context for determining which rows are for you and which not. The application doesn't need to bother figuring out authorization. The RLS, a static declaration and hence etter verifiable, will do that for the application.
Oauth2 is an example of using a third party authoriser, rdbms' can do it too.

Thread Thread
larsonnn profile image
Lars Feldeisen

Stop talking.

Collapse
aminmansuri profile image
hidden_dude

For maximum security I would suggest you use byte[] or char[] in memory rather than strings and overwrite the arrays right after you use them.

Strings are prone to linger in memory and can be recovered from a core dump or other such attacks.