DEV Community

Hamed Naeemaei
Hamed Naeemaei

Posted on • Updated on

Golang Web API: Project configuration management using Viper

In this section, I create separate config files (yaml) for the development environment and docker environment, then make a config package to read configuration from yaml files.
To see this session project you can visit Session 4 GitHub repo URL.

Create config files

In the first step, we need to make three config files for the docker, production, and development environments in src/config path:

1. config-development.yml

server:
  internalPort: 5005
  externalPort: 5005
  runMode: debug
logger:
  filePath: ../logs/
  encoding: json
  level: debug
  logger: zap
cors:
  allowOrigins: "*"
postgres:
  host: localhost
  port: 5432
  user: postgres
  password: admin
  dbName: car_sale_db
  sslMode: disable
  maxIdleConns: 15
  maxOpenConns: 100
  connMaxLifetime: 5
redis:
  host: localhost
  port: 6379
  password: password
  db: 0
  dialTimeout: 5
  readTimeout: 5
  writeTimeout: 5
  poolSize: 10
  poolTimeout: 15
  idleCheckFrequency: 500
password:
  includeChars: true
  includeDigits: true
  minLength: 6
  maxLength: 64
  includeUppercase: true
  includeLowercase: true
otp:
  expireTime: 120
  digits: 6
  limiter: 100
jwt:
  secret: "mySecretKey"
  refreshSecret: "mySecretKey"
  accessTokenExpireDuration: 1440
  refreshTokenExpireDuration: 60

Enter fullscreen mode Exit fullscreen mode

2. config-docker.yml

server:
  internalPort: 5000
  externalPort: 0
  runMode: release
logger:
  filePath: /app/logs/
  encoding: json
  level: debug
  logger: zap
cors:
  allowOrigins: "*"
postgres:
  host: postgres_container
  port: 5432
  user: postgres
  password: admin
  dbName: car_sale_db
  sslMode: disable
  maxIdleConns: 15
  maxOpenConns: 100
  connMaxLifetime: 5
redis:
  host: redis_container
  port: 6379
  password: password
  db: 0
  dialTimeout: 5
  readTimeout: 5
  writeTimeout: 5
  poolSize: 10
  poolTimeout: 15
  idleCheckFrequency: 500
password:
  includeChars: true
  includeDigits: true
  minLength: 6
  maxLength: 64
  includeUppercase: true
  includeLowercase: true
otp:
  expireTime: 120
  digits: 6
  limiter: 100
jwt:
  secret: "mySecretKey"
  refreshSecret: "mySecretKey"
  accessTokenExpireDuration: 60
  refreshTokenExpireDuration: 60
Enter fullscreen mode Exit fullscreen mode

3. config-production.yml

server:
  internalPort: 5010
  externalPort: 5010
  runMode: release
logger:
  filePath: logs/
  encoding: json
  level: debug
  logger: zap
cors:
  allowOrigins: "*"
postgres:
  host: localhost
  port: 5432
  user: postgres
  password: admin
  dbName: car_sale_db
  sslMode: disable
  maxIdleConns: 15
  maxOpenConns: 100
  connMaxLifetime: 5
redis:
  host: localhost
  port: 6379
  password: password
  db: 0
  dialTimeout: 5
  readTimeout: 5
  writeTimeout: 5
  poolSize: 10
  poolTimeout: 15
  idleCheckFrequency: 500
password:
  includeChars: true
  includeDigits: true
  minLength: 6
  maxLength: 64
  includeUppercase: true
  includeLowercase: true
otp:
  expireTime: 120
  digits: 6
  limiter: 100
jwt:
  secret: "mySecretKey"
  refreshSecret: "mySecretKey"
  accessTokenExpireDuration: 1440
  refreshTokenExpireDuration: 60

Enter fullscreen mode Exit fullscreen mode

4. Config structure in go

All configuration of the project must be placed in config-{env}.yml files. For now, server, logger, postgres, cors, and Redis configurations are placed in this file and will be completed later.
Now it's time to start making config.go file to the central point of the project environment the configuration management. this file is placed in src/config directory. First, we need to create structs corresponding to the config sections:

type Config struct {
    Server   ServerConfig
    Postgres PostgresConfig
    Redis    RedisConfig
    Password PasswordConfig
    Cors     CorsConfig
    Logger   LoggerConfig
    Otp      OtpConfig
    JWT      JWTConfig
}

type ServerConfig struct {
    InternalPort    string
    ExternalPort    string
    RunMode string
}

type LoggerConfig struct {
    FilePath string
    Encoding string
    Level    string
    Logger   string
}

type PostgresConfig struct {
    Host            string
    Port            string
    User            string
    Password        string
    DbName          string
    SSLMode         string
    MaxIdleConns    int
    MaxOpenConns    int
    ConnMaxLifetime time.Duration
}

type RedisConfig struct {
    Host               string
    Port               string
    Password           string
    Db                 string
    DialTimeout        time.Duration
    ReadTimeout        time.Duration
    WriteTimeout       time.Duration
    IdleCheckFrequency time.Duration
    PoolSize           int
    PoolTimeout        time.Duration
}

type PasswordConfig struct {
    IncludeChars     bool
    IncludeDigits    bool
    MinLength        int
    MaxLength        int
    IncludeUppercase bool
    IncludeLowercase bool
}

type CorsConfig struct {
    AllowOrigins string
}

type OtpConfig struct {
    ExpireTime time.Duration
    Digits     int
    Limiter    time.Duration
}

type JWTConfig struct {
    AccessTokenExpireDuration  time.Duration
    RefreshTokenExpireDuration time.Duration
    Secret                     string
    RefreshSecret              string
}
Enter fullscreen mode Exit fullscreen mode

5. Config functions

In this project, we use viper package to manage configurations. So first we should install the Viper package go get -u github.com/spf13/viper. Then run go mod tidy command to ensure that all imports are satisfied.
At this stage, we need some functions

5.1 Get config file path

this function returns a file path based environment.

func getConfigPath(env string) string {
    if env == "docker" {
        return "/app/config/config-docker"
    } else if env == "production" {
        return "/config/config-production"
    } else {
        return "/config/config-development"
    }
}
Enter fullscreen mode Exit fullscreen mode
5.2 Load config file with Viper

After the config file was found, we needed to load the config file with Viper based specified file type.

func LoadConfig(filename string, fileType string) (*viper.Viper, error) {
    v := viper.New()
    v.SetConfigType(fileType)
    v.SetConfigName(filename)
    v.AddConfigPath(".")
    v.AutomaticEnv()

    err := v.ReadInConfig()
    if err != nil {
        log.Printf("Unable to read config: %v", err)
        if _, ok := err.(viper.ConfigFileNotFoundError); ok {
            return nil, errors.New("config file not found")
        }
        return nil, err
    }
    return v, nil
}
Enter fullscreen mode Exit fullscreen mode
5.3 Parse Viper config

In this section, we should Unmarshal Viper config to the custom Config type declared in section 4. If this instruction was successful means the file is parsed and can be used.

func ParseConfig(v *viper.Viper) (*Config, error) {
    var cfg Config
    err := v.Unmarshal(&cfg)
    if err != nil {
        log.Printf("Unable to parse config: %v", err)
        return nil, err
    }
    return &cfg, nil
}
Enter fullscreen mode Exit fullscreen mode
5.4 Get config file from other layers

Finally, we write a function that is exposed to other layers to using config struct. This function returns a pointer of the config struct. Furthermore, some swagger port settings are handled in this function and I will explain about it in the next sections.

func GetConfig() *Config {
    cfgPath := getConfigPath(os.Getenv("APP_ENV"))
    v, err := LoadConfig(cfgPath, "yml")
    if err != nil {
        log.Fatalf("Error in load config %v", err)
    }

    cfg, err := ParseConfig(v)
    envPort := os.Getenv("PORT")
    if envPort != ""{
        cfg.Server.ExternalPort = envPort
        log.Printf("Set external port from environment -> %s", cfg.Server.ExternalPort)
    }else{
        cfg.Server.ExternalPort = cfg.Server.InternalPort
        log.Printf("Set external port from environment -> %s", cfg.Server.ExternalPort)
    }
    if err != nil {
        log.Fatalf("Error in parse config %v", err)
    }

    return cfg
}
Enter fullscreen mode Exit fullscreen mode

This is the final version of the config.go file:

package config

import (
    "errors"
    "log"
    "os"
    "time"

    "github.com/spf13/viper"
)

type Config struct {
    Server   ServerConfig
    Postgres PostgresConfig
    Redis    RedisConfig
    Password PasswordConfig
    Cors     CorsConfig
    Logger   LoggerConfig
    Otp      OtpConfig
    JWT      JWTConfig
}

type ServerConfig struct {
    InternalPort    string
    ExternalPort    string
    RunMode string
}

type LoggerConfig struct {
    FilePath string
    Encoding string
    Level    string
    Logger   string
}

type PostgresConfig struct {
    Host            string
    Port            string
    User            string
    Password        string
    DbName          string
    SSLMode         string
    MaxIdleConns    int
    MaxOpenConns    int
    ConnMaxLifetime time.Duration
}

type RedisConfig struct {
    Host               string
    Port               string
    Password           string
    Db                 string
    DialTimeout        time.Duration
    ReadTimeout        time.Duration
    WriteTimeout       time.Duration
    IdleCheckFrequency time.Duration
    PoolSize           int
    PoolTimeout        time.Duration
}

type PasswordConfig struct {
    IncludeChars     bool
    IncludeDigits    bool
    MinLength        int
    MaxLength        int
    IncludeUppercase bool
    IncludeLowercase bool
}

type CorsConfig struct {
    AllowOrigins string
}

type OtpConfig struct {
    ExpireTime time.Duration
    Digits     int
    Limiter    time.Duration
}

type JWTConfig struct {
    AccessTokenExpireDuration  time.Duration
    RefreshTokenExpireDuration time.Duration
    Secret                     string
    RefreshSecret              string
}

func GetConfig() *Config {
    cfgPath := getConfigPath(os.Getenv("APP_ENV"))
    v, err := LoadConfig(cfgPath, "yml")
    if err != nil {
        log.Fatalf("Error in load config %v", err)
    }

    cfg, err := ParseConfig(v)
    envPort := os.Getenv("PORT")
    if envPort != ""{
        cfg.Server.ExternalPort = envPort
        log.Printf("Set external port from environment -> %s", cfg.Server.ExternalPort)
    }else{
        cfg.Server.ExternalPort = cfg.Server.InternalPort
        log.Printf("Set external port from environment -> %s", cfg.Server.ExternalPort)
    }
    if err != nil {
        log.Fatalf("Error in parse config %v", err)
    }

    return cfg
}

func ParseConfig(v *viper.Viper) (*Config, error) {
    var cfg Config
    err := v.Unmarshal(&cfg)
    if err != nil {
        log.Printf("Unable to parse config: %v", err)
        return nil, err
    }
    return &cfg, nil
}
func LoadConfig(filename string, fileType string) (*viper.Viper, error) {
    v := viper.New()
    v.SetConfigType(fileType)
    v.SetConfigName(filename)
    v.AddConfigPath(".")
    v.AutomaticEnv()

    err := v.ReadInConfig()
    if err != nil {
        log.Printf("Unable to read config: %v", err)
        if _, ok := err.(viper.ConfigFileNotFoundError); ok {
            return nil, errors.New("config file not found")
        }
        return nil, err
    }
    return v, nil
}

func getConfigPath(env string) string {
    if env == "docker" {
        return "/app/config/config-docker"
    } else if env == "production" {
        return "/config/config-production"
    } else {
        return "/config/config-development"
    }
}

Enter fullscreen mode Exit fullscreen mode

In the end, I want to load the config from the config file and read the webserver port from the config instead of hard-coding.
For example, I made some changes in src/api/api.go file. First append cfg := config.GetConfig() to start of InitServer function and then load the webserver port from config r.Run(fmt.Sprintf(":%s", cfg.Server.Port)).
This is the final version of api.go file:

package api

import (
    "fmt"

    "github.com/gin-gonic/gin"
    "github.com/naeemaei/golang-clean-web-api/api/routers"
    "github.com/naeemaei/golang-clean-web-api/config" // Added
)


func InitServer(){
    cfg := config.GetConfig()  // Added
    r := gin.New()
    r.Use(gin.Logger(), gin.Recovery())

    v1 := r.Group("/api/v1/")
    {
        health := v1.Group("/health")
        routers.Health(health)
    }

    if err := r.Run(fmt.Sprintf(":%s", cfg.Server.InternalPort)); err != nil {
        panic(err)
    } // Added
}
Enter fullscreen mode Exit fullscreen mode

Now we can run the project with the go run cmd/main.go command from src directory.

Project execution result
In the next article, I will make a simple endpoint and explain how to validate the requests and how works validators in gin.

Project until here: Configuration management usingViper

Top comments (1)

Collapse
 
hendisantika profile image
Hendi Santika

Very helpful & nice article.

I want to ask something.

Where do we put APP_ENV and what is the value for that key?

Thanks