Background
When we write applications, we basically use configuration files, from various shell
s to nginx
, etc., all have their own configuration files. Although this is not too difficult, the configuration items are relatively cumbersome, and parsing and verifying them can be troublesome. In this article, I will tell how we simplify the definition and parsing of configuration files.
Scenario
If we want to write a Restful API
service, the configuration items will probably have the following contents.
-
Host
, the listeningIP
, if not filled, the default is0.0.0.0
-
Port
, the port to listen to, required, can only be a number, greater than or equal to 80, less than 65535 -
LogMode
, logging mode, onlyfile
orconsole
can be selected -
Verbose
, see whether to output detailed logs, optional, default isfalse
-
MaxConns
, the maximum number of concurrent connections allowed, default10000
-
Timeout
, timeout setting, default3s
-
CpuThreshold
, sets the threshold forCPU
usage to trigger load shedding, default900
,1000m
means100%
Previously we used json
for the configuration file, but there was a problem with json
that we couldn't add comments, so we switched to yaml
format.
Now let's see how easy it is to define and parse such a configuration file with go-zero
~
Defining the configuration
First, we need to define the above configuration requirements into a Go structure, as follows.
RestfulConf struct {
Host string `json:",default=0.0.0.0"`
Port int `json:",range=[80,65535)"`
LogMode string `json:",options=[file,console]"`
Verbose bool `json:",optional"`
MaxConns int `json:",default=10000"`
Timeout time.Duration `json:",default=3s"`
CpuThreshold int64 `json:",default=900,range=[0:1000]"`
}
As you can see, we have certain definitions and restrictions for each configuration item, some of which are defined as follows.
-
default
, the default value is used if the configuration is not filled, you can see that3s
in it will be automatically resolved intotime.Duration
type -
optional
, you can leave the item blank, if not, use the zero value of its type -
range
, which restricts the number type to the given range -
options
, which restricts the configured value to only one of the given values
And, some attributes can be used together, such as.
-
default
andrange
can be used together to both add a range restriction and provide a default value -
default
andoptions
can be used together to add option restrictions and provide a default value
Configuration files
Because we give a lot of default values when defining the configuration, as well as using optional
to specify as optional, so our configuration file has relatively few configuration items that can use the default values without writing explicitly, as follows.
# Because many of them have default values, you only need to write the ones
# that need to be specified and the ones that don't have default values
Port: 8080
LogMode: console
# You can read the values of environment variables
MaxBytes: ${MAX_BYTES}
Here is a note, if the value
of the configuration chars are all numbers, and you define the configuration type is string
, for example, some people often use 123456
to test the password, but the password will generally be defined as string
, the configuration should be written as follows (just an example, passwords are generally not recommended to be written in the configuration file)
Password: "123456"
Here the double quotes can not be missed, otherwise type mismatch
will be reported, because yaml
parser will parse 123456
as int
.
Loading configuration files
We have the configuration definition (config.go
) and the configuration file (config.yaml
), the next step is to load the configuration file, which can be done in three ways.
- must be loaded successfully, otherwise the program exits, we generally use so, if the configuration is not correct, the program will not be able to continue
// If error, exit the program directly
var config RestfulConf
conf.MustLoad("config.yaml", &config)
The default code generated by go-zero
's goctl
tool also uses MustLoad
to load the configuration file
- Load the configuration and handle the
error
on your own
// handle errors on your own
var config RestfulConf
// For simplicity, LoadConfig will be changed to Load later,
// LoadConfig has been marked as Deprecated.
if err := conf.LoadConfig("config.yaml", &config); err ! = nil {
log.Fatal(err)
}
- Load the configuration and read the environment variables
// Automatically read environment variables
var config RestfulConf
conf.MustLoad(configFile, &config, conf.UseEnv())
Here why we need to explicitly specify conf.UseEnv()
, because if default to use, you may need to escape
when writing specific characters in the configuration, so the default does not read the environment variables.
Implementation principle
We usually use encoding/json
or the corresponding yaml
library when implementing yaml/json
parsing, but for go-zero
, we need to have more precise control over unmarshal
, which leads us to customize yaml/json
parsing ourselves, the complete code is here.
Configuration file code: https://github.com/zeromicro/go-zero/tree/master/core/conf
yaml/json
parsing code: https://github.com/zeromicro/go-zero/tree/master/core/mapping
This is also a good example of how reflect
can be used and how unit tests can be used to ensure correctness in complex scenarios.
Summary
I always recommend the idea of Fail Fast
, we do the same on loading configuration files, once there is an error, immediately exit, so that ops
will find the problem in time when deploying services, because the process can not get up running.
All the configuration items of go-zero
services are loaded and automatically verified in this way, including the configuration of many of the tools I wrote are also based on this, hope it helps!
Project address
https://github.com/zeromicro/go-zero
Welcome to use go-zero
and star to support us!
Top comments (0)