Golang Service Configuration

Rob Pike wrote an interesting blog about API options, it can be found here. It’s worth a read, but the summary is that it’s a damn fine idea to design options as a variadic argument of configuring functors.

Dave Cheney presented a nice summary of the pros and cons of this versus other approaches (can be read here), and a nice example he gave of what it might look like is this:

func NewServer(addr string, options ...func(*Server)) (*Server, error)

func main() {
	src, _ := NewServer("localhost") // defaults

	timeout := func(srv *Server) {
		srv.timeout = 60 * time.Second
	}

	tls := func(srv *Server) {
		config := loadTLSConfig()
		srv.listener = tls.NewListener(srv.listener, &config)
	}

	// listen securely with a 60 second timeout
	src2, _ := NewServer("localhost", timeout, tls)
}

This pattern gives users of an API a simple syntax for expressing function options, but simultaneously allows the author of the API to dodge the problem of ambiguous zero values vs defaults, and also the ability to expand options without regression.

I love this methodology, but whilst working on web services I’ve been thinking a lot about configuration. When building a service it’s nice to treat various modular components as an API in itself, but the options for these components often match 1:1 with service configuration values. I therefore wanted to try and merge the two concepts to avoid duplication, this added the following requirements to my options solution:

• Individual component options can be aggregated into a single construct
• Easy to serialize/de-serialize all-encompassing construct

The first requirement is self explanatory. I want to define my options once and at the component level, but I want to treat my overall system configuration as one unit.

The second requirement includes de-serialization, which is easy to justify since we need to be able to read our configuration. Serialization will also give us a way of self documenting all configuration options at run time, allowing us to print our values before reading a configuration file (to show defaults) or afterwards to reveal any mistakes in a users config file.

Both of these requirements are easily satisfied with struct based options, but, as Dave Cheney rightfully pointed out, they carry issues of comparing intentional values vs zero values (is an empty string indicating the user wants the default option?) and the API usage can become long winded for the most simple use case (all default values).

My solution isn’t well suited for writing libraries, but for services I solved this problem by adding a function definition to every component that has an options struct, which returns an options object fully populated with default values.

To use a component you now generate a default options object, modify any fields of interest and then pass it on. The code looks like this:

fooOptions := foo.NewOptions()
fooOptions.Thing = "hello"
fooOptions.Bar.Baz = 500

bar := foo.New(fooOptions)

// When using only default values:

bar2 := foo.New(foo.NewOptions())

Defining our Foo looks like this:

// Options - options for our foo
type Options {
	Thing      string      `json:"thing"`
	OtherThing string      `json:"thing"`
	Bar        bar.Options `json:"bar"`
}

// NewOptions - returns a new options object with default values
func NewOptions() Options {
	return Options{
		Thing:      "wat",
		OtherThing: "don't change me plz",
		Bar:        bar.NewOptions(),
	}
}

// A foo
type Foo struct {
	options Options
	bar     *Bar
}

// Creates a new foo
func New(options Options) *Foo {
	return &Foo{
		options: options,
		bar:     bar.New(options.Bar),
	}
}

Take note that since we’re using structs we can easily nest options within the options of a parent object, this hierarchy also gives you the ability to intuitively separate options for an object type based on its lineage (you might have two SQL CRUD objects for two different areas of your code base for example).

This adds one extra step to defining your options, but solves our problem. Your main function for your service might look like this:

type serviceOptions struct {
	ThingOne   foo.Options `json:"thing_one"`
	ThingTwo   yaz.Options `json:"thing_two"`
	ThingThree jup.Options `json:"thing_three"`
}

func newServiceOptions() serviceOptions {
	return serviceOptions{
		ThingOne:   foo.NewOptions(),
		ThingTwo:   yaz.NewOptions(),
		ThingThree: jup.NewOptions(),
	}
}

func main() {
	options := newServiceOptions()

	// ... read some cmd flags ...

	if readConfig {
		readConfigFile(&options)
	}
	if printConfig {
		printConfigFile(options)
	}

	c1 := componentOne(options.ThingOne)
	c2 := componentTwo(options.ThingTwo)
	c3 := componentThree(options.ThingThree)

	go c1.Run()
	go c2.Run()

	c3.Run()
}

This solution falls short in that it isn’t intuitive to someone picking up a library for the first time, as a natural assumption from the method signature is to use bar := foo.New(foo.Options{}), which is why I would only recommend this for use inside of services where it can be mandated as a common, familiar approach.