Composing packages into applications?

Hi there! This is a question I recently posted to Reddit. I wanted to post it here as well in order to get even more insight :) .

I am a new Gopher coming from a more object oriented background (uh-oh! over complexity alert!). I have been really enjoying my time in Go so far, and I adore the simplicity. Something that I've been trying to understand is how to go about composing packages into an application (and if I'm overthinking it).

From what I've gathered so far, packages in Go should be organized by feature over function, and should be narrow in focus. That all makes sense to me, and its actually been a really natural process creating packages. What I can't quite figure out is the best way to compose those small individual packages, into a more complex application with business logic and cross cutting concerns. I also want to be clear, by cross cutting concerns I am not talking about things like logging. That seems to always be the default question on peoples minds when trying to understand relationships across domains.

Here's a (contrived) example of what I'm asking about.

I am creating a Go version of the cat-facts joke that circulated on Reddit years ago. Basically I want the ability to create and delete subscribers, and the ability to send a fact to all the subscribers.

I plan on having two binaries to accomplish this.

1. An API that exposes 2 endpoints. One for creating subscribers, and one for deleting subscribers.
2. A standalone binary that can be run via a cron job. It will get a list of subscribers from my data store, load a fact from an external api, and then loop through those subscribers sending the fact to each one.

In my mind this has broken into 3 packages.

1. subscribers
2. distribution
3. facts

I've included some sample code from each package below. The packages are all very narrow in focus and don't have much "business logic".

subscribers

Subscribers contains my subscriber model, as well as interfaces for reading, writing, listing and deleting subscribers. I also store implementations of those interfaces here.

package subscribers

type Contact = string

type Subscriber struct {
  Contact Contact
}

type SubscriberReader interface {
  Read(contact Contact) (*Subscriber, error)
}

type SubscriberWriter interface {
  Write(subscriber Subscriber) error
}

type SubscriberLister interface {
  List() ([]Subscriber, error)
}

type SubscriberDeleter interface {
  Delete(contact Contact) error
}

// implementations of interfaces...
// for instance, sqlite reader, writer, lister and deleter

distribution

Distribution contains the interface and implementation for sending things.

package distribution

type TextSender interface {
  SendText(to string, message string) error
}

// implementations of interfaces...
// for instance, twilio sender

facts

Facts contains my fact model and the interface and implementation for retrieving one.

package facts

type Fact = string

type FactRetriever interface {
  Retrieve() (Fact, error)
}

// implementations of interfaces...
// for instance, a retriever for the catfacts api

Like stated above, each of those packages are small and are good for actions in 1 domain. My confusion arises when I think about stitching those pieces together into an application.

Example 1 - Creating a Subscriber

When I create a subscriber I want to do 3 things.

1. Ensure a subscriber doesn't already exist with a particular Contact - I can utilize a subscribers.SubscriberReader
2. Write the subscriber to the data store - I can utilize a subscribers.SubscriberWriter
3. Send a welcome message to the subscriber - I can utilize a distribution.TextSender

func CreateSubscriber(
    reader subscribers.SubscriberReader,
    writer subscribers.SubscriberWriter,
    sender distribution.TextSender,
    subscriber subscribers.Subscriber) error {
    // error handling excluded for brevity...
    existing, _ := reader.Read(subscriber.Contact)

    if existing != nil {
        // error handling excluded for brevity...
    }

    // error handling excluded for brevity...
    _ := writer.Write(subscriber)

    // error handling excluded for brevity...
    _ := sender.SendText(subscriber.Contact, "Meow! Welcome to catfacts")

    return nil
}

In this example I want to execute logic that would cross boundaries. Because I want to perform actions from the subscribers package and the distribution package. I'm not sure where this code would live. It seems to me that this should not live in the subscribers package, because it is no longer just dealing with subscribers. Is there a common approach to this in Golang? A place where I would compose my business logic so to speak. Part of me feels like creating an additional application package that becomes the place where I compose all my functionality, but I'm not sure if that would be considered an anti-pattern.

Example 2 - Sending Facts

When I send facts I want to do 3 things.

1. Load the list of all subscribers
2. Load a random fact from a data source
3. Loop through the subscribers and send each one the fact

func SendFactToSubscribers(
    lister subscribers.SubscriberLister,
    retriever facts.FactRetriever,
    sender distribution.TextSender) error {
    // error handling excluded for brevity...
    subs, _ := lister.List()

    // error handling excluded for brevity...
    fact, _ := retriever.RetrieveFact()

    // error handling excluded for brevity...
    for _, subscriber := range subscribers {
        _ := sender.Send(subscriber.Contact, fact)
    }

    return nil
}

Again with this example I am executing logic that crosses domain boundaries. With this example I am also curious if I am "overloading" the function. Would someone with more Go experience than I write this function the same way? Or would it make more sense to break it down a bit. For instance pass in a []Subscriber as a parameter instead of an interface for listing them. If that's the case, then I would be back to trying to understand where to compose all the functionality together.

I am sure I am over complicating this. But part of the fun of learning a new language is learning all the best practices for that language. I love these types of design related challenges.

Any insight would be greatly appreciated!

Comments

[Jon Calhoun]

Try this instead:

1. Create a "top level" package. Call it catfacts or whatever else. In this define the things you need to build your app.

package catfacts

type Subscriber struct {
  Email string
}

// Feel free to tweak method names.
type SubscriberStore interface {
  Find(email string) (*Subscriber, error)
  Create(s *Subscriber) error
  List() ([]Subscriber, error)
  Delete(email string) error
}

type Sender interface {
  Send(s *Subscriber, message string) error
}

// In the context of your app, Sender and Retriever are probably descriptive enough. You could also call this FactStore if you want to be more consistent with the SubscriberStore.
type Retriever interface {
  Retrieve() (string, error)
}

Now when you want to implement Sender with say Twilio, create a package for that.

package twilio

type Sender struct {
  // ...
}

func (s *Sender) Send(subscriber *catfacts.Subscriber, message string) error {
  // ...
}

Now if you ever want to create a new sender, you just create a new package based on the impl. I just needs to implement catfacts.Sender.

Back to catfacts, you can create SendFactToSubscribers using your interfaces:

package catfacts

func SendFactToSubscribers(store SubscriberStore, retriever Retriever, sender Sender) {
  subs, _ := store.List()
  fact, _ := retriever.Retrieve()
  for _, sub := range subs {
    sender.Send(sub, fact)
  }
  return nil
}

Rinse and repeat for any other impls you need - eg package sqlite might have an sqlite impl of the SubscriberStore.

This may be a good point to continue this thinking: medium.com/@benbjohnson/standard-p...

I also started writing about this and hope to have more examples in the future, but I'll admit this series has taken the back seat to other work lately: calhoun.io/structuring-web-applica...

There isn't a single "best way" to write apps in Go, but this approach tends to work better for me.

[Jacob Smith]

Awesome! Thanks for such an informative response :) . I think when learning a new language it's good to try out different approaches to writing apps since like you said there is no one best way, this was a fantastic look at a way to organize code.

[Jon Calhoun]

You should definitely experiment a bit. Almost all of the pros/cons of each approach need to be learned firsthand to really sink it, and every approach doesn't fit every problem.

It can be a bit challenging with smaller apps because some of the problems (cyclical deps, etc) won't show up until an app gets to a certain size, but still worth experimenting.