Golang gRPC Example

We’re Earthly. We make building software simpler and therefore faster. Earthly is open-source and written in go. So if you’re interested in a simpler way to build then check us out.

Welcome back. I’m an experienced developer, learning Golang by building an activity tracker. Last time I added SQLite persistence. Today, I’m going to be porting everything to gRPC.

If you’re curious about gRPC – how it works, when to use it, what example code might look like – well, you are in luck because I’m going to be building a grpc client, a grpc server, and the protobuf files for my activity tracker. The full code is on GitHub.

Why gRPC

If the primary consumer of your service is client-side JavaScript or if your project is going to have many clients, some of which you won’t control, then a JSON-based REST service is a great way to go. JSON is human-readable, it’s simple to make requests at the command-line or using tools like postman, and it’s well understood how to write good REST APIs.

However, it has some downsides. JSON is a text format, so there is more data to send, and it is more expensive to serialize and de-serialize. A standardized API specification exists ( OpenAPI 2.0 aka Swagger 2.0 ) but generating a client and server from the specification is tricky.

gRPC addresses a lot of these issues: it’s a binary format, so it’s quicker to send. It’s faster to serialize and de-serialize, and it has better types than JSON. But, most notably for today, gRPC’s protoc tool has extensive code-generation abilities. So I can describe my service using .proto files and use an existing tool to do some heavy lifting.

Protocol Buffers

One of the significant advantages to using gRPC is protocol buffers (protobufs). With protobufs, you can encode the message semantics in a parsable form that the client and the server can share. Also, protobufs are a platform-neutral language for structuring data with built-in fast serialization and support for schema migration, which is vital if you want to change your message formats without introducing downtime. But that’s enough talk. Let’s start building something.

First thing I’ll do is create a message type:

syntax = "proto3";

package api.v1;

import "google/protobuf/timestamp.proto";

message Activity {
    int32 id = 1;
    google.protobuf.Timestamp time = 2;
    string description = 3;
}

There are a couple of things to note in this short example. First off, I’m using the latest version of the protobuf syntax proto3. Second, I’m specifying a package name package api.v1; – this will make it easier for me to import the generated code.

Then I’ll install the protobuf compiler:

brew install protobuf

I’ll make sure it’s installed:

protoc --version

libprotoc 3.19.4

Then I can use it to generate the go struct for the message type:

 protoc activity-log/api/v1/*.proto \
    --go_out=. \
    --go_opt=paths=source_relative \
    --proto_path=.

Please specify a program using absolute path or make sure the program is available in your PATH system variable
--go_out: protoc-gen-go: Plugin failed with status code 1.

Oh wait, first, it seems I need protoc-gen-go:

$ brew install grpc 
$ go install google.golang.org/protobuf/cmd/protoc-gen-go@v1.26

And I need to add it to my path:

export PATH="$PATH:$(go env GOPATH)/bin"

With that installed, I can successfully generate some code

 protoc activity-log/api/v1/*.proto \
    --go_out=. \
    --go_opt=paths=source_relative \
    --proto_path=.

And I get an activity struct that I can use in my service and client:

// Code generated by protoc-gen-go. DO NOT EDIT.
// versions:
//  protoc-gen-go v1.26.0
//  protoc        v3.19.4
// source: activity-log/api/v1/activity.proto

package api_v1

import (
 protoreflect "google.golang.org/protobuf/reflect/protoreflect"
 protoimpl "google.golang.org/protobuf/runtime/protoimpl"
 timestamppb "google.golang.org/protobuf/types/known/timestamppb"
 reflect "reflect"
 sync "sync"
)

type Activity struct {
 state         protoimpl.MessageState
 sizeCache     protoimpl.SizeCache
 unknownFields protoimpl.UnknownFields

 Id          int32                  `protobuf:"varint,1,opt,name=id,proto3" json:"id,omitempty"`
 Time        *timestamppb.Timestamp `protobuf:"bytes,2,opt,name=time,proto3" json:"time,omitempty"`
 Description string                 `protobuf:"bytes,3,opt,name=description,proto3" json:"description,omitempty"`
}

protoc also generates several helper methods for working with the protobuf message, such as field getters:

func (x *Activity) GetTime() *timestamppb.Timestamp {
 if x != nil {
  return x.Time
 }
 return nil
}

func (x *Activity) GetId() int32 {
 if x != nil {
  return x.Id
 }
 return 0
}

These are helpful if I want to create an interface to abstract across various message types.

Now that I have things working for one message type, I can define my whole service:

service Activity_Log {
    rpc Insert(Activity) returns (InsertResponse) {}
    rpc Retrieve(RetrieveRequest) returns (Activity) {}
    rpc List(ListRequest) returns (Activities) {}
}

message RetrieveRequest {
    int32 id = 1;
}

message InsertResponse {
    int32 id = 1;
}

message ListRequest {
    int32 offset = 1;
}

message Activities {
    repeated Activity activities = 1;
}

message ActivityQuery {
 int32 offset = 1;
}

I can then generate the client and the service code using protoc again.

 protoc activity-log/api/v1/*.proto \
    --go-grpc_out=. \
    --go-grpc_opt=paths=source_relative \
    --proto_path=.

Running this generates activity_grpc.pb.go with all the necessary code for a client and a server.

Note how this time I used go-grpc_out and go-grpc_opt=paths instead of go_out and go_opt=paths. I can combine these two to generate messages, the client, and the server.

  protoc activity-log/api/v1/*.proto \
    --go_out=. \
    --go_opt=paths=source_relative \
    --go-grpc_out=. \
    --go-grpc_opt=paths=source_relative \
    --proto_path=.

Golang gRPC Server

Golang Protobuf Types

Now that I’ve got all my code generated, it’s time for me to build the server-side. Let’s start at the database layer and work upwards

If you recall from when I was adding the sqlite feature, Activities handles all the data persistence. So the data persistence layer shouldn’t have change much at all. I just need to make sure I’m using my protoc generated struct. I can do this with an import change:

import 
- api "github.com/adamgordonbell/cloudservices/activity-log"
+ api "github.com/adamgordonbell/cloudservices/activity-log/api/v1"
+ "google.golang.org/protobuf/types/known/timestamppb"

google.protobuf.Timestamp

Previously my Activity struct used time.Time to represent time and net/http was mapping it back and forth to a JSON string. However, protobufs are typed, so I have chosen to use google.protobuf.Timestamp as my time type. This means I don’t have to worry about getting an invalid date sent in.

Unfortunately, my generated code now uses a google.protobuf.Timestamp, where my persistence layer needs a time.Time. This is painless fix with AsTime:

// AsTime converts x to a time.Time.
func (x *Timestamp) AsTime() time.Time {
       return time.Unix(int64(x.GetSeconds()), int64(x.GetNanos())).UTC()
}

func (c *Activities) Insert(activity *api.Activity) (int, error) {
 res, err := c.db.Exec("INSERT INTO activities VALUES(NULL,?,?);", 
-  activity.Time, 
+  activity.Time.AsTime(), 
  activity.Description)
 if err != nil {
  return 0, err
 }

And that is the only persistence layer change we need to make to switch from our hand-rolled struct to the protoc generated one. Again, you can see the full thing on github.

GRPC Service

Now that my persistence layer uses the gRPC messages, I need to create a grpc.Server and start it up.

Previously, in my http service, I had an httpServer, I’m going to rename that:

- type httpServer struct {
+ type grpcServer struct {
 Activities *Activities
}

And then I need to make an instance of it:

func NewGRPCServer() *grpc.Server {
 var acc *Activities
 var err error
 if acc, err = NewActivities(); err != nil {
  log.Fatal(err)
 }
 gsrv := grpc.NewServer()
 srv := grpcServer{
  Activities: acc,
 }
 api.RegisterActivity_LogServer(gsrv, &srv)
 return gsrv
}

And then wire that up to my main method, and I can start things up:

func main() {
 log.Println("Starting listening on port 8080")
 port := ":8080"

 lis, err := net.Listen("tcp", port)
 if err != nil {
  log.Fatalf("failed to listen: %v", err)
 }
 log.Printf("Listening on %s", port)
 srv := server.NewGRPCServer()

 if err := srv.Serve(lis); err != nil {
  log.Fatalf("failed to serve: %v", err)
 }
}

I haven’t written an implementation of any of the RPC methods yet, but I’m curious what happens if I run it.

$ go run cmd/server/main.go

# github.com/adamgordonbell/cloudservices/activity-log/internal/server
internal/server/server.go:30:39: cannot use &srv (type *grpcServer) as type api_v1.Activity_LogServer in argument to api_v1.RegisterActivity_LogServer:
        *grpcServer does not implement api_v1.Activity_LogServer (missing api_v1.mustEmbedUnimplementedActivity_LogServer method)

No luck, but it gives me a helpful error message. All I need to do is add an UnimplementedActivity_LogServer:

 type grpcServer struct {
+ api.UnimplementedActivity_LogServer
  Activities *Activities
 }

And with that, I can start running things:

grpcurl -plaintext -d  '{ "description": "christmas eve bike class" }' \
 localhost:8080 api.v1.Activity_Log/Insert

ERROR:
  Code: Unimplemented
  Message: method Insert not implemented

How does that work? How can I call the method that I haven’t implemented yet? Well, the protoc generated code contains UnimplementedActivity_LogServer, which looks like this:

// UnimplementedActivity_LogServer must be embedded to have forward compatible implementations.
type UnimplementedActivity_LogServer struct {
}

But, it also implements this interface:

type Activity_LogServer interface {
 Insert(context.Context, *Activity) (*InsertResponse, error)
 Retrieve(context.Context, *RetrieveRequest) (*Activity, error)
 List(context.Context, *ListRequest) (*Activities, error)
 mustEmbedUnimplementedActivity_LogServer()
}

and those implementations are what I’m hitting when I call insert:

func (UnimplementedActivity_LogServer) Insert(context.Context, *Activity) (*InsertResponse, error) {
 return nil, status.Errorf(codes.Unimplemented, "method Insert not implemented")
}

As a newcomer to GoLang, this is pretty nice! I can just read through the generated code and understand how it works without much difficulty.

brew install grpcurl

func main() {
 ...
 srv := server.NewGRPCServer()
+ // Register reflection service on gRPC server.
+ reflection.Register(srv)
 if err := srv.Serve(lis); err != nil {
  log.Fatalf("failed to serve: %v", err)
 }
}

$ grpcurl -plaintext -d  \
  '{ "description": "christmas eve bike class" }' \
  localhost:8080 api.v1.Activity_Log/Insert

{
  "id": 1
}

grpcurl -plaintext localhost:8080 describe

api.v1.Activity_Log is a service:
service Activity_Log {
  rpc Insert ( .api.v1.Activity ) returns ( .api.v1.InsertResponse );
  rpc List ( .api.v1.ListRequest ) returns ( .api.v1.Activities );
  rpc Retrieve ( .api.v1.RetrieveRequest ) returns ( .api.v1.Activity );
}

grpcurl -plaintext localhost:8080 describe
Error: server does not support the reflection API

 grpcurl -plaintext -d '{ "id": 1 }' \
   -proto ./activity-log/api/v1/activity.proto \
    localhost:8080 api.v1.Activity_Log/Retrieve

{
  "id": 1,
  "time": "1970-01-01T00:00:00Z",
  "description": "christmas eve bike class"
}

grpc_cli ls localhost:8080 -l

filename: activity-log/api/v1/activity.proto
package: api.v1;
service Activity_Log {
  rpc Insert(api.v1.Activity) returns (api.v1.Activity) {}
  rpc Retrieve(api.v1.RetrieveRequest) returns (api.v1.Activity) {}
  rpc List(api.v1.ListRequest) returns (api.v1.Activities) {}
}

gRPC Service Implementation

So as it stands now, I have completed the database layer, and the service can start up and receive gRPC requests, but there is no service implementation connecting these two parts. Let’s write that.

I can follow the provided interface to create my implementation. Insert looks like this:

Insert(context.Context, *Activity) (*InsertResponse, error)

And I can implement it by just calling through to my database layer, handling the error conditions, and wrapping the response back up in the expected type:

func (s *grpcServer) Insert(ctx context.Context, activity *api.Activity) (*api.InsertResponse, error) {
 id, err := s.Activities.Insert(activity)
 if err != nil {
  return nil, fmt.Errorf("Internal Error: %w", err)
 }
 res := api.InsertResponse{Id: int32(id)}
 return &res, nil
}

I can repeat this for List and Retrieve, and I have a working solution. (Though the error handling has room for improvement. I’ll get back to that later on in the article).

Testing A gRPC Server

Previously, I had tested my REST service by starting it up in a docker container and exercising some endpoints via a small bash script test.sh. I then ran it all in an Earthfile in GitHubActions that looked like this:

test:
    FROM +test-deps
    COPY test.sh .
    WITH DOCKER --load agbell/cloudservices/activityserver=+docker
        RUN  docker run -d -p 8080:8080 agbell/cloudservices/activityserver && \
                ./test.sh
    END

To get this working with gRPC, all I need to do is change test.sh to use grpcurl:

# echo "=== Test Reflection API ==="
grpcurl -plaintext localhost:8080 describe

echo "=== Insert Test Data ==="

grpcurl -plaintext -d  '{ "description": "christmas eve bike class" }' localhost:8080 api.v1.Activity_Log/Insert

echo "=== Test Retrieve Descriptions ==="

grpcurl -plaintext -d '{ "id": 1 }' localhost:8080 api.v1.Activity_Log/Retrieve | grep -q 'christmas eve bike class'

echo "=== Test List ==="

grpcurl -plaintext localhost:8080 api.v1.Activity_Log/List | jq '.activities | length' |  grep -q '1'

echo "Success"

And additionally, I need to make sure my +test-deps container has grpcurl installed. There are lots of ways to get grpcurl into my alpine base image, but the way I did it was to just copy it from the official grpcurl alpine image into my test-deps image:

+ grpcurl:
+    FROM fullstorydev/grpcurl:latest
+    SAVE ARTIFACT /bin/grpcurl ./grpcurl

 test-deps:
     FROM earthly/dind
     RUN apk add curl jq
+    COPY +grpcurl/grpcurl /bin/grpcurl

And with that, my gRPC server example is working and has end-to-end tests running in CI.

Now I can move on to the gRPC client example.

Golang gRPC Client Example

How does the client code get generated? It is generated using protoc just like the service. In fact, I’ve already generated it without realizing it.

protoc created the client code when given --go-grpc_out:

    protoc activity-log/api/v1/*.proto \
    --go_out=. \
    --go_opt=paths=source_relative \
    --go-grpc_out=. \
    --go-grpc_opt=paths=source_relative \
    --proto_path=.

It looks like this:

type activity_LogClient struct {
 cc grpc.ClientConnInterface
}

func NewActivity_LogClient(cc grpc.ClientConnInterface) Activity_LogClient {
 return &activity_LogClient{cc}
}

My Activities client is going to contain an instance of this client:

type Activities struct {
 client api.Activity_LogClient
}

And I’ll initialize the client with an active connection like this:

func NewActivities(URL string) Activities {
 conn, err := grpc.Dial(URL, grpc.WithTransportCredentials(insecure.NewCredentials()))
 if err != nil {
  log.Fatalf("did not connect: %v", err)
 }
 client := api.NewActivity_LogClient(conn)
 return Activities{client: client}
}

Back in my main method, I initialize the client and also create a context. This context lets the client track request-specific details. I’m building mine with a timeout so my service can’t hang my client if something goes sideways.

ctx, cancel := context.WithTimeout(context.Background(), time.Second)
defer cancel()

go run cmd/client/main.go -get 3
Error: Insert failure: rpc error: code = Canceled desc = context canceled
exit status 1

go run cmd/client/main.go -get 1
Error: Insert failure: rpc error: code = Canceled desc = grpc: the client connection is closing
exit status 1

Golang GRPC Client Implementation

The last step I need to do is call the generated client code and handle any possible errors. Here is the insert:

func (c *Activities) Insert(ctx context.Context, activity *api.Activity) (int, error) {
 resp, err := c.client.Insert(ctx, activity)
 if err != nil {
  return 0, fmt.Errorf("Insert failure: %w", err)
 }
 return int(resp.GetId()), nil
}

Did I say handle errors? That is where things get a little trickier. In a REST service, I can infer meaning from response codes. Insert, shown above, is pretty simple, but I need to differentiate between a server error and an id not existing when implementing Retrieve. That was straightforward with Rest: I had 404s and 500s.

It turns out gRPC has something similar.

gRPC Error Codes

In the gRPC service above, I constructed errors like this:

func (s *grpcServer) Retrieve(ctx context.Context, req *api.RetrieveRequest) (*api.Activity, error) {
 resp, err := s.Activities.Retrieve(int(req.Id))
 if err == ErrIDNotFound {
  return nil, fmt.Errorf("id was not found %w", err)
 }
 ...
}

And if I send in an invalid ID, I get a response like this:

$ grpcurl -plaintext -d '{ "id": 5 }' \
  localhost:8080 api.v1.Activity_Log/Retrieve 

ERROR:
  Code: Unknown
  Message: id was not found Id not found

My client can only understand that message by matching on the string. And I don’t want my client coupled to the exact strings used by my server.

So, I’m going to change the server to return proper gRPC status codes:

package server

import (
+ codes "google.golang.org/grpc/codes"
+ status "google.golang.org/grpc/status"
)
func (s *grpcServer) Retrieve(ctx context.Context, req *api.RetrieveRequest) (*api.Activity, error) {
 resp, err := s.Activities.Retrieve(int(req.Id))
 if err == ErrIDNotFound {
-  return nil, fmt.Errorf("id was not found %w", err)
+  return nil, status.Error(codes.NotFound, "id was not found")
 }
 if err != nil {
-  return nil, fmt.Errorf("Internal Error: %w", err)
+  return nil, status.Error(codes.Internal, err.Error())
 }
 return resp, nil
}

And then I get proper status codes:

$ grpcurl -plaintext -d '{ "id": 5 }' \
  localhost:8080 api.v1.Activity_Log/Retrieve 

ERROR:
  Code: NotFound
  Message: id was not found Id not found

Then I can unwrap the errors using status.FromError on the client-side. This allows me to handle code.NotFound separately from other errors:

func (c *Activities) Retrieve(ctx context.Context, id int) (*api.Activity, error) {
 resp, err := c.client.Retrieve(ctx, &api.RetrieveRequest{Id: int32(id)})
 if err != nil {
  st, _ := status.FromError(err)
  if st.Code() == codes.NotFound {
   return &api.Activity{}, ErrIDNotFound
  } else {
   return &api.Activity{}, fmt.Errorf("Unexpected Insert failure: %w", err)
  }
 }
 return resp, nil
}

And with that implementation in place, the client works. Here is the Earthly build:

Playing Nice With Others

I mentioned earlier that using an OS package manager installed protoc was not necessarily the best way to do things. Now let me show you why:

 // Code generated by protoc-gen-go. DO NOT EDIT.
 // versions:
-//     protoc-gen-go v1.26.0
-//     protoc        v3.19.4
+//     protoc-gen-go v1.27.1
+//     protoc        v3.13.0

That was a diff created by running protoc on a Debian environment. And then when I was back on my mac, I got this diff.

 // Code generated by protoc-gen-go. DO NOT EDIT.
 // versions:
-//     protoc-gen-go v1.27.1
-//     protoc        v3.13.0
+//     protoc-gen-go v1.26.0
+//     protoc        v3.19.4

The problem is that I’m using different versions of protoc. I need a way to pin the version of protoc and protoc-gen-go. Then I wouldn’t have these messy diffs. And this is just a side-project – this will get worse with larger teams.

This is one of the big reasons we see people reach for Earthly. With Earthly, I can add a target that installs a specific version of protoc into a container:

proto-deps:
    FROM golang:buster
    RUN apt-get update && apt-get install -y wget unzip
    RUN wget -O protoc.zip \
        https://github.com/protocolbuffers/protobuf/releases/download/v3.13.0/protoc-3.13.0-linux-x86_64.zip
    RUN unzip protoc.zip -d /usr/local/
    RUN go get google.golang.org/protobuf/cmd/protoc-gen-go \
               google.golang.org/grpc/cmd/protoc-gen-go-grpc

Then put my code generating protoc command in a target as well:

protoc:
    FROM +proto-deps
    WORKDIR /activity-log
    COPY go.mod go.sum ./ 
    COPY api ./api
    RUN protoc api/v1/*.proto \
            --go_out=. \
            --go_opt=paths=source_relative \
            --go-grpc_out=. \
            --go-grpc_opt=paths=source_relative \
            --proto_path=.
   SAVE ARTIFACT ./api AS LOCAL ./api 

And then if everyone runs earthly +protoc instead of calling protoc directly then we will all always get the same output. And an added benefit is it makes on-boarding people to your project easier because they don’t have to install any gRPC specific tools locally and you can bump the versions for everyone by just editing the Earthfile.

Was This Worth It?

The whole gRPC solution is a bit less code than the previous REST solution, if I exclude the generated code. And although it did take me a bit longer to get working, the advantages with this approach should increase as my messages and service endpoints get more complex. Also, I learned a lot, so I think this was a worthwhile change.

Also, Earthly made it simple to test the whole solution and to pin a specific version of the protocol buffer compiler. So, if you are looking for a vendor-neutral way to describe your build and test process, take a look at Earthly, and if you want to read the next installment of this series, sign up for the newsletter.

Also if you have any feedback on this tutorial, you can find me @adamgordonbell.