Introduction into Protobuf for Cosmos

When developing and interacting with Cosmos SDK based coins, you will inevitably reach a point where you need to use Protobuf. Now you may be asking yourself, what is Protobuf and what do I need it for? You’ll find the answer here.

A brief introduction into Protobuf

According to Wikipedia, Protobuf is “a free and open-source cross-platform data format used to serialize structured data”. Even though that’s a bit abstract this summarises everything Protobuf does. Protobuf, along with Go, has also been developed by Google and therefore both can be used very well together. This is probably also the reason why Cosmos, which is written in Go, uses Protobuf.

Furthermore, the gRPC protocol uses Protobuf for the data serialization. gRPC is also integrated into Cosmos which means it is very useful to understand Protobuf and being able to work with it.

How does Protobuf work?

Now as we know what Protobuf can be used for it’s time to look into how a Protobuf data structure looks like:

message Person { required string name = 1; required int32 id = 2; optional string email = 3; }

We’ve now defined something called a “message”, which will act as the base structure for our data. If you are familiar with object-oriented programming languages, you will probably recognise this as a kind of class. And that’s how protobufs are used. Let’s see how this would look in Python:

import person_pb2 # compiled person.proto person = person_pb2.Person() person.name = "Ben" person.id = 1 person.email = "[email protected]"# Alternative person = person_pb2.Person( name="Ben", id=1, email="[email protected]" )

Protobuf does now provide methods to serialize this data into other formats (e.g. json, string, …) or we can submit it through gRPC.

This was a very brief introduction into Protobuf. If you want to learn more you can check out the Protobuf documentation.

Protobuf and Cosmos

Now that we have a general idea of what Protobuf is and what it does, you might want to know how Cosmos uses Protobuf. And actually the answer is quite simple. All data in Cosmos is structured and wrapped by Protobuf. I suggest you check out my cosmospy-protobuf repository on Github. Even if you’re not a Python developer, you’ll find all the relevant Cosmos Protobuf files there.

Knowing this is very helpful when interacting with Cosmos SDK based coins, as you can basically use it as documentation. For example, if you want to query a suggestion, you can look at the corresponding Protobuf file to find out what data you will get.

The Protobuf files can also contain comments which may help you to understand what the different values mean. I find myself using protobufs a lot, and I recommend you do the same. Especially since the documentation in the Cosmos can be very outdated. The protobufs aren’t.

Example Usage

Now that we have the basics covered, I will show you an example of how I use protobufs to make gRPC queries. Our goal is to query all the balances for an account. Before we start, we need to know how gRPC calls work in general:

This shows us that we need a gRPC stub in our client, which acts as an interface to the gRPC client. We will then send a request with our serialised protobuf message and get a response which will also be serialised as a protobuf message. Now we need the appropriate structure for our request and we also need to know what kind of response we can expect. So let’s take a look at the cosmos protobuf files. We will find what we need in the query.proto file belonging to the bank module.

This tells us that we need to pass the address of the account we want to query. We could also use pagination for our request, which would require us to pass a PageRequest message to the request, but we won’t do this in the example as it is optional. The response we get will then contain the balances as repeated Coin. Repeated means that it can contain more than one Coin and represents the data type of an array (or a list, vector, etc, depending on the programming language). However, the balances array can be empty. If there is a balance in it, the item in the array it will be formatted as a Coin structure. Let’s see what that is.

Now we know everything about the data we need for this request. Implementing this in python could look like this:

import grpc import cosmospy_protobuf.cosmos.bank.v1beta1.query_pb2_grpc as query_pb2_grpc import cosmospy_protobuf.cosmos.bank.v1beta1.query_pb2 as query_pb2 host = "osmosis.strange.love" port = "9090" c = grpc.insecure_channel(f'{host}:{port}') stub = query_pb2_grpc.QueryStub(c) r = stub.AllBalances(query_pb2.QueryAllBalancesRequest(address="osmo15hzhcvgs2ljfng6unghvr5l32prwqdyq4aguxn")) for coin in r.balances: print(f"{coin.amount} {coin.denom}")

As you can see Protobuf already provides the stub (interface) for the requestion which means everything we need to do is to pass our protobufs and get the answer

Final words

I hope this has helped you understand protobufs a bit better. There’s a lot more to learn about protobufs, as it also comes with some custom data types. There will be some more specific information about this in the Python tutorial series.

Let me know in the comments if anything is unclear or needs improvement.