Protobuf is so much faster than JSON

At work, I have a program that needs to load an external JSON file that is 1.4GB in size, it takes about 45 seconds for it to load and json.Unmarshal. I took that 45 second start-up time for granted.

Today, I thought, what if I convert that data to protobuf? Will it speed up the start-up time?

(P.S. there’s a very good benchmark of different go serialization here: https://github.com/alecthomas/go_serialization_benchmarks)

So I did the switch. Data encoded in protobuf is only 481MB in size, and to my surprise, loading the data only took 4 seconds!

Such great improvement… I wish I had made the switch earlier.

It makes me think, for Kubernetes APIServer, most clients are still only accepting JSON for LIST requests. JSON will probably consume a lot more APIServer CPU, and more bandwidth. Making the switch to protobuf could potentially solve a lot of problems.