Top 23 Go Serialization Projects

• jsoniter

  12 13,100 0.0 Go

  A high-performance 100% compatible drop-in replacement of "encoding/json" (by json-iterator)

  

Project mention: Handling high-traffic HTTP requests with JSON payloads | /r/golang | 2023-12-07

Since most of the time would be spent decoding json, you could try to cut this time using https://github.com/bytedance/sonic or https://github.com/json-iterator/go, both are drop-in replacements for the stdlib, sonic is faster.

• goprotobuf

  13 9,573 2.8 Go

  Go support for Google's protocol buffers (by golang)

  

Project mention: Protoc Plugins with Go | dev.to | 2023-08-19

Now let’s take a look at the source code of the protoc-gen-go plugin:

• InfluxDB

  www.influxdata.com featured

  Power Real-Time Data Analytics at Scale. Get real-time insights from all types of time series data with InfluxDB. Ingest, query, and analyze billions of data points in real-time with unbounded cardinality.

  

• mapstructure

  16 7,709 0.0 Go

  Go library for decoding generic map values into native Go structures and vice versa.

Project mention: How do I marshal a JSON array into a map? | /r/golang | 2023-12-07

• gogoprotobuf

  8 5,633 0.0 Go

  [Deprecated] Protocol Buffers for Go with Gadgets (by gogo)

  

• msgpack

  3 2,300 5.1 Go

  msgpack.org[Go] MessagePack encoding for Golang (by vmihailenco)

Project mention: Rob Pike: Gobs of data (2011) | news.ycombinator.com | 2023-12-04

Someone made a benchmark of serialization libraries in go [1], and I was surprised to see gobs is one of the slowest ones, specially for decoding. I suspect part of the reason is that the API doesn't not allow reusing decoders [2]. From my explorations it seems like both JSON [3], message-pack [4] and CBOR [5] are better alternatives.

By the way, in Go there are a like a million JSON encoders because a lot of things in the std library are not really coded for maximum performance but more for easy of usage, it seems. Perhaps this is the right balance for certain things (ex: the http library, see [6]).

There are also a bunch of libraries that allow you to modify a JSON file "in place", without having to fully deserialize into structs (ex: GJSON/SJSON [7] [8]). This sounds very convenient and more efficient that fully de/serializing if we just need to change the data a little.

--

1: https://github.com/alecthomas/go_serialization_benchmarks

2: https://github.com/golang/go/issues/29766#issuecomment-45492...

--

3: https://github.com/goccy/go-json

4: https://github.com/vmihailenco/msgpack

5: https://github.com/fxamacker/cbor

--

6: https://github.com/valyala/fasthttp#faq

--

7: https://github.com/tidwall/gjson

8: https://github.com/tidwall/sjson

• go-codec

  1 1,823 6.5 Go

  idiomatic codec and rpc lib for msgpack, cbor, json, etc. msgpack.org[Go] (by ugorji)

• hprose

  0 1,255 7.1 Go

  Hprose is a cross-language RPC. This project is Hprose for Golang.

  

• SaaSHub

  www.saashub.com featured

  SaaSHub - Software Alternatives and Reviews. SaaSHub helps you find the best software and product alternatives

  

• go-capnp

  3 1,158 6.3 Go

  Cap'n Proto library and code generator for Go

  

Project mention: Sandstorm: Open-source platform for self-hosting web app | news.ycombinator.com | 2023-06-04

The Go implementation of capnp is alive and well, too. I highly recommend it: https://github.com/capnproto/go-capnp

• csvutil

  1 894 4.3 Go

  csvutil provides fast and idiomatic mapping between CSV and Go (golang) values.

• cbor

  4 666 8.6 Go

  CBOR codec (RFC 8949) with CBOR tags, Go struct tags (toarray, keyasint, omitempty), float64/32/16, big.Int, and fuzz tested billions of execs.

Project mention: Rob Pike: Gobs of data (2011) | news.ycombinator.com | 2023-12-04

Someone made a benchmark of serialization libraries in go [1], and I was surprised to see gobs is one of the slowest ones, specially for decoding. I suspect part of the reason is that the API doesn't not allow reusing decoders [2]. From my explorations it seems like both JSON [3], message-pack [4] and CBOR [5] are better alternatives.

By the way, in Go there are a like a million JSON encoders because a lot of things in the std library are not really coded for maximum performance but more for easy of usage, it seems. Perhaps this is the right balance for certain things (ex: the http library, see [6]).

There are also a bunch of libraries that allow you to modify a JSON file "in place", without having to fully deserialize into structs (ex: GJSON/SJSON [7] [8]). This sounds very convenient and more efficient that fully de/serializing if we just need to change the data a little.

--

1: https://github.com/alecthomas/go_serialization_benchmarks

2: https://github.com/golang/go/issues/29766#issuecomment-45492...

--

3: https://github.com/goccy/go-json

4: https://github.com/vmihailenco/msgpack

5: https://github.com/fxamacker/cbor

--

6: https://github.com/valyala/fasthttp#faq

--

7: https://github.com/tidwall/gjson

8: https://github.com/tidwall/sjson

• karmem

  20 630 0.6 Go

  Karmem is a fast binary serialization format, faster than Google Flatbuffers and optimized for TinyGo and WASM.

• go-capnproto

  0 288 0.0 Go

  Cap'n Proto library and parser for go. This is go-capnproto-1.0, and does not have rpc. See https://github.com/zombiezen/go-capnproto2 for 2.0 which has rpc and capabilities.

• watson

  6 270 1.3 Go

  WATSON: Wasted but Amazing Turing-incomplete Stack-based Object Notation (by genkami)

• php_session_decoder

  0 161 0.0 Go

  PHP session encoder/decoder written in Go

• structomap

  0 144 0.0 Go

  Easily and dynamically generate maps from Go static structures

• binstruct

  0 90 4.7 Go

  Golang binary decoder for mapping data into the structure

• bebop

  1 67 6.8 Go

  bebop wire format in Go (by 200sc)

• bambam

  0 65 0.0 Go

  auto-generate capnproto schema from your golang source files. Depends on go-capnproto-1.0 at https://github.com/glycerine/go-capnproto

• mus-go

  6 63 7.2 Go

  MUS format serializer with validation support for Golang

  

Project mention: Serializing complex data structures with mus-go | /r/golang | 2023-10-31

Now, thanks to the pm package, mus-go can serialize data structures such as a graph or linked list.

• parco

  3 60 0.0 Go

  🏇🏻 generalist, fast and tiny binary parser and compiler generator, powered by Go 1.18+ Generics

• polyglot

  1 39 7.2 Go

  A high-performance serialization framework used for encoding and decoding arbitrary datastructures across languages. (by loopholelabs)

  

• bel

  0 37 0.0 Go

  Generate TypeScript interfaces from Go structs/interfaces - useful for JSON RPC

• go-cardano-serialization

  1 29 0.0 Go

  Golang library for serialization & deserialization of Cardano data structures.

  

• SaaSHub

  www.saashub.com featured

  SaaSHub - Software Alternatives and Reviews. SaaSHub helps you find the best software and product alternatives

  

Go Serialization related posts

• Handling high-traffic HTTP requests with JSON payloads

  5 projects | /r/golang | 7 Dec 2023

• How do I marshal a JSON array into a map?

  1 project | /r/golang | 7 Dec 2023

• Is there any equivalent to pydantic, serde, etc?

  8 projects | /r/golang | 6 Dec 2023

• Rob Pike: Gobs of data (2011)

  10 projects | news.ycombinator.com | 4 Dec 2023

• A Journey building a fast JSON parser and full JSONPath

  5 projects | news.ycombinator.com | 11 Oct 2023

• Protoc Plugins with Go

  2 projects | dev.to | 19 Aug 2023

• Struggling to get JSON response data into usable struct

  2 projects | /r/golang | 6 Jul 2023
• A note from our sponsor - InfluxDB
  www.influxdata.com | 20 May 2024

  Get real-time insights from all types of time series data with InfluxDB. Ingest, query, and analyze billions of data points in real-time with unbounded cardinality. Learn more →

Index

Sponsored

SaaSHub - Software Alternatives and Reviews

SaaSHub helps you find the best software and product alternatives

www.saashub.com

Do not miss the trending Go projects with our weekly report!