unify-jdocs VS go

JSONPath is good when it comes to querying large JSON documents. But in my opinion, more than this is the need to simplify reading and writing from JSON documents. We use POJOs / model classes which can become a chore for large JSON documents. While it is possible to read paths, I had not seen any tool using which we could read and write JSON paths in a document without using POJOs. And so I wrote unify-jdocs - read and write any JSON path with a single line of code without ever using POJOs. And also use model documents to replace JSONSchema. You can find this library here -> https://github.com/americanexpress/unify-jdocs.

I went the other way - because I so much disliked working with DTOs to work with JSON, I wrote a library that allows you to get rid of DTOs in the first place. No more POJO / DTO in order to work with JSON data and much more. You could take a look at https://github.com/americanexpress/unify-jdocs. I think you will find it interesting!

"And that’s when a real problem emerged; nobody knew who I was" and "I felt invisible, like I was all alone in this world. I could have had the best thing in the world, and no one would care": Could not agree with you more on this - over the past many years it has been more and more important to build a digital brand for yourself and have many followers - to be networked with people and know people who can provide you with reach - something which is difficult for introverts and challenging to do now since I have crossed 50. I personally have felt this helplessness in trying to promote a couple of opensource Java libraries I released on behalf of my employer (shameless plug here -> https://github.com/americanexpress/unify-jdocs and https://github.com/americanexpress/unify-flowret). I thought I had done something of value which would be readily adopted by people after they saw it - guess what - first I have not been able to get through to a wide audience and second - I underestimated what it takes to get people out of their comfort zone and their traditional thinking. It is so difficult for people to accept that there may be better ways of doing things once they get used to a certain way. Anyway, I don't think you failed - you learnt and it is never too late to learn and do something new. Something will click eventually and I wish you all the very best.

Auther here - thanks for responding! I really do appreciate it.

> The trend seems to be in the opposite direction. People became frustrated with the lack of types in Python and JavaScript, hence we get Python with typing and TypeScript.

Regarding typing, what I have tried to achieve is the best of both worlds. In unify-jdocs, we have the concept of typed document. This is a document that defines the structure of the JSON document. It defines what the "type" of a leaf node is i.e. integer, string etc. The validation / determination against this type is however done at runtime. This, I feel is acceptable because whenever we add a JSON path to a document, the first thing we would do is to test the read / write of that path. And any type mismatch would get caught there immediately. And so, from the point of view of being able to read / write / validate in a single line of code (even though dynamically) provides much simplicity and ease of use as compared to using POJOs. Plus we always know the exact JSON path we are dealing with.

Usually, I would prefer static typing but, in a scenario, where we can have hundreds of JSON document types (we deal with more than 500 in the same application), complex JSON document (ours go down more than ten levels deep with hundreds of JSON paths) and where the document structures may undergo change over project lifetime, the use of read / writing using a single line of code has many benefits. I shudder to think of hundreds (if not thousands) of POJO classes, the writing of accessor methods (null / empty handling, namespace etc.) and what it would take to refactor in the face of change. Just my opinion based on my experience in the past.

> I think you will find less traction with this method of posting to HN. People want a clickable link to look at the project

The clickable link to the repo / documentation is actually in the text (https://github.com/americanexpress/unify-jdocs). In the case of this post, I felt it more important for people to read the text rather than be directly pointed to the contents of the link and hence this approach.

Nice work! I see that that this is for processing / parsing large data sets and where documents do not conform to a fixed structure and for Go language.

I made something similar in Java - unify-jdocs - https://github.com/americanexpress/unify-jdocs - though this is not for parsing - it is more for reading and writing when the structure of the document is known - read and write any JSONPath in one line of code and use model documents to define the structure of the data document (instead of using JSONSchema which I found very unwieldy to use) - no POJOs or model classes - along with many other features. Posting here as the topic is relevant and it may help people in the Java world. We have used it intensively within Amex for a very large complex project and it has worked great for us.

Nice! Talking of JSON schemas and validating JSON documents against schemas, for Java, I wrote unify-jdocs where I do not use JSON schemas but still do validations (I found them unwieldy to use and was looking for something simpler). You can find details here -> https://github.com/americanexpress/unify-jdocs. Also, no POJOs / model classes, just reading and writing JSON paths in a single line of code. It's helped us tremendously in managing complexity in a very large internal project. I am hoping it helps others.

This article got my attention. Related to what you are saying, in Java, the problem that I was really fed up of was creating domain specific JSON object models to map the JSON documents into to use in code. In other words, mapping JSON to rigidly typed language structure. Its boiler plate, is tedious to do (as the author points out in the article), difficult to change and usually a pain. I solved this problem by creating unify-jdocs which completely eliminates the need to create object models or POJO classes to represent your JSON object. You can read more about it here -> https://github.com/americanexpress/unify-jdocs

The description indicates it is not production ready, and is archived at the same time.

If you pull all stops in each respective language, C# will always end up winning at parsing text as it offers C structs, pointers, zero-cost interop, Rust-style struct generics, cross-platform SIMD API and simply has better compiler. You can win back some performance in Go by writing hot parts in Go's ASM dialect at much greater effort for a specific platform.

For example, Go has to resort to this https://github.com/golang/go/blob/4ed358b57efdad9ed710be7f4f... in order to efficiently scan memory, while in C# you write the following once and it compiles to all supported ISAs with their respective SIMD instructions for a given vector width: https://github.com/dotnet/runtime/blob/56e67a7aacb8a644cc6b8... (there is a lot of code because C# covers much wider range of scenarios and does not accept sacrificing performance in odd lengths and edge cases, which Go does).

Another example is computing CRC32: you have to write ASM for Go https://github.com/golang/go/blob/4ed358b57efdad9ed710be7f4f..., in C# you simply write standard vectorized routine once https://github.com/dotnet/runtime/blob/56e67a7aacb8a644cc6b8... (its codegen is competitive with hand-intrinsified C++ code).

There is a lot more of this. Performance and low-level primitives to achieve it have been an area of focus of .NET for a long time, so it is disheartening to see one tenth of effort in Go to receive so much spotlight.

A Discussion about including this package in Go as encoding/json/v2 has been started on the Go Github project on 2023-10-05. Please provide your feedback there.

I like the Principles section. Very measured and practical approach to releasing new stdlib packages. https://go.dev/blog/randv2#principles

The end of the post they mention that an encoding/json/v2 package is in the works: https://github.com/golang/go/discussions/63397

There used to be the GO FIPS branch :

https://github.com/golang/go/tree/dev.boringcrypto/misc/bori...

But it looks dead.

And it looks like https://github.com/golang-fips/go as well.

I'm not sure what exactly you mean by acknowledgement, but here are some counterexamples:

- A proposal for sum types by a Go team member: https://github.com/golang/go/issues/57644

- The community proposal with some comments from the Go team: https://github.com/golang/go/issues/19412

Here are some excerpts from the latest Go survey [1]:

- "The top responses in the closed-form were learning how to write Go effectively (15%) and the verbosity of error handling (13%)."

- "The most common response mentioned Go’s type system, and often asked specifically for enums, option types, or sum types in Go."

I think the problem is not the lack of will on the part of the Go team, but rather that these issues are not easy to fix in a way that fits the language and doesn't cause too many issues with backwards compatibility.

[1]: https://go.dev/blog/survey2024-h1-results

Now, I’m not going to use C++ again; I left that chapter years ago, and it’s not going to happen. C++ isn’t memory safe and easy to use and would require extended time for developers to adapt. Rust is the new kid on the block, but I’ve heard mixed opinions about its developer experience, and there aren’t many libraries around it yet. LLRD is too new for my taste, but **Go** caught my attention.

Generative AI development has been democratised, thanks to powerful Machine Learning models (specifically Large Language Models such as Claude, Meta's LLama 2, etc.) being exposed by managed platforms/services as API calls. This frees developers from the infrastructure concerns and lets them focus on the core business problems. This also means that developers are free to use the programming language best suited for their solution. Python has typically been the go-to language when it comes to AI/ML solutions, but there is more flexibility in this area. In this post you will see how to leverage the Go programming language to use Vector Databases and techniques such as Retrieval Augmented Generation (RAG) with langchaingo. If you are a Go developer who wants to how to build learn generative AI applications, you are in the right place!

net/http: add methods and path variables to ServeMux patterns Discussion about ServeMux enhancements

Make sure you have Go installed https://go.dev/.