JSON in data science projects: tips & tricks

• orjson

  17 5,629 8.3 Python

  Fast, correct Python JSON library supporting dataclasses, datetimes, and numpy

orjson is the fastest JSON library available for python. It natively manages dataclass objects, datetime, numpy and UUID objects.

• NumPy

  272 26,510 10.0 Python

  The fundamental package for scientific computing with Python.

  

Data science projects often use numpy. However, numpy objects are not JSON-serializable and therefore require conversion to standard python objects in order to be saved:

• InfluxDB

  www.influxdata.com featured

  Power Real-Time Data Analytics at Scale. Get real-time insights from all types of time series data with InfluxDB. Ingest, query, and analyze billions of data points in real-time with unbounded cardinality.

  

• jsonlines

  6 118 3.9 CSS

  Documentation for the JSON Lines text file format

This can be remedied by using the [JSON Lines] format (https://jsonlines.org/). This involves nothing more and nothing less than placing one JSON object per line, so that you can browse the objects without having to parse the entire collection all at once.

Suggest a related project