-
InfluxDB
Power Real-Time Data Analytics at Scale. Get real-time insights from all types of time series data with InfluxDB. Ingest, query, and analyze billions of data points in real-time with unbounded cardinality.
We're fans of ag, The Silver Searcher.
fyi ack is like this but for perl instead of python https://beyondgrep.com/
Unrelated, are you the same burntsushi that wrote xsv?
Oh yes, absolutely. I ran into a similar problem with a project I'm working on, and I had a conditional that would only import numpy if it already existed. Then again, mine wasn't something you'd call all the time, so the slowdown was worth it. Ultimately, I wound up using a C library to allocate the array (and to shuffle the ints), but it's worth noting that this is an array of uint32_t, and I'm not performing any regexes on them.
It's vitally important that you make tests on small changes, and also have consistent testing methodology - same file and search parameters from one run to the next. I personally like using hyperfine to call the Python script, but anything should work. Also, if you're testing this on a laptop, it needs to be plugged in (and the CPU power management tuned for max performance when on wall power), and not too hot - thermal throttling can kill test results.