openmeter
metering-operator
openmeter | metering-operator | |
---|---|---|
11 | 1 | |
925 | 338 | |
5.5% | -0.3% | |
9.9 | 0.0 | |
4 days ago | 11 days ago | |
Go | Go | |
Apache License 2.0 | Apache License 2.0 |
Stars - the number of stars that a project has on GitHub. Growth - month over month growth in stars.
Activity is a relative number indicating how actively a project is being developed. Recent commits have higher weight than older ones.
For example, an activity of 9.0 indicates that a project is amongst the top 10% of the most actively developed projects that we are tracking.
openmeter
- OpenMeter – open-source Realtime Metering
- Ask HN: Which JavaScript lib for metering APIs should I use?
- Looking for feedback on our website for an Open Source project
-
ClickHouse for usage metering with Kafka Connect
This plugin guarantees exactly-once delivery between Kafka topics and ClickHouse tables, which is critical, as Kafka Connect tasks are only aware of the latest topic offset acknowledged by the consumer. For example, consumers can fail to acknowledge a processed offset due to a network error or an exception. This is great as exactly-once inserts prevent dropping or double-inserting usage, leading to incorrect billing.
In OpenMeter, we pre-aggregate usage events into one-minute tumbling windows to reduce the number of rows we need to scan at query time. To do this, with ClickHouse, we use the AggregatingMergeTree table engine that enables incremental data aggregation when combined with MaterializedView. In ClickHouse, MaterializedViews are trigger-based and update when new records are inserted into the source table. Consequently, the corresponding materialized views are updated whenever Kafka Connect transfers a batch of events to ClickHouse. This also means inserts can fail when the view cannot process a record at trigger. We send failed events into the Dead Letter Queue topic for later processing.
To help ClickHouse with hot topics, we will consider adding an extra streaming aggregation step for high-producers, but this time with a more horizontally scalable stream processor like Arroyo. This would reduce ClickHouse insert batch sizes. Based on our tests, ClickHouse works best if batch sizes are 50-100k and less frequent than per second.
To see it in action, check out our open-source repo: https://github.com/openmeterio/openmeter
- Real-Time and Scalable Usage Metering
- GitHub - openmeterio/openmeter: Accurate and real-time usage metering for AI, DevOps, billing and analytics.
-
How to meter POD execution duration for billing?
I’m researching how to meter a POD execution duration for auditable use cases like billing. We are working on an infrastructure metering solution, and our current example polls the k8s API every second to see which pods are running: see code here - wondering what folks think about this approach.
-
Real-Time, Open Source Usage Metering built on Kafka
OpenMeter leverages stream processing to update meters in real-time while simultaneously processing large volumes of events. The core is written in Go and uses the CloudEvents format to describe usage, Kafka to ingest events, and ksqlDB to dedupe and aggregate meters. We are also working on a Postgres sink for long-term storage. Check out our GitHub to learn more: https://github.com/openmeterio/openmeter
- Launch HN: OpenMeter (YC W23) – Real-Time, Open Source Usage Metering
metering-operator
-
K8s cost accounting/metering option
Metering Operator. But looks like it's not under active development.
What are some alternatives?
lago - Open Source Metering and Usage Based Billing API ⭐️ Consumption tracking, Subscription management, Pricing iterations, Payment orchestration & Revenue analytics
argocd-operator - A Kubernetes operator for managing Argo CD clusters.
vanus - Vanus is a Serverless, event streaming system with processing capabilities. It easily connects SaaS, Cloud Services, and Databases to help users build next-gen Event-driven Applications.
opensearch-k8s-operator - OpenSearch Kubernetes Operator
kafka_manager - Simplifies eventing between microservices using kafka with kafka-go client
capsule - Multi-tenancy and policy-based framework for Kubernetes.
sloth - 🦥 Easy and simple Prometheus SLO (service level objectives) generator
kubeplus - Kubernetes Operator to create Kubernetes-native APIs from Helm charts for multi-instance SaaS