machine learning in snowflake, unhappy data scientists

• spark-streaming-jdbc-source

  2 25 1.8 Scala

I think it is more about efficiency and not about limits. If you use JDBC/ODBC, it means your process is running outside of Snowflake, you need transfer data over network into your engine. If you use SparkSQL to read a JDBC source, it doesn't support streaming data - you need to load whole data into DataFrame before you process it. To stream you will need some custom workaround, for example that. If you work with Snowpark you don't need it (not sure how it works internally).

• dbt-fal

  12 851 7.7 Python

  Discontinued do more with dbt. dbt-fal helps you run Python alongside dbt, so you can send Slack alerts, detect anomalies and build machine learning models.

  

Happy data scientists use fal and dbt

• InfluxDB

  www.influxdata.com featured

  Power Real-Time Data Analytics at Scale. Get real-time insights from all types of time series data with InfluxDB. Ingest, query, and analyze billions of data points in real-time with unbounded cardinality.

  

Suggest a related project