Apache Spark
Visual Studio Code
Apache Spark | Visual Studio Code | |
---|---|---|
101 | 2,875 | |
38,605 | 159,341 | |
0.6% | 0.6% | |
10.0 | 10.0 | |
7 days ago | 5 days ago | |
Scala | TypeScript | |
Apache License 2.0 | MIT License |
Stars - the number of stars that a project has on GitHub. Growth - month over month growth in stars.
Activity is a relative number indicating how actively a project is being developed. Recent commits have higher weight than older ones.
For example, an activity of 9.0 indicates that a project is amongst the top 10% of the most actively developed projects that we are tracking.
Apache Spark
- "xAI will open source Grok"
-
Groovy 🎷 Cheat Sheet - 01 Say "Hello" from Groovy
Recently I had to revisit the "JVM languages universe" again. Yes, language(s), plural! Java isn't the only language that uses the JVM. I previously used Scala, which is a JVM language, to use Apache Spark for Data Engineering workloads, but this is for another post 😉.
-
🦿🛴Smarcity garbage reporting automation w/ ollama
Consume data into third party software (then let Open Search or Apache Spark or Apache Pinot) for analysis/datascience, GIS systems (so you can put reports on a map) or any ticket management system
-
Go concurrency simplified. Part 4: Post office as a data pipeline
also, this knowledge applies to learning more about data engineering, as this field of software engineering relies heavily on the event-driven approach via tools like Spark, Flink, Kafka, etc.
-
Five Apache projects you probably didn't know about
Apache SeaTunnel is a data integration platform that offers the three pillars of data pipelines: sources, transforms, and sinks. It offers an abstract API over three possible engines: the Zeta engine from SeaTunnel or a wrapper around Apache Spark or Apache Flink. Be careful, as each engine comes with its own set of features.
-
Apache Spark VS quix-streams - a user suggested alternative
2 projects | 7 Dec 2023
-
Integrate Pyspark Structured Streaming with confluent-kafka
Apache Spark - https://spark.apache.org/
-
Spark – A micro framework for creating web applications in Kotlin and Java
A JVM based framework named "Spark", when https://spark.apache.org exists?
- Rest in Peas: The Unrecognized Death of Speech Recognition (2010)
-
PySpark SparkSession Builder with Kubernetes Master
I recently saw a pull request that was merged to the Apache/Spark repository that apparently adds initial Python bindings for PySpark on K8s. I posted a comment to the PR asking a question about how to use spark-on-k8s in a Python Jupyter notebook, and was told to ask my question here.
Visual Studio Code
-
Essential Tools & Technologies for New Developers
For beginners, the best code editor is Vscode.
-
How to Handle File Uploads with ASP.NET Core
An IDE or text editor; we'll use Visual Studio 2022 for this tutorial, but a lightweight IDE such as Visual Studio Code will work just as well
-
How to Scrape Google Finance
Choosing IDE: Selecting the right Integrated Development Environment (IDE) can make your coding experience smoother. Consider popular options like as PyCharm, Visual Studio Code, or Jupyter Notebook. Install your preferred IDE and configure it to work with Python.
-
Tools that keep me productive
It all starts with the editor. Visual Studio Code (VS Code) is my go-to editor. I was using the Insider’s Edition for the longest time, but some extensions would try to log in and redirect to VS Code regular edition, so I decided to go back to it. That said, VS Code Insider's is very stable.
-
Developing a Generic Streamlit UI to Test Amazon Bedrock Agents
Meanwhile, a developer workflow that does not require access to AWS Management Console may provide a better experience. As a developer, I appreciate having an integrated development environment (IDE) such as Visual Studio Code where I can code, deploy, and test in one place.
-
How to make ESLint and Prettier work together? 🛠️
Good to know: If you're a Visual Studio Code user, you can enhance your coding experience by installing the ESLint and Prettier extensions. These extensions provide real-time error and warning highlighting, as well as automatic formatting and code fixing on save.
-
Create a simple Server using Express.js.
Download any code editor e.g. VS code. Visual Studio code which is a code editor with support for development operations like debugging, task running, and version control. Go to https://code.visualstudio.com
-
How to Add Firebase Authentication To Your NodeJS App
A code editor (VS Code is my go-to IDE), but feel free to use any code editor you're comfortable with.
-
Create a Chat App With Node.js
First, grab your favorite command-line tool, Terminal or Warp, and a code editor, preferably VS Code and let’s begin.
-
Asynchronous Programming in C#
C# is very good as a language, have developed in it for 5+ years. The problem is the gap between what MSFT promises to management and actually delivers to developers. You really really need to fully read the fine print, think of the omissions in documentation and implement a proof-of-concept that almost implements the full solution to find out the hidden gotchas.
For example, even probably their best product VS Code only got reasonable multiple screens support last year: https://github.com/microsoft/vscode/issues/10121#issuecommen...
And then, on the other end of the spectrum, you have Teams.
What are some alternatives?
Trino - Official repository of Trino, the distributed SQL query engine for big data, former
thonny - Python IDE for beginners
Pytorch - Tensors and Dynamic neural networks in Python with strong GPU acceleration
reactide - Reactide is the first dedicated IDE for React web application development.
Airflow - Apache Airflow - A platform to programmatically author, schedule, and monitor workflows
Spyder - Official repository for Spyder - The Scientific Python Development Environment
Scalding - A Scala API for Cascading
doom-emacs - An Emacs framework for the stubborn martian hacker [Moved to: https://github.com/doomemacs/doomemacs]
mrjob - Run MapReduce jobs on Hadoop or Amazon Web Services
KDevelop - Cross-platform IDE for C, C++, Python, QML/JavaScript and PHP
luigi - Luigi is a Python module that helps you build complex pipelines of batch jobs. It handles dependency resolution, workflow management, visualization etc. It also comes with Hadoop support built in.
vscodium - binary releases of VS Code without MS branding/telemetry/licensing