buckaroo VS sketch

I created buckaroo [1] as a better dataframe viewer for jupyter with built in summary stats. It's built to bring a better dataframe experience to people already using pandas/polars. All of it is extensible [2] so that you can customize stats and transformations to your workflow.

[1] https://github.com/paddymul/buckaroo

Great tables has done some really nice work on python/jupyter tables. It looks like they are almost building a "grammar of tables" similar to a grammar of graphics. More projects should write about their philosophy and aims like this.

I have built a different table library for jupyter called buckaroo. My approach has been different. Buckaroo aims to allow you to interactively cycle through different formats and post-processing functions to quickly glean important insights from a table while working interactively. I took the view that I type the same commands over and over to perform rudimentary exploratory data analysis, those commands and insights should be built into a table.

Great tables seems built so that you can manually format a table for presentation.

https://github.com/paddymul/buckaroo

https://youtu.be/GPl6_9n31NE

JLisp.

3. It was very easy to define new lowcode commands, and have the frontend add them to the palette. Each command defines two methods "transform" which manipulates the dataframe, and "transform_to_py" which takes the same arguments but emits python code.

Adoption of my library in general, and the low code UI specifically has been very limited. I'm in the middle of plumbing the lowcode support back in after a refactor of other parts.

I would like to build a whole ecosystem around JLisp and Buckaroo. Specifically I have some "auto-cleaning" functionality that emits JLisp cleaning and normalization commands, these commands can then be editted in the UI (delete, edit parameters). It's easier to emit JLisp than raw python syntax, it's also much easier to make a UI to manipulate it.

Do you have a repo to look at? What usecase did you have in mind when you were building it?

If I were evaluating a low-code backend builder I'd be interested in the examples, and tests. Hopefully the tests would double as examples. For a Workflow type low-code-builder I'd be most interested in the cron functionality.

[1] https://github.com/paddymul/buckaroo

[2] http://norvig.com/lispy2.html

I used Norvig’s lisp2.py to build a low code UI. I modified the interpreter to accept JSON flavored lisp, basically replace parens with brackets. The upside is that it was very very easy to make a react front end that manipulates JSON (JLisp). My thinking was, I need a serialization format for operations from the front end, and a way to interpret them. I could write my own language that no one has heard of, or use lisp, which few have used.

https://github.com/paddymul/buckaroo/blob/main/buckaroo/jlis...

Location: Boston

Remote: Yes

Willing to relocate: Yes

Technologies: talking to users, python, pandas/numpy, jupyter, js/ts

Résumé/CV: https://www.linkedin.com/in/paddymullen/

Email: [email protected]

In my next role, I want a broad mandate to make a meaningful impact within an organization by developing products that address genuine business challenges, with a preference for data related problems.

Recently I built the data table for Jupyter/Pandas that I have wanted for over a decade. The open source Buckaroo (https://github.com/paddymul/buckaroo) data table combines a performant table, summary statistics, and a low code UI to expedite common data analysis tasks.

wanted for over a decade. The open source Buckaroo https://github.com/paddymul/buckaroo data table combines a performant table, summary statistics, and a low code UI to

This morning I added a "Related Projects" [3] Section to the Buckaroo docs. If Buckaroo doesn't solve your problem, look at one of the other linked projects (like Mito).

[1] https://github.com/approximatelabs/sketch

[2] https://github.com/paddymul/buckaroo

[3] https://buckaroo-data.readthedocs.io/en/latest/FAQ.html

We've made a lot of data tooling things based on LLMs, and are in the process of rebranding and launching our main product.

1. sketch (in notebook, ai for pandas) https://github.com/approximatelabs/sketch

2. datadm (open source, "chat with data", with support for the open source LLMs (https://github.com/approximatelabs/datadm)

3. Our main product: julyp. https://julyp.com/ (currently under very active rebrand and cleanup) -- but a "chat with data" style app, with a lot of specialized features. I'm also streaming me using it (and sometimes building it) every weekday on twitch to solve misc data problems (https://www.twitch.tv/bluecoconut)

For your next question, about the stack and deploy:

This morning I added a "Related Projects" [3] Section to the Buckaroo docs. If Buckaroo doesn't solve your problem, look at one of the other linked projects (like Mito).

[1] https://github.com/approximatelabs/sketch

[2] https://github.com/paddymul/buckaroo

[3] https://buckaroo-data.readthedocs.io/en/latest/FAQ.html

For GPT/Copilot style help for pandas, in notebooks REPL flow (without needing to install plugins), I built sketch. I genuinely use it every-time I'm working on pandas dataframes for a quick one-off analysis. Just makes the iteration loop so much faster. (Specifically the `.sketch.howto`, anecdotally I actually don't use `.sketch.ask` anymore)

https://github.com/approximatelabs/sketch

https://github.com/approximatelabs/lambdaprompt It has served all of my personal use-cases since making it, including powering `sketch` (copilot for pandas) https://github.com/approximatelabs/sketch

Core things it does: Uses jinja templates, does sync and async, and most importantly treats LLM completion endpoints as "function calls", which you can compose and build structures around just with simple python. I also combined it with fastapi so you can just serve up any templates you want directly as rest endpoints. It also offers callback hooks so you can log & trace execution graphs.

All together its only ~600 lines of python.

I haven't had a chance to really push all the different examples out there, but most "complex behaviors", so there aren't many patterns to copy. But if you're comfortable in python, then I think it offers a pretty good interface.

I hope to get back to it sometime in the next week to introduce local-mode (eg. all the open source smaller models are now available, I want to make those first-class)

Please look at sketch and langchain pandas/SQL plugins. I have seen excellent results with both of these approaches. Both of these approaches will require you to send metadata to openAI.

👉 Understand your data through questions 👉 Create code from plain text Quick Read: https://www.marktechpost.com/2023/02/01/meet-sketch-an-ai-code-writing-assistant-for-pandas/ Github: https://github.com/approximatelabs/sketch

(3) Asking for re-writes of failed queries (happens occasionally) also helps

The main challenge I think with a lot of these "look it works" tools for data applications, is how do you get an interface that actually will be easy to adopt. The chat-bot style shown here (discord and slack integration) I can see being really valuable, as I believe there has been some traction with these style integrations with data catalog systems recently. People like to ask data questions to other people in slack, adding a bot that tries to answer might short-circuit a lot of this!

We built a prototype where we applied similar techniques to the pandas-code-writing part of the stack, trying to help keep data scientists / data analysts "in flow", integrating the code answers in notebooks (similar to how co-pilot puts suggestions in-line) -- and released https://github.com/approximatelabs/sketch a little while ago.

Python AI Helper https://github.com/approximatelabs/sketch