scrapio VS nixos-machines

> Nah these things are all stupid as hell. Any back and forth between a human and an LLM in terms of problem solving coding tasks is an absolute disaster.

I actually agree in the general case, but for specific applications these tools can be seriously awesome. Case in point - this repo of mine, which I think it's fair to say was 80% written by GPT-4 via Aider.

https://github.com/epiccoleman/scrapio

Now of course this is a very simple project, which is obviously going to have better results. And if you read through the commit history [1], you can see that I had to have a pretty good idea of what had to be done to get useful output from the LLM. There are places where I had to figure out something that the LLM was never going to get on its own, places where I made manual changes because directing the AI to do it would have been more trouble than it was worth, etc.

But to me, the cool thing about this project was that I just wouldn't have bothered to do it if I had to do all the work myself. Realistically I just wanted to download and process a list of like 15 urls, and I don't think the time invested in writing a scraper would have made sense for the level of time I would have saved if I had to figure it all out myself. But because I knew specifically what needed to happen, and was able to provide detailed requirements, I saved a ton of time and labor and wound up with something useful.

I've tried to use these sorts of tools for tasks in bigger and more complicated repos, and I agree that in those cases they really tend to swing and miss more often than not. But if you're smart enough to use it as the tool it is and recognize the limitations, LLM-aided dev can be seriously great.

[1]: https://github.com/epiccoleman/scrapio/commits/master/?befor...

Thanks for open sourcing this project! I've packaged it with nix to make it easier for others to use: https://github.com/nixvital/ml-pkgs/blob/main/pkgs/aider/def...

If you are running nixos, an example of using it can be found here: https://github.com/breakds/nixos-machines/blob/main/flake.ni...

I have been there. The progress was rather slow until I started to use NixOS. The learning curve is a bit steep but is very rewarding. It is not specific to self-hosting stuff, but as a side effect it makes self hosting super easy (declarative, readable, etc).

For most of the services that you would like, you just write a simple configuration and deploy it. For example, to run the service shiori (https://github.com/breakds/nixos-machines/blob/main/machines...), or to host a game (terraria) server (https://github.com/breakds/nixos-machines/blob/main/machines...), or tailscale (https://github.com/breakds/nixos-machines/blob/main/base/tai...). Since Nix is also a very good package manager, you also do not have to deal with installing packages and managing their dependencies.

With my NixOS server I am running all the services you mentioned, and also my router is just a bunch of services running on a NixOS box.