mindforger VS obsidian-export

Rear is a really interesting project with admirable goals. I believe this is just the beginning, but you have already done a great job!

I have been working on my note-taking application (https://github.com/dvorka/mindforger) for some time and wanted to go in the same direction. However, I gave up (for now). I used ggerganov/llama.cpp to host LLM models locally on a CPU-only machine with 32GB RAM, and used them for both RAG and note-taking use cases (like https://www.mindforger.com/index-200.html#llm). However, it did not work well for me - the performance was poor (high hardware utilization, long response times, failures, and crashes) and the actual responses were rarely useful (off-topic and impractical responses, hallucinations). I tried llama-2 7B with 4b quantization and a couple of similar models. Although I'm not happy about it, I switched to an online commercial LLM because it performs really well in terms of response quality, speed, and affordability. I frequently use the integrated LLM in my note-taking app as it can be used for many things.

Anyway, Reor "only" uses the locally hosted LLM in the generation phase of the RAG, which is a nicely constraint use case. I believe that a really lightweight LLM - I'm thinking about a tiny base model fine-tuned for summarization - could be the way to go (fast, non-hallucinating). I'm really curious to know if you have any suggestions or if you will have any in the future!

As for the vector DB, considering the resource-related problems I mentioned earlier, I was thinking about something similar to facebookresearch/faiss, which, unlike LanceDB, is not a fully-fledged vector DB. Have you made any experiments with similarity search projects or vector DBs? I would be interested in the trade-offs similar to small/large/hosted LLMs.

Overall, I think that both RAG with my personal notes as a corpus and a locally hosted generic purpose LLM for the use cases I mentioned above can take personal note-taking apps to a new level. This is the way! ;)

Good luck with your project!

Please share your suggestions, ideas or constructive criticism! You may install or update from GitHubreleases or PPA.

I finally managed to complete new MindForger release:

https://www.mindforger.com/NimbusnoteWikidpadBecause you mentioned writing:

Neat, the Similarity type reminds me of MindForger's Associations feature that also displays similarity scores between your current note and other existing notes

[Mindfrogger](https://github.com/dvorka/mindforger)

Mindforger: https://github.com/dvorka/mindforger/

Found: https://github.com/zoni/obsidian-export but hope this can be part of a single solution.

I use this obsidian-export CLI program to convert prior to pushing to my repo and it's been working pretty well. This gives me a read-only version of my notes that is accessible from devices I don't have obsidian on (work laptop, for example).

If you limit your use of third party plugins, you can always use https://github.com/zoni/obsidian-export for this as well. I originally built it for exactly this use case (but now also use it as a crucial step in my pipeline to publish content to my own website)

It came from here! https://github.com/zoni/obsidian-export

Worth noting I maintain a project which does exactly this: https://github.com/zoni/obsidian-export

I have similar folders to [Oudwin](https://www.reddit.com/user/Oudwin/)... - dm - _inbox - assets - checklist - communications - research-reference - elements - sessions Additionally, I have had reasonable success using [obsidian-export](https://github.com/zoni/obsidian-export) to export my Obsidian vault to CommonMark. From there you have more options. I then build html pages using [mdbook](https://rust-lang.github.io/mdBook/) to control the information that is revealed to players. I am playing with using [MkDocs](https://www.mkdocs.org/) to see if it offers more control/flexibility. Regardless, the /elements folder contains all the lore chunks of the world including information I keep on the PCs. The /communications and /sessions folders can contain info with links to /elements that are revealed as needed. I make heavy use of transclusion ![[CoolThingFormAnotherFolder]] to keep it a bit more elegant and some custom styles are needed to make it how it look how I wish.

I have had reasonable success using obsidian-export to export a vault to CommonMark. From there you have more options. I am using it for world-building in D&D and I then build html pages using mdbook to control the information that is revealed to players.

Shameless plug: obsidian-export. It will convert [[WikiLinks]] and ![[Embeds]] to plain Markdown (among a few other things) so you'll always have a way to go back if Obsidian doesn't work out the way you hoped.

So really you can't get what you want at all. You could try an external tool like this to export your notes to commonmark which is more widely supported. Ultimately if you are changing the path to files outside of obsidian (meaning they won't be automatically updated) you will break links. So maybe your best bet is to use wikilinks + an export tool.