For my single user instance, I can be charitable and say that it’s running on hardware that I already had that is running regardless on spare otherwise unused resources with a already registered domain so the only cost is time spent setting it up. Or I could apply all the costs from the server Lemmy, then it would be about $1200 initially plus ~$10/mo per user.
What you are looking for is a RAG and is one of the few legitimately useful implementations of LLMs outside the wall of hype.