Every disappointing retrieval-augmented system I have looked at had the same diagnosis, and it was never the generation step.

Measure retrieval separately

Before judging any answer, ask whether the supporting document was in the retrieved set at all. If it was not, no amount of prompt engineering will help, and you have been debugging the wrong component. Build a small labelled set — a hundred questions with their correct sources — and track recall on it. It is unglamorous and it is the whole game.

Things that reliably help

• Hybrid search. Dense vectors miss exact identifiers, product codes and names; keyword search catches them. Run both and fuse the rankings.
• Chunking that respects document structure. Splitting at a fixed token count cuts tables in half and separates a heading from what it introduces.
• Keeping the metadata. Date, department and document type filter out most wrong answers before ranking even begins.

The interesting research is in the model. The value in your application is almost entirely in the retrieval.