Methods and Instruments for Determining the Reliability for LLM and RAG Systems in Renewable Energy Domain

This paper explores the evaluation of large language models (LLMs) and retrieval-augmented generation (RAG) systems within the renewable energy domain, emphasizing the need for reliable advisory tools in high-stakes technical applications. We review existing evaluation methods, identifying gaps in answer quality, source grounding, and system stability. To address these, we propose a three-dimensional evaluation framework integrating correctness, attribution accuracy, and robustness, tailored for renewable energy's technical complexity and regulatory demands. This framework supports users without deep AI expertise and fits into RAG chatbot development and operational cycles. Drawing from a literature review, we synthesize complementary evaluation approaches and highlight domain-specific adaptations, including handling specialized terminology, multi-source integration, and evolving standards. Our study underlines the importance of combining automated and human-centered evaluation to ensure trustworthy deployment of LLM/RAG systems, bridging current methodological gaps and fostering safer, more transparent AI applications in renewable energy.