Izkušnje z uporabo lokalnih modelov LLM v sistemu COBISS

In the past years, we have developed a smart assistant in the COBISS system based on large language models and the RAG approach. The initial implementation was based on the use of external services (OpenAI), but then we simplified the development and improved the scalability of the solutions by introducing local models and the LangChain4j library. Positive experiences have confirmed our strategic orientation towards the use of locally installed open source models, which allows for greater control over data, greater privacy and lower costs. We have also extended artificial intelligence to other areas of the COBISS system. In the COBISS Lib application, we have upgraded existing assistants and additionally developed several new solutions. The CAT-AI microservice enables automated extraction of bibliographic metadata from documents and images. The AI-supported help desk enables semantic search in the internal knowledge base. We have implemented natural language search, which converts user queries into structured search requests for searching bibliographic or research sources. The solutions are based on the Java ecosystem (Quarkus, LangChain4j) and microservices architecture. The paper presents practical experiences in implementing local models and integrating artificial intelligence into the COBISS system.