Measurable Productivity Gains with LLMs and AI Agents

LLMs and AI agents: what they can do

Many companies want to build their own AI systems: internal chatbots, smart assistants or fully automated workflows based on generative language models. Key questions often remain open. Which technical components are needed? How can internal knowledge be integrated securely? And how does that become a reliable system in production?

Large language models (LLMs) are AI models trained on large volumes of text. They understand, analyse and generate language. In companies they are usually deployed in two ways:

• General-purpose models such as GPT, Claude or Gemini are powerful but do not know the company’s data.
• Enterprise-customised models integrate internal knowledge through vector databases or retrieval-augmented generation (RAG). That lets them answer questions based on data from SharePoint, CRM systems or internal knowledge bases. For specific tasks they can also be fine-tuned on internal data.

AI agents extend language models with goals, decision logic and process knowledge. They respond to input and also take on tasks independently: they research, evaluate sources, summarise content, plan steps, adapt to intermediate results and make decisions. Through interfaces such as APIs, databases or ERP systems, they work with other systems.

At first glance, AI agents look like powerful chatbots. A classic chatbot answers individual questions or holds simple dialogues. An agent pursues a goal and automates entire processes without each step having to be triggered. A typical example is summarising large amounts of information from technical documentation or regulatory texts, even when they run to hundreds or thousands of pages.

The components of an AI agent

A typical AI agent consists of several building blocks that together enable targeted, autonomous information processing:

• Goal definition: it starts with a clear goal, for example “summarise the key findings from our project reports” or “analyse the regulatory requirements in current legislation”.
• Planning: the agent breaks the task into steps and plans which information it needs, where to find it and in what order to process it.
• Problem solving with the language model: the model writes queries, processes content, creates summaries or assessments and decides how to proceed with the results.
• Execution: the agent works with connected systems, retrieves content from databases and document archives, uses APIs and searches internal knowledge sources.
• Results and communication: finally it delivers reports, answers, summaries or processed data for downstream systems, as text, structured output or visualisations.
• Context memory and feedback: previous steps, decisions and intermediate results are stored, so the agent works consistently and learns from feedback.

Our RPP Group case study shows what a system of several specialised agents looks like in daily use: time spent on routine work fell by 70%.

What studies show about productivity gains

• Harvard Business School and BCG (2023): in a field experiment with 758 Boston Consulting Group consultants, participants using GPT-4 on suitable tasks completed 12.2% more tasks, worked 25.1% faster and delivered more than 40% higher quality than the control group. On tasks outside the model’s capabilities, results got worse.
• MIT, published in Science (2023): on writing tasks for 453 professionals, ChatGPT reduced the time taken by 40% and raised quality by 18%.
• McKinsey (2023): generative AI could add $2.6 trillion to $4.4 trillion in economic value annually, especially in customer operations, marketing and sales, software engineering, and research and development.
• Microsoft and LinkedIn, Work Trend Index (2024): 75% of knowledge workers use AI at work, and 79% of leaders agree that adopting AI is critical to staying competitive.

Sources: Dell’Acqua et al., Navigating the Jagged Technological Frontier; Noy and Zhang, Experimental Evidence on the Productivity Effects of Generative AI; McKinsey, The economic potential of generative AI; Microsoft, 2024 Work Trend Index

Three efficiency levers in practice

1.Text and documentation

Proposals, minutes, summaries or FAQs are drafted in seconds. Specialists review and refine them instead of starting from scratch.

2.Information retrieval, internal and external

With vector databases and RAG, chatbots access company-specific knowledge directly, in context and up to date. Employees ask in natural language and get an answer with a source.

3.Process automation with AI agents

They check requests, create quotes or update CRM data. In a case study on expense processing at a large Korean company, processing time for paper receipts fell by more than 80%, with fewer errors and better compliance.

Source: Jeong et al., E2E Process Automation Leveraging Generative AI and IDP-Based Automation Agent, arXiv 2025

How a production-ready chatbot or AI agent is built

A company’s own LLM system is built in several coordinated steps:

• Choose the language model: options include large models from OpenAI, Anthropic, Google, Mistral or Meta as well as smaller open-source models such as Phi, Gemma, Qwen or DeepSeek. The choice depends on performance, data protection, cost and technical flexibility. Our article Local LLMs in Practice explains when a locally hosted model pays off.
• Connect internal data: for the system to answer company-specific questions, documentation, database entries and knowledge articles are indexed and made semantically accessible through vector databases and RAG.
• Prompt engineering and system logic: this defines tasks, roles and rules: how detailed answers should be, which tone to use and which functions the system may trigger. Guardrails prevent unwanted answers and protect sensitive data.
• Integrate into existing interfaces: the system is embedded where people already work, for example in Microsoft Teams, Slack, the intranet or custom web apps.
• Governance, security and quality assurance: access rights, usage logs, feedback functions and monitoring ensure the system stays compliant, keeps improving and runs reliably in production.

Challenges in adoption and scaling

• Data protection: companies often work with personal data, internal strategy documents or confidential client data. Clear data processing rules, tiered access rights and a deliberate choice between cloud and on-premise build trust and meet legal requirements.
• Quality assurance: language models can produce false information. Outputs should be reviewed regularly and checked against facts. RAG helps because the model draws on reliable internal sources.
• Acceptance: if employees do not understand or trust the technology, its potential goes unused. Hands-on training, open communication and easy-to-grasp use cases build trust.
• Scalability: a promising prototype is not enough if it cannot be used in production. Mechanisms for feedback, evaluation and continuous improvement therefore belong in from the start.

Conclusion: the value comes from connecting to your own data

Companies that invest in LLMs and AI agents gain more efficient workflows, easier access to knowledge, better quality and noticeably relieved teams. The models create the most value where they are closely connected to internal systems, data and workflows. AI agents make that connection usable, as systems that act and deliver real business value.

A structured start, solid technical foundations and an actively managed rollout lay the groundwork for lasting impact. We help identify the right use cases, develop solutions and integrate them securely into existing infrastructure, from internal knowledge assistants to AI agent development.

Frequently asked questions