Custom LLM development for your data and your systems

Custom LLM development means adapting a large language model to one company’s documents, vocabulary and processes, and building the system around it: validation, integration into existing software and a deployment that meets the company’s security rules. The adaptation can be prompt engineering, retrieval-augmented generation (RAG), fine-tuning, or a combination. theBlue.ai builds these systems on-premise, in the cloud or hybrid.

Use RAG when the knowledge changes often and every answer needs a source, for example product documentation or internal policies. Fine-tune when the vocabulary is highly specialised, the output style is fixed, or a smaller model has to process a high volume. Many tasks, such as extraction from email, work with prompt engineering alone. We start with the lightest approach that meets the requirement and decide on the data, in the process analysis.

No. For sensitive processes theBlue.ai runs local models on your own hardware, where your documents never leave your network and cannot reach any provider. When we use a cloud provider, we always make sure the option to use your data for training is switched off. With API deployments, your data is not used for training by default. Which route fits is settled before anything is built.

Yes. theBlue.ai deploys on-premise, in the cloud, hybrid or with a hosted model, depending on your security requirements and infrastructure. For Tirol Kliniken the system runs on the hospital’s own servers, and for Fr. Meyer’s Sohn in Docker on the company’s servers. We also run three local models in production for our own work.

Both hosted and open-source models. As an OpenAI Select Partner we use OpenAI’s models where they fit; the Fr. Meyer’s Sohn system uses GPT-3.5 and GPT-4. For local deployments we work with open-source families such as Gemma, Llama and Qwen. The model is chosen per task, weighing accuracy, speed, context length, language support and hardware.

Our systems in production handle German and English, and several handle more: the pharmacovigilance system reads medical texts in German, English, French, Italian and Spanish, and our tender screening works on Polish documents. Language support is one of the criteria when the model is chosen.

First working components typically arrive six to eight weeks into the build. Where a system already exists, it can be faster: we took Radaway’s existing LLM system from technical review to production in three weeks.

The process analysis has a fixed price from €3k and ends with an architecture proposal that states scope, timeline and cost for the build. The build itself is priced in milestones, so you know the cost of each step before it starts. You decide after the analysis whether to proceed.

Usually without starting again. Prototypes that work in controlled conditions and fail in production are a common pattern, and the cause normally sits in a few components: prompts, output formats, validation and edge cases. We review what exists, keep what works and rebuild the rest. That is how Radaway’s system reached production in three weeks.

The system goes live with real data and we monitor it closely, refine it on real-world performance and hand over with full documentation and training. Ongoing support is available, which matters most for local models: they have to be hosted, monitored and updated.