AI Avatar at the 141st Congress of the German Society of Surgery
Industry: Healthcare
About the project
In collaboration with apoQlar, we developed a cutting-edge AI Avatar that was showcased from April 23 to 26, 2024, during the 141st Congress of the German Society of Surgery (DCK 2024) at the Leipzig Congress Centre. The primary aim of the AI Avatar was to create an innovative interface for engaging with congress visitors, providing information about the event, the main organizer, and answering questions regarding the event schedule—all through speech interaction. The avatar was presented in two formats: on a large screen with a microphone and in Virtual Reality using the Meta Quest 3 headset.
Challenges
During the execution of our project, we faced several significant challenges that required innovative solutions and careful planning:
- Communication Barriers: The congress venue was extremely noisy, crowded with numerous stands and visitors, which made clear communication difficult.
- Latency: Minimizing latency was crucial to ensure smooth, real-time conversations with the avatar.
- Session Scheduling: With a densely packed schedule featuring many parallel events, recommending sessions required precise and efficient handling of information.
Solution
To effectively tackle the challenges faced during the project, we implemented a comprehensive solution architecture utilizing state-of-the-art technology:
- Advanced Language Models: We utilized Large Language Models (LLMs) for enhanced natural language understanding and generation.
- Optimized Speech Models: The fastest text-to-speech and speech-to-text models were employed to ensure seamless verbal interactions.
- Efficient Communication System: Web sockets and a message queue system were implemented for efficient communication between all components, with partial parallelization to reduce latency.
- Intelligent Retrieval System: We integrated a function calling mechanism with a Retrieval Augmented Generation (RAG) architecture to manage additional information and provide accurate responses.
- Noise Reduction Techniques: Professional microphones and software-level adaptations were used to mitigate noise and enhance performance in the noisy congress environment.
Technological Insights
The success of the AI Avatar was primarily attributed to the seamless integration of multiple advanced technologies:
- Large Language Models (LLMs): These models facilitated a sophisticated understanding and generation of human language, which was pivotal for the system’s core functionalities.
- Text-to-Speech and Speech-to-Text Models: These technologies provided rapid and natural-sounding speech synthesis along with precise speech recognition capabilities, enhancing user interactions.
- Web Sockets and Message Queue Systems: These components ensured efficient, real-time communication capabilities while minimizing latency through effective parallel processing techniques.
- Function Calling Mechanism and Retrieval-Augmented Generation (RAG) Architecture: These technologies allowed for dynamic information retrieval and accurate responses, significantly enhancing the avatar’s interactive abilities.
- Professional Audio Equipment: This was integrated with specialized software adaptations to effectively manage noisy environments, ensuring clear audio output under varied conditions.
Through this innovative approach, we successfully created an AI Avatar that not only met but exceeded the expectations of the congress attendees, providing an engaging interactive experience.
Contact us
How can we assist you? Please give us a brief description of your project so that we can provide you with the best possible support.