Tech conglomerate enhances GenAI chatbot with TrainAI
To strengthen their competitive advantage, a global technology company wanted to improve the accuracy and capabilities of their generative AI chatbot. However, they quickly realized they needed help to find sufficient technical, data and linguistic resources to evaluate, review and refine prompt-response pairs. TrainAI by ɫƵ improved the client’s AI chatbot by harnessing the expertise of content writers, AI data specialists and linguists across the globe.
Large Language Models, or LLMs, are a type of artificial intelligence (AI) trained on vast amounts of text data to understand and generate human-like language.
They produce natural language outputs that enable users to engage in human-like conversations, ask for help to complete tasks, compose multilingual content and much more. When developing new tools based on rapidly evolving LLM technologies, the sheer volume of data required to train and fine-tune LLMs is always a challenge.
Our client wanted to protect the competitive advantage of its proprietary search engine by improving the capabilities and accuracy of its generative AI chatbot. The chatbot was designed to gather and compile real-time data from the internet and deliver it to users browsing for information in an interactive manner.
The client’s goals
Gain a competitive advantage against a rival company developing an LLM-powered chatbot to research and deliver internet-derived content
Secure sufficient technical, data and linguistic resources to train its new model
Train and fine-tune the chatbot with enhanced prompt-response pairs to improve its accuracy
Challenges
Gain a competitive advantage against a rival LLM-powered chatbot
Secure sufficient technical, data and linguistic expertise to help train the chatbot
Train and fine-tune the chatbot on specific topic areas to improve its accuracy, performance and usability across the globe
Rapidly recruited and onboarded 5,000 writers, AI data specialists and linguists
Delivered 264,000 hours of work in 8 months
Localized chatbot’s UI, help articles and marketing materials
Review and improve 30,000+ chatbot responses per week
Providing support for 53 locales and counting
A perfect mix of AI know-how and talent
The client chose ɫƵ for its ability to handle large-scale projects, its AI, content and language expertise, and its vast pool of in-house and external talent.
Key to this decision was the experience of ɫƵ’s TrainAI team, who provides comprehensive AI data and generative AI training and fine-tuning services, including:
Prompt engineering
Reinforcement learning from human feedback (RLHF)
Red teaming/jailbreaking
Domain expertise
Locale-specific support in any language and at any scale
The unique combination of scalability and know-how, as well as our preferred vendor status, made ɫƵ an ideal choice.
264,000 hours of work in 8 months
5,000 AI data experts, writers and linguists
30,000+ AI responses processed per week
Recruiting and training at speed
Upon project launch, the first challenge was to meet the tight timelines for the volume of work required – up to 30,000 responses per week – with a flexible, scalable team.
Recruiting got off to a quick start due to TrainAI’s ability to tap into ɫƵ’s Editorial Services team, which includes a large pool of in-house writers, and its TrainAI community of AI data specialists. This approach ensured a diverse and representative team of project contributors located across the globe that was key to project success.
TrainAI created a complete writer assessment and onboarding program that included comprehensive training delivered via face-to-face calls and pre-recorded sessions based on documentation provided by the client.
Enhancing prompt-response pairs to optimize chatbot performance
Writers were required to review chatbot responses (outputs) to specific prompts (inputs) and improve them using two primary approaches:
Prompt creation, editing and integration: To keep efforts focused, writers were provided with general topics and asked to create new prompts to input into the chatbot. The writer would then evaluate the chatbot’s output and rewrite or restructure the response as needed.
Response review and validation, and fact extraction and verification: The team reviewed the source data that the chatbot used to generate its response for incorrect factual information or details that may have been missed. If issues were found, the writer addressed them by restructuring or rewriting the response.
Rating chatbot responses to improve output quality
Rating generative AI responses is a crucial aspect of its learning and improvement. Response ratings function as labels that guide the model to adjust its parameters and generate better quality responses. This creates a reward signal for reinforcement learning, enabling the model to iteratively fine-tune itself based on response feedback. This enhances its ability to produce accurate, high-quality and contextually relevant responses over time.
In addition to enhancing the AI chatbot’s prompt-response pairs, the client also asked the team to perform:
Response rating: TrainAI data specialists rated prompt responses in accordance with the client’s guidelines, focusing on four key pillars: creativity, factuality, persona and safety.
Expanding the chatbot’s global reach with locale-specific support
Most of the work to date has been completed in English. However, to expand the chatbot’s global reach, the client also required:
Locale-specific chatbot support: TrainAI was able to leverage ɫƵ’s in-house Language Services team to deliver expertise in Japanese, Korean, Brazilian Portuguese, Mexican Spanish and 48 other languages. This effort expanded the chatbot’s language capabilities, with the goal of adding additional locales as the chatbot’s performance becomes more robust.
UI and content localization: TrainAI was also able to leverage ɫƵ’s core translation and localization competencies to localize the chatbot’s user interface (UI), help articles, marketing materials and perform accessibility testing in multiple languages.
TrainAI from ɫƵ excelled by delivering a complete range of generative AI fine-tuning services to improve the chatbot’s performance - all from a single provider.
Gaining a competitive edge in a fast-moving field
TrainAI helped the technology conglomerate’s generative AI chatbot gain a competitive advantage against its rivals by:
Rapidly scaling the team to 5,000 AI data specialists from the TrainAI community, as well as writers and linguists from ɫƵ’s Editorial and Language Services teams
Delivering in excess of 264,000 hours of work within the first 8 months of the project
Processing more than 30,000 chatbot responses per week
Providing support for 53 locales and counting
Delivering the complete range of services required to improve the chatbot’s performance all from a single provider
The project scope continues to expand to meet the client’s needs.
Shaping the future of AI
Over 2,000 ɫƵ linguists now play an integral role in Canva’s user journey, helping make every touchpoint—from onboarding and template customization to educational resources—feel natural to millions of local users.
The localized content that ɫƵ creates is attracting strong interest from Canva’s international audience. For example, a campaign landing page for Canva’s Droptober 2024 event had 40,000 call-to-action clicks within the first seven days.
