Leveraging Large Language Models to Build a Low-Resource Customer Support Chatbot in a Three-Sided Marketplace

EasyChair Preprint no. 13944

Abstract

The development of fully autonomous conversational agents using large language models (LLMs) remains a significant challenge due to the inherent complexity and critical importance of customer-facing systems. This paper investigates a hybrid approach that integrates LLMs within traditional task-oriented systems to enhance performance while maintaining safety and reliability.

We propose a modular design, incorporating LLMs into specific modules of the system to leverage modern architecture capabilities while mitigating risks like hallucinations. Our work outlines the architectural design, assesses the performance of the latest LLMs on intent classification tasks, discusses unique challenges posed by three-sided marketplaces, and trade-offs in designing and deploying such systems at scale.

We demonstrate how this system can achieve real-world impact with minimal investments in architecture, modeling, and dataset collection, achieving up to a 35% reduction in workload while maintaining customer satisfaction.

Keyphrases: conversational agents, dialogue systems, Generative AI, Intent Classification, large language models

@Booklet{EasyChair:13944,
  author = {Biagio Antonelli and Gonzalo Cordova},
  title = {Leveraging Large Language Models to Build a Low-Resource Customer Support Chatbot in a Three-Sided Marketplace},
  howpublished = {EasyChair Preprint no. 13944},

  year = {EasyChair, 2024}}