LLM-Based Conversational AI Agents: Prompt Engineering, Design, and Enterprise Standards

Abstract

Enterprise adoption of large language model (LLM)-based conversational agents has outpaced the design guidance available to build them responsibly. Teams building a low-stakes internal helpdesk bot and teams building a regulated, customer-facing assistant frequently draw on the same generic prompt-engineering advice, despite facing materially different risk profiles. We propose a requirement-driven framework linking functional and non-functional requirements to concrete choices in prompt-engineering technique, agent architecture, model selection, and governance control. Building on prompting and agent research together with enterprise AI-governance frameworks, we introduce a layered reference architecture, a requirement-to-technique mapping, and model-selection criteria for production-grade conversational agents, illustrated through a generalized enterprise scenario rather than a specific client engagement. The aim is to give architects and technical leads a structured path from ad hoc LLM chatbot pilots to conversational agents that are reliable, secure, and governable at production scale.