A SaaS platform letting businesses stand up an AI-powered support agent with configurable infrastructure: choice of LLM provider, choice of self-hosted or shared vector DB, and custom ticketing integration - each combination provisioned on demand and delivered as a single embeddable iframe.

The Challenge

Every tenant could independently choose their LLM (hosted API like ChatGPT, or self-hosted OpenLlama/Mistral/Meta via Ollama), their vector DB (shared platform infra or self-hosted ChromaDB), and their ticketing setup (shared or dedicated OSTicket/Zoho instance with custom credentials) - turning deployment into a combinatorial infrastructure problem rather than a single config. The client additionally required everything to run on AWS, including the self-hosted options, which meant working within AWS account-level service limits (Lambda, Docker, EC2) as tenant count grew, not just writing infrastructure-as-code once and walking away.

Key Decisions & Tradeoffs

• Built a job queue with live status tracking surfaced in the Flask admin portal, driving Python scripts that performed the actual per-tenant provisioning - since no single Terraform run could account for every possible LLM/vector-DB/ticketing combination a tenant might choose.
• Minimized container footprint deliberately: Debian-slim and Python-slim base images, trimmed OS and Python dependencies for Ollama and ChromaDB specifically, to keep per-tenant deployment overhead and AWS resource consumption low.
• Implemented a RAG context layer using ChromaDB so each tenant's agent could be grounded in their own uploaded policy documents, rather than relying on generic model responses.
• Worked around AWS account-level quota restrictions that surfaced as tenant count grew, since the client required even self-hosted tenant options to run on AWS infrastructure (Lambda and Docker, not just EC2).
• Shipped in two agile increments under tight time pressure: a first pass with API-only LLM integration (ChatGPT) at 100 man-hours, followed by the full self-hosted-capable version at 480 total man-hours - split across 2 developers, each simultaneously carrying two other concurrent projects within a 45-hour work week.

Outcome

Delivered a working multi-tenant platform supporting 7+ LLM providers behind a single embeddable iframe, with subscription tiers reflecting infrastructure choice (shared vs. dedicated). Final phase of the project was handed off to the client as a completed deliverable.

Built under NDA at iTechnolabs; described here at an architectural level. Client identity and specific business details are withheld.