Overview
The Agentic AI Orchestration System using Google Cloud is a cloud-native framework that routes prompts across multiple LLMs (Groq, Together, Ollama), evaluates responses using heuristic scoring and human-in-the-loop feedback, and logs results in BigQuery for monitoring. Built with Google Cloud Functions, Pub/Sub, and Python utilities, it supports real-time orchestration, zero-downtime model hot-swapping, and CLI-based debugging. Core algorithms include dynamic prompt routing, parallel inference, response evaluation, and config-driven hot-swapping, with planned extensions for semantic scoring, ensemble ranking, and fine-tuned evaluators.