Engineering Manager - Customer Experience AI
Coinbase · India · 8+ yrs experience · Posted 2026-06-04
Tech stack: AWS, Go, Golang, Kubernetes, Ruby
About the role
Lead AI Strategy & Execution: Drive the roadmap for our conversational AI stack, moving beyond simple decision trees into LLM-driven reasoning, RAG, and agentic workflows. Orchestrate the AI Ecosystem: Oversee the integration of third-party AI solutions while simultaneously scaling our in-house LLM infrastructure to handle high-stakes crypto support queries. Build Evaluation & Guardrails: Establish rigorous AI evaluation frameworks (LLM-as-a-judge) and feedback loops to ensure our models are accurate, grounded, and compliant with global financial regulations. Agentic Automation: Move from "chat" to "action" by building secure pathways for AI agents to perform complex tasks (e.g., transaction troubleshooting, account recovery) via internal APIs. Drive Technical Architecture: Define how we handle vector databases, prompt engineering, and context window management to provide a personalised experience for every Coinbase user. Operational Excellence: Own the reliability of AI services, including latency optimisation, cost management (token usage), and fallback mechanisms to human agents. Responsibilities: - Lead AI Strategy & Execution: - Drive the roadmap for our conversational AI stack, moving beyond simple decision trees into LLM-driven reasoning, RAG, and agentic workflows. - Orchestrate the AI Ecosystem: Oversee the integration of third-party AI solutions while simultaneously scaling our in-house LLM infrastructure to handle high-stakes crypto support queries. - Build Evaluation & Guardrails: Establish rigorous AI evaluation frameworks (LLM-as-a-judge) and feedback loops to ensure our models are accurate, grounded, and compliant with global financial regulations. - Agentic Automation: Move from "chat" to "action" by building secure pathways for AI agents to perform complex tasks (e.g., transaction troubleshooting, account recovery) via internal APIs. - Drive Technical Architecture: - Define how we handle vector databases, prompt engineering, and context window management to provide a personalised experience for every Coinbase user. - Operational Excellence: - Own the reliability of AI services, including latency optimisation, cost management (token usage), and fallback mechanisms to human agents. Qualifications: - 8+ years of software engineering experience, with 2+ years leading high-performing teams in a fast-paced environment. - Hands-on AI/ML Leadership: Proven experience shipping products powered by Large Language Models (LLMs). You understand the nuances of prompt engineering, fine-tuning, and the current landscape of model providers (OpenAI, Anthropic, etc.). - Systems Thinking: - Experience building RAG (Retrieval-Augmented Generation) pipelines and managing the data lifecycle required to ground AI in real-time knowledge. - Platform Mindset: You’ve built scalable, distributed systems and understand how to integrate AI components into a high-traffic production environment (Go, Ruby, or similar). - Evaluation Obsessed: You don’t just "vibe check" AI; you have experience with quantitative evaluation frameworks to measure hallucination rates, accuracy, and customer sentiment. - Security & Safety First: A deep understanding of how to build AI "guardrails"—ensuring models don't leak PII or hallucinate financial advice. - Nice to haves: - Experience with Vector Databases (e.g., Pinecone, Weaviate, Milvus) and AI Orchestration frameworks (e.g., LangChain, LlamaIndex) - Experience in FinTech or Crypto, specifically navigating the balance between AI innovation and strict regulatory/compliance requirements. - Background in NLP (Natural Language Processing) or traditional Machine Learning before the Generative AI boom. - Proficiency in Golang and experience with modern cloud-native infrastructure (AWS, Kubernetes). - Job ID - P75782
Qualifications
- 8+ years of software engineering experience, with 2+ years leading high-performing teams in a fast-paced environment.
- Hands-on AI/ML Leadership: Proven experience shipping products powered by Large Language Models (LLMs).
- You understand the nuances of prompt engineering, fine-tuning, and the current landscape of model providers (OpenAI, Anthropic, etc.).
- Systems Thinking: Experience building RAG (Retrieval-Augmented Generation) pipelines and managing the data lifecycle required to ground AI in real-time knowledge.
- Platform Mindset: You’ve built scalable, distributed systems and understand how to integrate AI components into a high-traffic production environment (Go, Ruby, or similar).
- Evaluation Obsessed: You don’t just "vibe check" AI
- you have experience with quantitative evaluation frameworks to measure hallucination rates, accuracy, and customer sentiment.
- Security & Safety First: A deep understanding of how to build AI "guardrails"—ensuring models don't leak PII or hallucinate financial advice.
- Nice to haves:
- Experience with Vector Databases (e.g., Pinecone, Weaviate, Milvus) and AI Orchestration frameworks (e.g., LangChain, LlamaIndex)
- Experience in FinTech or Crypto, specifically navigating the balance between AI innovation and strict regulatory/compliance requirements.
- Background in NLP (Natural Language Processing) or traditional Machine Learning before the Generative AI boom.
- Proficiency in Golang and experience with modern cloud-native infrastructure (AWS, Kubernetes).
- Job ID P75782
Responsibilities
- Lead AI Strategy & Execution:
- Drive the roadmap for our conversational AI stack, moving beyond simple decision trees into LLM-driven reasoning, RAG, and agentic workflows.
- Orchestrate the AI Ecosystem: Oversee the integration of third-party AI solutions while simultaneously scaling our in-house LLM infrastructure to handle high-stakes crypto support queries.
- Build Evaluation & Guardrails: Establish rigorous AI evaluation frameworks (LLM-as-a-judge) and feedback loops to ensure our models are accurate, grounded, and compliant with global financial regulations.
- Agentic Automation: Move from "chat" to "action" by building secure pathways for AI agents to perform complex tasks (e.g., transaction troubleshooting, account recovery) via internal APIs.
- Drive Technical Architecture:
- Define how we handle vector databases, prompt engineering, and context window management to provide a personalised experience for every Coinbase user.
- Operational Excellence: Own the reliability of AI services, including latency optimisation, cost management (token usage), and fallback mechanisms to human agents.