Ai Engineer

Job Title: AI Engineer

Job Description

This role focuses on designing, building, and deploying production-ready AI and multi-agent systems within enterprise environments. You will serve as the primary AI engineering point of contact for sprint-based engagements, partnering closely with domain experts to translate business needs into robust, scalable solutions. The position emphasizes solution architecture, agent orchestration, observability, evaluation discipline, and the creation of reusable patterns that enable other teams to build AI agents effectively and reliably.

Responsibilities

• Embed within product or service line teams as the primary AI engineering point of contact for short, sprint-based engagements.
• Lead discovery sessions with subject matter experts to clarify goals, constraints, workflows, and adoption requirements.
• Deliver at least bi-weekly demos to stakeholders and drive alignment from initial discovery through go-live.
• Operate under sponsor-driven prioritization while maintaining strong engineering rigor and production readiness.
• Translate business and technical requirements into clear agent architectures, data requirements, and execution plans.
• Design and build multi-agent systems using Azure AI Foundry and orchestration frameworks, including routing, state management, memory, tool use, retries, fallbacks, and guardrails.
• Integrate AI agents with enterprise systems such as SharePoint, Teams, Azure AI Search, Databricks, SQL databases, and internal or external APIs as needed.
• Make pragmatic engineering tradeoffs to deliver value quickly while preserving reliability, security, and maintainability.
• Own the deployment of AI solutions into production, primarily as Azure-hosted services, typically using containers or other appropriate delivery mechanisms.
• Implement production-grade practices including environment separation, configuration management, secure secrets handling, telemetry-by-default, and rollback-friendly release strategies.
• Design solutions that operate effectively within enterprise constraints, including identity, access, compliance, and platform standards, without requiring deep specialization in any single corporate system.
• Ensure all delivered agents follow established tracing, logging, and observability patterns.
• Instrument agent systems using supported tooling such as Application Insights, Agent 365, LangSmith, and other evolving internal standards.
• Contribute to the definition and adoption of agent registration standards, including required metadata, automated population, and consistent inventory and visibility across the organization.
• Help standardize patterns and foundations so that AI delivery can scale across teams without repeatedly reinventing core components.
• Establish robust evaluation approaches that combine curated golden datasets, human review workflows, and automated testing both before and after go-live.
• Create and maintain evaluation datasets, including labeling strategies, edge cases, and regression test packs as part of delivery.
• Explain how evaluation metrics are calculated, when they are trustworthy, and how they translate into concrete system improvements rather than superficial metrics.
• Develop reusable assets such as reference architectures, templates, scaffolding, test harnesses, and practical "how-to" playbooks for AI agent development.
• Coach and support other teams building agents in platforms such as Copilot Studio, providing patterns, guardrails, integration guidance, and operational expectations where applicable.

Essential Skills

• 3–5+ years of professional engineering experience delivering production software or services.
• Strong proficiency in Python, including building services, integrations, agent workflows, and evaluation harnesses.
• Proven ability to translate ambiguous requirements into clear, buildable architectural and implementation plans.
• Hands-on experience building LLM-powered agents and/or orchestration workflows, with multi-agent systems strongly preferred.
• Experience with Azure AI Foundry or comparable platforms for agents, evaluations, deployments, and safety or control mechanisms.
• Experience with orchestration frameworks such as LangGraph (or similar) and a strong understanding of stateful orchestration and tool-driven execution.
• Experience implementing tracing, logging, and observability for AI systems in production, including correlation IDs, traceability across agent and tool calls, and telemetry to support debugging and iteration.
• Comfort owning deployments into enterprise environments and working within established platform standards.
• Practical understanding of LLM behavior in real-world scenarios, including context limits, embeddings and RAG fundamentals, tool use failure modes, nondeterminism, and common evaluation pitfalls.
• Strong skills in artificial intelligence and applied AI engineering within cloud environments such as Azure.

Additional Skills & Qualificati