Lead QA Engineer (AI Voice)

Lingaro
Poland
Remote

Who this role is best for

Aimed at mid-to-senior QA professionals with voice AI and conversational AI expertise who can lead testing processes and manage cross-functional teams.

Best fit for

  • Candidates with 6+ years in QA and 2+ years in voice AI testing in live environments
    — “At least 6 years of experience in software QA. At least 2 years of experience in testing voice agents or conversational AI in live delivery.”
  • Individuals who have led end-to-end and UAT processes independently
    — “Has led E2E and UAT, and worked as the lone or lead QA with limited guidance.”
  • Professionals with strong multilingual and international team experience
    — “Experience in international or distributed teams.”

Things to consider

  • Candidates must be capable of evaluating both predictable and non-deterministic AI behaviors
    — “Test both predictable behaviour (intents, routing, integrations) and non-deterministic AI behaviour.”

How to stand out

  • Highlight your experience in managing AI testing across multiple languages and markets
    — “Define the automation approach and how it scales, including multilingual coverage of around 100 languages.”
  • Demonstrate your ability to lead UAT and coordinate with business testers
    — “Lead UAT: planning, scenarios, test data, coordination of business testers and sign-off recommendation.”
  • Showcase your understanding of voice AI testing, including speech recognition, output, and noise handling
    — “Voice AI testing: speech recognition and speech output, latency, interruptions, accents, noise.”
  • Emphasize your experience with test automation in Python or JavaScript/TypeScript and CI/CD integration
    — “Test automation in Python or JavaScript/TypeScript, with CI/CD and API testing (e.g. Postman).”
  • Demonstrate your ability to communicate structured recommendations to both technical and business stakeholders
    — “Structured, recommendation-first communication with technical and business stakeholders.”
Pace · Fast PacedCollaboration · HighAutonomy · MediumDecision Impact · Team

Derived from job-description analysis by Serendipath's career intelligence engine.

What success looks like

  • Design the voice AI test strategy and process
  • Lead UAT planning and scenarios
  • Own defect and risk management
Typical background
At least 6 years of experience in software QAAt least 2 years of experience in testing voice agents or conversational AI in live delivery

Skills & requirements

Required

Voice AI TestingSpeech Recognition And Output TestingTest AutomationCI/CDAPI TestingTroubleshooting From Logs, Transcripts, And Call Recordings

Preferred

LLM Evaluation ToolsTelephony BasicsIstqb Certification

Stack & domain

PythonJavaScriptTypeScriptPostmanJiraAzure DevopsHammingCekuraCovalBotiumPromptfooDeepevalLangsmithSIPTwilioAzure Communication ServicesARM Cortex-m McusRTOSEthercatCan/can-fdEthernet-based ProtocolsLinuxGitCommunicationTeamworkProblem-solvingIstqbAIVoice RecognitionLLMTelephony

About the role

Original posting from Lingaro via Lever

Growth through diversity, equity, and inclusion. As an ethical business, we do what is right — including ensuring equal opportunities and fostering a safe, respectful workplace for each of us. We believe diversity fuels both personal and business growth. We're committed to building an inclusive community where all our people thrive regardless of their backgrounds, identities, or other personal characteristics.

What you'll be doing:

  • Design the voice AI test strategy and process, from speech input through speech recognition (ASR), the LLM, tools and integrations, to speech output.
  • Define the automation approach and how it scales, including multilingual coverage of around 100 languages.
  • Test both predictable behaviour (intents, routing, integrations) and non-deterministic AI behaviour.
  • Evaluate prompts and LLM responses, using test datasets and scoring criteria.
  • Lead UAT: planning, scenarios, test data, coordination of business testers and sign-off recommendation.
  • Gather business acceptance criteria and turn them into measurable targets.
  • Own defect and risk management, and give release go/no-go recommendations.
  • Replan and communicate options when development is delayed.
  • Challenge stakeholders on readiness and quality, and mentor the Regular engineer.

What we're looking for:

  • At least 6 years of experience in software QA.
  • At least 2 years of experience in testing voice agents or conversational AI in live delivery.
  • Has led E2E and UAT, and worked as the lone or lead QA with limited guidance.
  • Experience in international or distributed teams.
  • Voice AI testing: speech recognition and speech output, latency, interruptions, accents, noise.
  • Voice quality metrics such as word error rate, intent accuracy and task completion.
  • Practical understanding of how LLMs work and fail. Deep LLM engineering is not needed.
  • Test automation in Python or JavaScript/TypeScript, with CI/CD and API testing (e.g. Postman).
  • Use of cloud speech services (Azure, Google, AWS or similar) to generate test calls.
  • Practical experience with Jira or Azure DevOps, and a test management tool.
  • Practical experience with troubleshooting from logs, transcripts and call recordings.
  • Voice testing platforms (Hamming, Cekura, Coval, Botium) is a plus.
  • Experience with LLM evaluation tools (Promptfoo, DeepEval, LangSmith) is a plus.
  • Experience with Telephony basics (SIP, Twilio, Azure Communication Services) is a plus.
  • ISTQB certification is a plus.
  • Experience in Responsible-AI testing (PII, bias, safety) is a plus.
  • Experience validating customer journeys across multiple markets is a plus.
  • Structured, recommendation-first communication with technical and business stakeholders.
  • English at least B2.

We offer:

  • Stable employment. On the market since 2008, 1500+ talents currently on board in 7 global sites.
  • “Office as an option” model. You can choose to work remotely or in the office.  
  • Workation. Enjoy working from inspiring locations in line with our workation policy. 
  • Great Place to Work® certified employer.
  • Flexibility regarding working hours and your preferred form of contract.  
  • Comprehensive online onboarding program with a “Buddy” from day 1.   
  • Cooperation with top-tier engineers and experts.  
  • Unlimited access to the Udemy learning platform from day 1.
  • Certificate training programs. Lingarians earn 500+ technology certificates yearly. 
  • Upskilling support. Capability development programs, Competency Centers, knowledge sharing sessions, community webinars, 110+ training opportunities yearly.
  • Grow as we grow as a company. 76% of our managers are internal promotions.  
  • A diverse, inclusive, and values-driven community.   
  • Autonomy to choose the way you work. We trust your ideas.  
  • Create our community together. Refer your friends to receive bonuses.  
  • Activities to support your well-being and health.
  • Plenty of opportunities to donate to charities and support the environment. 
  • Modern office equipment. Purchased for you or available to borrow, depending on your location.

Source: Lingaro careers (Lever)

Similar roles

More roles at Lingaro