Senior IVA QA Automation Engineer

This role is intended exclusively for Asia-Pacific residents. Please only apply if you're located in the APAC region.

About DevSavant

DevSavant is a boutique staff augmentation firm serving B2B Tech & AI companies in the US. We specialize in integrating top-tier LATAM professionals into client teams with speed, precision, and cultural alignment. Our mission is to be an extension of our clients' organizations, building sustainable, high-performance teams that scale fast and deliver results.

Responsibilities:

1. Stakeholder Engagement & Requirements Gathering

   • Coordinate with product, development, and QA Director

   • Define business and technical requirements for voice testing

2. Current State Analysis

   • Document existing manual test cases and tools

   • Analyze conversation flows and performance metrics

3. Scope Definition & Prioritization

   • Identify high-priority intents and modules

   • Define automation boundaries and test coverage goals

4. Tool Evaluation & Selection

   • Assess and choose tools for STT, TTS, test automation, and CI/CD

5. Framework Design & Environment Setup

   • Architect the voice testing framework

   • Set up environments, SDKs, APIs, and version control

   • Establish CI/CD pipelines

6. Test Development & Automation

   • Build reusable components and utilities

   • Automate high-priority test cases (intent, flow, entity, fallback)

   • Expand to multi-turn and regression scenarios

7. CI/CD Integration

   • Integrate voice tests into nightly runs and PR triggers

   • Validate deployments and enable parallel execution

8. Test Data & Reporting

   • Manage test data setup, teardown, and utterance variations

   • Implement reporting, playback, and performance metrics

9. Stabilization & Coverage Expansion

   • Fix flaky tests and improve reliability

   • Expand coverage to 70–80% of regression suite

   • Add multi-language, accent, and device testing

10. Non-Functional Testing

    • Implement performance, load, latency, and security testing

11. Optimization & Documentation

    • Optimize execution time and resource usage

    • Finalize documentation, best practices, and usage guides

12. Training & Handover

    • Train QA and developers on framework usage and maintenance

    • Conduct retrospectives and plan for continuous improvement

Requirements:

Technical Skills:

• Programming: Expert in Python for in-house tool development and test scripting..

• IVA Testing: Proficiency in testing AI-driven voice applications (e.g., NLP, ASR) using tools like Twilio Autopilot, Cyara, or Dialogflow.

• Tools: Experience with Postman, Cyara, Twilio.

• CI/CD: Expertise in building pipelines from scratch (Jenkins, GitHub Actions, GitLab CI) and integrating voice testing environments.

• Telephony: Knowledge of SIP and cloud telephony platforms (e.g., Twilio, AWS Connect) for voice-to-voice IVA automation.

Soft Skills:

• Analytical mindset to debug complex IVA failures.

• Strong collaboration with developers and stakeholders for framework design and defect resolution.

• Clear documentation and KPI reporting for transparency.

Experience:

• 3+ years in QA automation, with 1+ years in IVA systems.

• Proven experience building QA frameworks and CI/CD pipelines from scratch.

• Hands-on expertise in voice-to-voice call automation and AI-driven IVA testing.