QA Azure AI Engineer

EDV Werke is looking for a QA Azure AI Engineer

Working Model: On-site, Kuala Lumpur

Form of cooperation: B2B Contract

Tasks:

• Generative AI Development – Build and implement AI solutions using generative models and RAG pipelines.

• Cloud AI Deployment – Deploy, manage, and maintain AI models on cloud platforms with scalable architecture.

• Python Engineering – Write complex Python scripts for data processing, testing, and model automation.

• QA & Evaluation – Evaluate models using advanced metrics, including LLM-as-a-Judge and deterministic assertions.

• DataOps & CI/CD Pipelines – Develop automated pipelines for AI testing and deployment on cloud platforms.

• Technical Leadership – Provide guidance on AI model architecture and best practices for deployment.

• Cross-team Collaboration – Work with data, backend, and product teams to integrate AI solutions into production.

• Documentation & Reporting – Maintain reproducible workflows, technical documentation, and experiment results.

Must-Have Skills:

• Generative AI Expertise – 1–2 years hands-on experience with generative AI and RAG architecture (vector search + LLM).

• Cloud Platform Experience – 5+ years deploying AI models on cloud AI services, not just locally.

• Python Specialist – 7+ years coding in Python, including complex scripting (Pandas, PyTest) beyond basic automation.

• QA & Automation Knowledge – 8+ years QA experience with LLM evaluation and test automation.

• DataOps & CI/CD – 5+ years building pipelines for automated AI model testing and deployment.

• Technical Degree – Degree in CS, Engineering, Math, or equivalent heavy experience.

• Azure AI Deployment – Practical experience deploying and managing AI models in cloud AI services.

• Evaluation Metrics Mastery – Ability to design and interpret advanced AI evaluation metrics and regression testing.

Should-Have Skills:

• Agentic AI / Multi-Agent Systems – Experience with agentic AI frameworks or multi-agent AI systems.

• Cloud Search Expertise – Experience with cloud AI search services, including indexing, vector profiles, and hybrid search.

• Python AI Frameworks – Experience using LangChain or Semantic Kernel for AI test harnesses.

• Evaluation Pipelines – Experience using prompt flow to build model evaluation pipelines.

• Golden Dataset Creation – Knowledge of creating ground truth datasets for regression testing.

• Master’s Degree – Advanced degree in Data Science or AI-related field.

Nice to Have:

• Small Model Fine-Tuning – Experience fine-tuning small language models like Phi-3 or LLaMA.

• Cloud ML Tools – Experience managing ML workspaces, compute instances, and experiment tracking.

• Backend Integration Skills – Background in C# / .NET for AI integration with backend systems.

• Security Testing – Experience with red-teaming, adversarial, or jailbreak testing for AI models.

Benefits:

• Competitive salary with performance-based bonuses.
• Opportunities for professional development and advancement.
• Dynamic and collaborative work environment.