Data Engineer

EDV Werke is looking for a Data Engineer

Working Model: Remote

Form of cooperation: B2B Contract

Tasks:

• Data Pipeline Development – Develop and maintain AWS-based data pipelines including Lambda processing jobs and ingestion workflows.
• Pipeline Feature Enhancements – Modify existing Lambda functions and processing logic to support new data processing features.
• External Data Integration – Refactor pipelines to ingest and process external data sources reliably.
• Data Quality Implementation – Implement automated validation checks ensuring accuracy and completeness of datasets.
• Data Model Design – Design optimized data models using Iceberg format for structured “silver” and “gold” data layers.
• Query Performance Optimization – Optimize SQL queries and schemas to meet defined performance thresholds.
• Data Governance Support – Apply validation rules and monitoring processes to prevent data quality incidents.
• Documentation & Knowledge Sharing – Document pipelines and transformations while maintaining versioned code in Git repositories.

Must-Have Skills:

• Python Development Expertise – Strong coding experience in Python or similar languages using Test-Driven Development practices.
• SQL & Data Modeling Skills – Solid SQL skills for designing efficient schemas and managing structured datasets.
• AWS Data Platform Experience – Hands-on experience with Athena, Lambda, Aurora, EventBridge, Glue, and SageMaker.
• Data Pipeline Engineering – Experience designing, building, and maintaining scalable data pipelines.
• Data Quality & Validation Knowledge – Experience implementing automated data validation and quality checks.
• Communication Skills – Ability to communicate clearly with engineering, product, and data stakeholders.
• Ownership Mindset – Ability to drive features from design through production with measurable outcomes.
• Analytical & Problem-Solving Skills – Strong analytical skills supporting troubleshooting and optimization.

Nice to Have:

• Software Architecture Knowledge – Understanding of software architecture principles and design patterns.
• Pipeline Orchestration Experience – Experience with orchestration tools such as Airflow.
• Data Quality Frameworks – Familiarity with tools like Pandera or Great Expectations.
• Advanced Data Modeling – Experience designing complex analytical data models.

Benefits:

• Competitive salary with performance-based bonuses.
• Opportunities for professional development and advancement.
• Dynamic and collaborative work environment.