Sr. ML Data Engineer (ETL) | TELECOMMUTE

100% TELECOMMUTE

Description : We seek a talented Senior Data Engineer with ML feature engineering expertise to join our Consumer ML team This role involves designing and implementing advanced feature engineering and ETL pipelines to enable robust machine learning applications The ideal candidate has hands-on experience with Databricks a deep understanding of the medallion architecture and a proven track record of supporting the end-to-end ML lifecycle Experience with MLflow and an aptitude for creating data driven insights are highly desirable.

Responsibilities :

• Feature Engineering Data Integration Develop and maintain feature engineering pipelines using Databricks to support ML models effectively
• Data Pipeline Development Integrate diverse data sources eg clickstreams user behaviour demographic data to create user behaviour features profiles for complex ML tasks
• Medallion Architecture Design and implement ETL, ELT pipelines aligned with the bronze silver and gold layers of the medallion architecture
• Model Support Build data pipelines to support ML model training calibration and deployment leveraging MLflow for experiment tracking and performance monitoring
• Query Optimization Low Latency Pipelines Design low latency production ready data pipelines to support real-time and batch model inference
• CICD Practices Apply CICD principles for seamless pipeline deployment
• Data Governance Ensure pipelines comply with security and regulatory standards particularly for handling PII and maintain metadata and master data across the data catalogue
• Collaboration Work closely with ml scientists ml engineers and other stakeholders to align data transformation with business objectives

Qualifications :

Mandatory Skills : Apache Spark, Databricks, Java, Python, Scala, SparkSQL

Required Skills : Machine Learning

Basic Qualification :

Additional Skills :

This is a high PRIORITY requisition. This is a PROACTIVE requisition

Background Check : No

Drug Screen : No