Overview
We are seeking a Lead Data Platform Engineer to design, build, and optimize scalable data platforms and pipelines that support enterprise analytics and data-driven solutions. This role combines hands-on engineering with technical leadership, driving best practices in data architecture, platform performance, and governance while mentoring engineering teams.
Key Responsibilities
-
Design, develop, and maintain scalable data pipelines and ETL/ELT workflows.
-
Build and optimize data platforms using Hadoop, Databricks, and cloud-based technologies.
-
Integrate structured and semi-structured data into reliable, high-quality data solutions.
-
Partner with cross-functional teams to translate business and analytics requirements into scalable engineering solutions.
-
Lead technical design discussions and promote best practices in data modeling, performance optimization, and governance.
-
Mentor data engineers and contribute to engineering standards, architecture, and platform scalability.
-
Support innovation through proof-of-concepts, automation, and continuous platform improvements.
-
8+ years of experience in data engineering, including 2+ years in a technical leadership role.
-
Strong Python skills (Pandas, NumPy, PySpark) and experience with Impala.
-
Hands-on experience with Hadoop, Databricks, and large-scale data processing.
-
Advanced SQL and experience with relational and distributed databases.
-
Experience with cloud platforms such as Azure or AWS, including Databricks or Snowflake.
-
Strong knowledge of ETL/ELT tools such as Apache Airflow, Apache NiFi, or Azure Data Factory.
-
Experience with CI/CD, DevOps practices, and enterprise data platforms.
-
Understanding of data modeling, governance, and performance optimization.
Nice to Have
-
Experience supporting AI/GenAI solutions through scalable data pipelines.
-
Knowledge of machine learning workflows, feature engineering, and model serving.
-
Experience processing unstructured data and implementing data governance, privacy, and security best practices.
-
Strong analytical and problem-solving skills with the ability to communicate effectively across technical and business teams.
