Job Overview
As a Data Engineering Manager, you will be responsible for leading a team of data engineers to design, develop, and maintain our data infrastructure. You will work closely with cross-functional teams, including data science, analytics, and product teams, to ensure seamless data integration and accessibility.
Key Responsibilities
- Lead, mentor, and grow a team of data engineers, fostering a culture of collaboration and innovation.
- Collaborate with data science, analytics, and software engineering teams to optimize data workflows.
- Drive the adoption of best practices in data engineering, including performance optimization, automation, and documentation.
- Stay up to date with emerging technologies and trends in data engineering and big data.
- Architect and maintain microservices-based data platforms to support high-availability and real-time data processing.
- Design and optimize NoSQL databases to handle large-scale structured and unstructured data.
- Manage stakeholder expectations and translate business requirements into technical solutions.
- Monitor and improve data processing performance, reliability, and cost efficiency.
Required Qualifications
- 8+ years of experience in data engineering, with at least 3+ years in a leadership or management role.
- Excellent problem-solving and team leadership skills.
- Ability to communicate technical concepts effectively to non-technical stakeholders.
- Strong proficiency in SQL, Python.
- Hands-on experience with big data technologies such as Spark, Kafka, or similar.
- Expertise in cloud-based data solutions (AWS) and modern data warehousing (Databricks, etc.).
- Proficiency in NoSQL databases (e.g., MongoDB, DynamoDB).
- Experience designing and implementing microservices architecture for scalable and distributed data platforms.
- Strong understanding of data governance, security, and compliance standards.
Preferred Qualifications
- Familiarity with DevOps practices and CI/CD pipelines for data engineering.
- Experience with ETL/ELT pipeline development and orchestration tools (Airflow, dbt, Dagster, etc.).