Senior Data Engineer

About the Role

Nexus Data Corp is scaling its real-time data infrastructure to handle 50TB+ daily ingestion across our cloud analytics platform. We're looking for a Senior Data Engineer who thrives at the intersection of distributed systems and business intelligence.

You'll own the design and delivery of mission-critical data pipelines, collaborate cross-functionally with data scientists and product teams, and mentor junior engineers on the team.

What You'll Do

• Architect and build scalable ETL/ELT pipelines using Apache Spark (PySpark) and Apache Kafka
• Design and maintain data warehouse schemas in Snowflake and AWS Redshift
• Implement CDC strategies using Apache Iceberg and Delta Lake for real-time data availability
• Build and monitor data quality frameworks to ensure 99.9% pipeline reliability
• Collaborate with ML engineers to build feature stores and training data pipelines
• Optimise query performance and reduce data transfer costs by 30%+
• Write clean, production-grade Python with comprehensive unit tests and documentation

What We're Looking For

• 5+ years in data engineering with a focus on distributed systems
• Expert-level PySpark and SQL — you can optimise a 10TB join without breaking a sweat
• Deep hands-on experience with Kafka (topics, partitioning, consumer groups, schema registry)
• Production experience with Snowflake, Redshift, or BigQuery
• Strong Python skills — object-oriented design, testing, packaging
• Familiarity with orchestration tools: Airflow, Prefect, or Dagster
• AWS or GCP certified (preferred but not mandatory)

Our Stack

Benefits

• Fully remote-friendly with flexible hours
• ₹30–50 LPA depending on experience and skills
• ESOP package with 4-year vesting
• Annual learning budget: ₹1,00,000 for courses and conferences
• Health insurance for you and your family