Azure Data Engineer Senior

THIS IS A HYBRID, OFFICE BASED POSITION WORKING AT OUR ELLIS PRESERVE OFFICE LOCATED IN NEWTOWN SQUARE, PA.

WE WILL NOT CONSIDER REMOTE CANDIDATES!

Role Overview

The Senior Azure Data Engineer designs, builds, and optimizes large scale cloud data solutions on Microsoft Azure. The role requires deep hands on expertise with modern ingestion frameworks to ingest diverse data sources, on distributed compute, real-time streaming, and dimensional modeling, combined with strong Agile delivery practices and leadership in guiding junior developers. Experience with GenAI technologies is increasingly important as the organization evolves toward intelligent data products.

Work Arrangement

* This a HYBRID office based position working in our Ellis Preserve office located in Newtown Square, PA. Core office days are Tuesday, Wednesday, and Thursday.

Responsibilities

• Architecture & Implementation: Design and implement scalable engineering solutions using Azure Data Factory, Azure Databricks, Azure Event Hubs, Kafka, HVR, Snowflake, and Azure Data Lake Storage
• Pipeline Development: Build robust ETL/ELT pipelines for batch and streaming workloads using PySpark and SQL, ensuring fault tolerance, schema evolution handling, and idempotency
• API Ingestion: Implement reliable API data ingestion patterns including authentication, pagination, rate limit handling, retries, and schema mapping
• Real time Integration: Design and operate event driven ingestion and processing using Event Hubs and Kafka, and integrate CDC solutions using HVR
• Performance Optimization: Tune Databricks jobs and clusters (cluster sizing, autoscaling, caching, partitioning, shuffle optimization, Delta Lake optimizations) to meet SLAs and cost targets
• DevOps & CI/CD: Use Azure DevOps for backlog management, Git workflows, YAML pipelines, automated testing, and repeatable deployments across environments
• Code Quality & Mentorship: Perform code rev...