Deerfield, IL

Databricks Data Engineer

Job Description

Job Description:

7+ years of hands-on experience in Design and Implementation of Big Data technology solutions
Candidate should have excellent hands-on knowledge in Data engineering activities including skills in Scala, SQL
Ability to query tuning, performance tuning, troubleshooting, and debugging Spark and other big data solutions.
Familiarity with a full range of data engineering approaches, covering theoretical best practices and the technical applications of these methods
Knowledge about Azure technologies

Responsibility of / Expectations from the Role:

End-to-end development and deployment of data lake solution
Building Proof of concept and prototypes
Help architecture team in architecting solutions
Pair-programming with customer teams
Build data pipeline between various data lake layers e.g., RAW layer, Curation layer and Serve Layer
Work with data modeling team in designing schema evolution, data models for RAW layer, Curation layer
Work with Walgreens Dev Ops/Run ops team in deployments of solution
Building and deploying a range of data engineering pipelines into production, including using automation best practices for CI/CD
Build data interfaces with various downstream and upstream systems (e.g. EDW, IC+ and others)
Build data streaming and ingestion with various source systems both in batch mode and real time integration mode.
Work with Performance engineering team in improving overall performance of data lake solution across various architectural components
Query tuning and troubleshooting and debugging data lake solution
Coordinate with Account Executives, Customer Engineers and Solution Architects for expanding the use of IDI Data lake platform within Walgreens
Use expertise in data engineering best practices to guide Walgreens to do the same

Click here to apply

Browse other jobs