Fairfax, VA

R-00074421 Description Job Description: Leidos seeks a data engineer to provide programming support for loading large volume data collections into a variety of data holding areas both on premises and in the cloud. The successful candidate will join a highly collaborative, cross disciplinary team of data scientists, data engineers, and applications developers to deliver mission critical analytics support to our customer. Primary Responsibilities: * Must have a Bachelor's degree in a relevant field and minimum of 12 years experience OR a Master's degree and at least 10 years of experience
* Must have a TS/SCI with Poly to be considered
* Analyze new data collections and determine optimal data ingest processes
* Create and maintain NiFi Schemas for new and existing data feeds
* Develop tools or modify existing ones to preprocess, modify, aggregate, load, index, and archive data collections into clusters and AWS in near real time
* Implement all proper access controls
* Generate metrics to track data ingest optimization and ensure data integrity and provenance are maintained
* Document data flows Basic Qualifications: * Experience in Computer Science, Computer Engineering, Systems engineering, or closely related discipline
* Extensive experience in Extract-Transform-Load processes
* Demonstrated extensive experience with cloud services (AWS strongly preferred)
* Demonstrated experience with cloud based database services such as Databricks and EMR
* Understanding of SQL database structures and mapping data structures between different SQL databases
* Knowledge of API development techniques Preferred Qualifications: * Demonstrated experience with AWS long-term storage options
* Experience with NiFi strongly preferred, but would train if candidate has experience in Java or using Pentaho
* Experience working with a Hadoop cluster
* Familiarity with open source data engineering tools and processes
* Experience designing for intelligence applications Pay Range: