[Hiring] Databricks Data Engineer @Horizon Industries

Role Description

We are seeking a Databricks Data Engineer for a full-time position supporting our various clients. We are committed to your growth and success in the IT software development industry. We are looking for a Databricks Data Engineer ready to assume leadership responsibilities in a supportive team environment focused on your success as a function of client success.

• Build end-to-end implementation of multiple ETL/ELT pipelines, demonstrating efficient data transformation and ingest patterns to move raw data from data producers to an enterprise data ecosystem, with a focus on performance and reliability.
• Assess and understand the ETL jobs, workflows, BI tools, and reports.
• Address technical inquiries concerning customization, integration, enterprise architecture and general feature/functionality of data products.
• Experience in crafting database/data warehouse solutions in the cloud (Preferably AWS, Azure, Alternatively GCP).
• Key must-have skill sets – Python, SQL, Databricks, AWS Data Services.
• Experience with message queuing, stream processing, and highly scalable ‘big data’ data stores.
• Experience manipulating, processing, and extracting value from large, disconnected datasets.
• Experience manipulating structured and unstructured data for analysis.
• Experience with data modeling tools and processes.
• Experience aggregating and transforming data from multiple datasets to create data products.
• Support an Agile software development lifecycle.

Qualifications

• Ability to hold a position of public trust with the US government.
• B.S. in Computer Science or equivalent.
• 4+ years' experience in data engineering and big data. At least 2 years' of professional services experience interacting directly with clients.
• Big data tools: Hadoop, Spark, Kafka, etc.
• Relational SQL and NoSQL databases and experience working with relational databases.
• AWS cloud services: EC2, S3, RDS, Glue, Step Functions, Lambda, EMR, DynamoDB, DocumentDB, Redshift, Aurora, Athena.
• Data Platforms: Databricks.
• Data streaming systems: Batch, Kafka, Storm, Spark-Streaming, etc.
• Languages: Python, R, Scala, Go.
• Ability to inspect existing data pipelines, discern their purpose and functionality, and re-implement them efficiently in Databricks.
• Extensive knowledge of data warehousing concepts and hands-on experience deploying pipelines using Databricks.
• Data modeling and database design skills and knowledge of version control.
• Excellent verbal and written communication skills.
• Experience architecting scalable and fault-tolerant data solutions across Azure, AWS, and Databricks.
• Databricks Data Engineer Professional certification a plus.
• Preference for candidates with Databricks Professional certifications.

Requirements

• Bachelor’s degree in a technical field such as software engineering, engineering, computer science, etc.

Benefits

• A comprehensive benefits package including healthcare (medical, dental, vision and disability).
• A 401k program where you are 100% vested from day one with an employer match after 90 days.
• An Educational Assistance program.
• A Student Loan Repayment Program.
• Gym Reimbursement Program.
• Paid Time off.
• Dynamics, passionate, multi-disciplinary team of creative minds to work with, and many more.