[Hiring] Principal Data Engineer @Creative Testing Solutions

Role Description

The Principal Data Engineer holds a pivotal role in enabling data-driven decision-making across CTS. This position involves designing, building, and maintaining the processes that collect, store, and process data for analysis and business use. This individual is responsible for creating data pipelines that transform raw data into usable formats for analysts, data scientists, visualization engineers, and other stakeholders. This includes developing and maintaining databases, data warehouses, and other data infrastructure. The Principal Data Engineer collaborates closely with stakeholders to deliver and support business operations, with a focus on democratizing data and building robust analytics solutions. This role will develop data solutions, enhance internal analytics platforms, and streamline processes to drive efficiency.

• Conceptualizes and implements foundational data models, Data Lake, ETL pipelines and curated layer servicing multiple reporting platforms.
• Writes and optimizes SQL queries for high performance and cost-effectiveness in large-scale data environments.
• Develops and manages data engineering architecture, and designs and implements Lakehouse and analytics solutions.
• Develops and coordinates cloud architecture across diverse areas including application development, identity and access management, network and data management, and security.
• Implements data management best practices including cleansing, deduplication, validation, and governance to ensure data integrity and quality.
• Designs, develops, and maintains robust scalable ETL/ELT data pipelines across multiple data sources and destinations.
• Collaborates with data analysts, data scientists, and software engineers to understand data needs and translates them into technical solutions.
• Optimizes data workflows for performance, scalability, and reliability.
• Builds and maintains data models and warehouse schemas in support of analytics and business intelligence initiatives.
• Optimizes query performance and database operations.
• Ensures data quality, integrity, and compliance across all systems.
• Participates in architectural discussions and helps drive best practices in data engineering.
• Collaborates with fellow data & analytics colleagues to foster a culture of teamwork and innovation.
• Mentors junior data engineers and contributes to the development of team standards and practices.
• Monitors data pipelines and resolves production issues in a timely manner.
• Partners closely with DevOps and platform teams to manage infrastructure and deployment of data tools.
• Performs all assigned duties in compliance with internal SOPs and external regulations. Brings compliance issues to the attention of management.
• Assures quality customer service to all customers.
• Maintains good attendance and punctuality per the absence policy.
• Performs all other duties, at the discretion of management, as assigned.

Qualifications

• Education: Bachelor’s degree in Computer Engineering or related field of study, or equivalent technical experience.
• Experience: A minimum of ten (10) years of data engineering related experience; to include:
• Experience designing and implementing Data Warehouse Architectures, preferably in a lead or architect capacity.
• Experience with ETL/ELT pipelines, to-and-from various datastores (incl. operating systems, network share protocols, web APIs, and databases), as well as for data processing automation.
• Advanced experience with database concepts; solid understanding of query languages (e.g., SQL), data transformation techniques, data modeling, warehousing concepts, and best practices.

Requirements

• Knowledge of Azure cloud platform, Fabric data platform, ADF and DevOps is highly preferred.
• Professional database certification.

Skills & Abilities

• Strong business acumen, critical thinking, and ability to work cross functionally through collaboration with engineering and business partners.
• Ability to read and understand various data structures.
• Ability to work independently and as part of a team.
• Strong analytical, technical, and troubleshooting skills.
• Familiarity with data governance, privacy, and compliance standards.
• Must possess the skills and abilities to successfully perform all assigned duties and responsibilities.
• Must be able to maintain confidentiality.
• Ability to work in a team environment and participate as an active team member.
• Effective oral and written communication skills.
• Able to organize, prioritize, and execute a variable workload and multiple priorities.

Physical & Travel Requirements

• Office work environment.
• Travel: < 10%
• 5+ hours per shift: Sitting, finger dexterity, reaching waist to shoulder.
• Up to 3 hours per shift: Standing, walking, bending, squatting, kneeling, handling, driving, reaching above shoulders, reaching waist to shoulder, reaching knee to waist, reaching floor to knee, grasping items, lift/carry/move items up to 40 pounds, push/pull items up to 10 pounds.