Principal Data Engineer

Irving, TX 75014

Posted: 03/12/2024 Industry: Information Technology Job Number: 6375 Pay Rate: 175000 - 185000

Job Description

The Lead Data Engineer holds the primary responsibility for skillfully designing, developing, implementing, and supporting enterprise Microsoft Azure Data Warehouse. This role assumes accountability for ensuring the dependable, efficient, and secure operation and advancement of On-Premise and Cloud Data Warehouses to effectively meet crucial business needs.

 

 

Essential Duties and Responsibilities:

  • Provide support in designing and overseeing enterprise-grade data pipelines and data stores, essential for the development of advanced analytics programs, machine learning models, and statistical methods.
  • Implement automation and streamline processes to optimize the entire data and analytics platform, ensuring efficient throughput and high-performance outcomes.
  • Recognize, devise, and execute internal process enhancements, including automation of manual tasks, optimizing data delivery, and redesigning architecture or infrastructure to enhance scalability.
  • Collate large, intricate datasets that align with functional and non-functional business demands.
  • Develop processes that facilitate data transformation, manage data structures, metadata, dependencies, and workload management.
  • Collaborate with business users to understand functional and data requirements, contributing to the enhancement of data models and pipelines.
  • Apply expert-level analytical and problem-solving skills to diagnose and resolve intricate technical issues.
  • Create, maintain, and continuously enhance scalable data pipelines, while also developing new data source integrations to accommodate the growing volume and complexity of data.
  • Designing, implementing, and managing data extraction, transformation, and loading (ETL) processes.
  • Creating comprehensive technical specification documents and application interface designs.
  • Creating data processing and integration solutions for both batch and real-time scenarios, proficiently handling structured and unstructured data.

 

Job Requirements

Qualifications/Competencies:

Key Technical Knowledge, Skills, and Abilities:

  • Exhibit a thorough understanding of Data Lake architectures, including raw, enriched, and curated layer concepts, and ETL/ELT operations.
  • Exhibit a solid understanding of database design, data warehousing concepts, big data platforms, and ETL operations.
  • Experience working with data integration techniques & self-service data preparation.
  • Experience in requirements analysis, design, and prototyping.
  • Experience deploying modern data solutions leveraging components like Azure functions, Azure Data Factory, Data Flows, Azure Data Lake, Azure SQL, Azure Synapse, Streaming Analytics or equivalent on another cloud provider such as AWS or Google Cloud.
  • Experience with DevOps tools like Azure DevOps, Jenkins, Maven etc.
  • Experience in building/operating/maintaining fault tolerant and scalable data processing integrations.
  • Demonstrated experience of turning business use cases and requirements into technical solutions.
  • Strong level of understanding on Azure Data Factory, SQL/Synapse, ADLS, and Azure DevOps.
  • Ability to conduct data profiling, cataloging, and mappings for technical design and construction of data flows.]

 

Education and Experience:

  • Bachelor’s degree with a minimum of 10 years of related experience.
  • 10+ years of hands-on Data Warehouse architecture and development experience
  • Demonstrated expertise in Microsoft SQL Server and/or Azure development.

Additional Information

At Top Prospect Group, we believe in building lasting relationships with candidates. We support your job search journey, from resume building to interview preparation. Our goal is to help you meet your professional goals and maximize your growth. Our team of experienced recruiters takes a personalized approach to help you identify opportunities that align with your skills, interests, and career aspirations. We work closely with you to understand your unique needs and preferences, whether you're looking for a temporary contract or a permanent placement.

Meet Your Recruiter

Sean Coreth
Senior Technical Recruiter

Apply Online

Send an email reminder to:

Share This Job:

Related Jobs:

Login to save this search and get notified of similar positions.

About Irving, TX

Discover exciting job opportunities in the vibrant city of Irving, Texas! Known for its diverse community and thriving economy, Irving offers endless possibilities for career growth and development. With its close proximity to Dallas Fort Worth International Airport, major corporations, and charming neighborhoods like Las Colinas, Irving is the perfect place to advance your career. Explore job listings in this bustling city and be part of the dynamic energy that surrounds attractions like the Toyota Music Factory, the Irving Arts Center, and the Mustangs of Las Colinas sculpture. Embrace the blend of Texan hospitality and cosmopolitan flair that makes Irving a truly exceptional place to work and live. Take the next step in your career journey and find your perfect match in Irving, Texas!