Senior Data Engineer- 12 Month Daily Rate Contract

About the Company

My client is a leading information and technology-enabled health services business. They are dedicated to modernizing the health care system and improving the lives of people and communities. Serving virtually every dimension of the health system, they work with a diverse set of clients across 150 countries – from those who diagnose and treat patients to those who pay for care, deliver health services, and those who supply the cures.

About the Position

Key Responsibilities

• Develop, and implement scalable data pipelines using Azure Databricks and Azure Data Lake.​​​​​​
• Create and maintain ETL workflows to ensure data quality, integrity, and availability.
• Optimize graph ETL pipelines/queries for high performance and cost-efficiency.
• Clear communicator, have a positive attitude at work & collaborate well with mixed resource group based in IRL & US - its an important requirement for this role to ensure working closely with the lead and support in technical work wherever required & have a coaching mentality for the junior team members
• Demonstrated ability to manage and analyze data at scale within the Azure ecosystem.
• Coach junior data engineers on technologies like Azure databricks, graph data modelling so the candidate should be proficient in such topics with greater amount of hands-on experience.
• Implement CI/CD pipelines using Azure DevOps to streamline deployment processes.
• Conduct code reviews, provide mentorship, and facilitate knowledge sharing sessions within the team.
• Integrate data from various sources into a centralized data lake, enabling unified access and analysis.
• Perform data cleansing, transformation, and migration tasks as required.
• Monitor and troubleshoot data pipelines and resolve any issues promptly.

Experience/Requirements
Required Qualifications:

• Bachelor’s or Master’s degree in Computer Science, Information Technology, or a related field.
• 7+ years of experience in data engineering or related roles.
• 5+ years of experience in Azure Databricks, Azure Data Lake, & engineered the delivery of a graph product in previous roles & proven track record of building data engineering solutions
• Proficient in programming languages such as Python, SQL, and Scala.
• Experience with big data technologies including Apache Spark, Hadoop, and Hive.
• Hands-on experience with cloud platforms, particularly Microsoft Azure.
• Proficiency in ETL processes, data warehousing, and data modeling.
• Knowledge of AWS Neptune or have a previous experience is a value add
• Knowledge of tools and technologies such as Azure Data Factory, AzureSynapse Analytics, and Apache Kafka.
• Solid understanding of CI/CD pipelines and hands-on experience with DevOpstools such as Git for continuous integration and deployment.
• Excellent problem-solving skills and the ability to troubleshoot complex data issues.
• Strong coaching abilities, Excellent communication and collaboration skills, with the ability to work effectively with cross-functional teams.

Preferred Qualifications:

• Microsoft Certified: Azure Data Engineer Associate.
• Databricks Certified Associate Developer for Apache Spark.
• Experience with data governance and data security practices.
• Knowledge of Docker and Kubernetes for containerization and orchestration.

Please contact Charlie Bigger on 01 5927867 or email or simply click the apply button. To view all live jobs with Brightwater and market insights, please visit our website: www.brightwater.ie