Cloud Data Distribution Engineer

hace 5 días


Madrid, Madrid, España Basf A tiempo completo

About BASF Digital Hub Madrid

BASF is a leading chemical company that has been innovating for over 150 years. Our digital hub in Madrid is at the forefront of developing innovative digital solutions for our customers.

We are seeking a highly motivated and detail-oriented Cloud Data Distribution Engineer - Big Data Expert to join our team.

The Role

We are looking for a talented engineer with experience in designing, implementing, and maintaining data distribution strategies using Unity Catalog with Azure Databricks and Azure Data Factory.

You will be responsible for ensuring the accuracy and consistency of data across various platforms and systems, as well as collaborating with the data science team to understand their data needs and provide suitable solutions.

Key Responsibilities

  1. Design, implement, and maintain data distribution strategies using Unity Catalog with Azure Databricks and Azure Data Factory.
  2. Maintain and monitor data distribution across various platforms and systems with Azure Data Factory.
  3. Develop, extend, and optimize our data distribution pipeline.
  4. Implement quality control measures to ensure accuracy and consistency of data.
  5. Collaborate with the data science team to understand their data needs and provide suitable solutions.
  6. Troubleshoot data-related issues and optimize data distribution for performance and scalability.
  7. Develop and maintain technical documentation related to data distribution processes.

Requirements

  • Bachelor's degree in computer science, information technology, engineering, business, or related fields.
  • Minimum 3-4 years related working experience as a DataOps Engineer using Azure, Python, Azure Data Factory, and Databricks.
  • Experience in Agile way of working with a DevOps mindset.
  • Strong knowledge in automation & scripting with tools like Azure DevOps pipelines, Azure Data Factory, etc.
  • In-depth knowledge with Big Data concepts and Databricks.
  • Strong software engineering and development skills to maintain and extend our Data Distribution Pipeline.
  • Knowledge on Unity Catalog for Data Distribution through Databricks.

Benefits

  • A secure work environment because your health, safety, and wellbeing is always our top priority.
  • Flexible work schedule and home-office options, so that you can balance your working life and private life.
  • Learning and development opportunities.
  • 23 holiday days per year.
  • 5 additional days (readjustment).
  • 2 cultural days.
  • A collaborative, trustful, and innovative work environment.
  • Being part of an international team and working in global projects.

Estimated Salary: €80,000 - €110,000 per annum, depending on qualifications and experience.

Relocation assistance to Madrid provided.



  • Madrid, Madrid, España Data Engineer Jobs A tiempo completo

    Job OverviewWe are seeking an experienced Data Engineer to join our team in Jersey City, NJ. The successful candidate will have a strong background in designing and developing cloud data pipelines using AWS services.About the RoleThe Data Warehouse Dimensional Modeler is a mid-to-senior level position that provides data pipeline expertise and business...


  • Madrid, Madrid, España Basf A tiempo completo

    About UsBASF Digital Hub Madrid is a leading digital solutions provider for the chemical industry. We believe in innovation, creativity, and collaboration to drive business growth and efficiency.Job DescriptionWe are seeking a highly motivated Cloud Data Engineer to join our team at BASF Digital Hub Madrid. The successful candidate will be responsible for...


  • Madrid, Madrid, España Basf A tiempo completo

    About UsBASF Digital Hub Madrid is a leading digital solutions provider, driving innovation and business growth in the chemical industry.Job SummaryWe are seeking a highly motivated Cloud Data Engineer to join our ambitious team. As a key member of our Big Data Management unit, you will be responsible for designing, implementing, and maintaining data...

  • Cloud Data Engineer

    hace 4 semanas


    Madrid, Madrid, España Affirm A tiempo completo

    Job OverviewInnovate with Affirm, a leading fintech company, as a Cloud Data Engineer. We're seeking a skilled engineer to design, build, and maintain durable and strongly consistent cloud-based storage solutions that empower our customers and drive innovation.


  • Madrid, Madrid, España Basf A tiempo completo

    About Basf Digital Hub MadridAt Basf Digital Hub Madrid, we aim to develop innovative digital solutions that drive business growth and efficiencies. Our team believes in creativity, trial and error, and collaboration to achieve success. We are seeking a highly motivated Big Data Engineer to join our team and contribute to the development of scalable cloud...

  • Cloud Data Engineer

    hace 2 días


    Madrid, Madrid, España Thoughtworks Inc. A tiempo completo

    Job ResponsibilitiesWe are seeking a Cloud Data Engineer to join our team at Thoughtworks Inc. The ideal candidate will have expertise in designing and implementing cloud-based data pipelines, as well as experience with big data tools and frameworks.About the Role:Design and develop cloud-based data pipelines using big data tools and frameworksDevelop data...


  • Madrid, Madrid, España Basf Se A tiempo completo

    About UsBASF SE is a global leader in the chemical industry, and our Data Foundation - Big Data Management unit aims to provide organizations with robust and scalable solutions for managing vast amounts of data. We utilize Azure's PaaS components to streamline deployment and handling of Big Data workloads, empowering clients to make data-driven decisions and...


  • Madrid, Madrid, España Spectro Cloud A tiempo completo

    About Spectro CloudSpectro Cloud is a leading provider of infrastructure boundaryless solutions for the enterprise. Our mission is to make infrastructure seamless and flexible, enabling businesses to run applications on Kubernetes across various platforms.Job DescriptionWe are seeking a talented software engineer to join our Engineering team at Spectro...


  • Madrid, Madrid, España BASF SE A tiempo completo

    We are looking for a talented Cloud Data Infrastructure Engineer to join our team. As a Cloud Data Infrastructure Engineer, you will be responsible for designing, building, and maintaining large-scale data infrastructure on cloud platforms such as Azure and Databricks.About the Job:This role involves developing, analyzing, improving, and testing quality...

  • Data Engineer

    hace 4 semanas


    Madrid, Madrid, España Ust Global A tiempo completo

    Job Title: Data Engineer We are seeking a highly skilled Data Engineer to join our team at UST Global. As a key member of our data engineering team, you will be responsible for designing, developing, and maintaining large-scale data pipelines using Snowflake cloud databases.Main Responsibilities:* Design and implement data pipelines using Snowflake cloud...


  • Madrid, Madrid, España Ixceed Solutions A tiempo completo

    Job Overview:">At Ixceed Solutions, we are seeking an experienced Senior Cloud Data Engineer to join our team in Madrid, Spain. This is a permanent role that requires strong proficiency in cloud data engineering and experience working with GCP.">About the Role:">We are looking for a skilled Senior Cloud Data Engineer to design, implement, and manage...

  • Data Engineer

    hace 1 semana


    Madrid, Madrid, España Redeploy A tiempo completo

    About UsRedeploy Insight is a leading provider of premium IT-consulting services, specializing in building modern cloud data and AI/ML platforms. Our team of experts empowers enterprise companies in the Nordics to stay ahead of the curve by delivering tailored impactful and scalable solutions.Job Title: Data EngineerAbout the RoleWe are seeking a highly...


  • Madrid, Madrid, España Deltacubes Pvt Ltd A tiempo completo

    About Deltacubes Pvt LtdDeltacubes Pvt Ltd is a company that seeks to drive technological excellence across its organization. We are looking for a versatile engineer with expertise in cloud computing, MLOps, and beyond.Salary InformationThe estimated salary range for this position is ₹1500000 - ₹2000000 per annum, depending on experience.Job...

  • Big Data Engineer

    hace 4 semanas


    Madrid, Madrid, España BASF SE A tiempo completo

    Job DescriptionAs a Data Distribution Specialist, you will be responsible for designing, implementing and maintaining data distribution strategies using Unity Catalog with Azure Databricks and Azure Data Factory. You will work closely with the data science team to understand their data needs and provide suitable solutions.The ideal candidate has a strong...


  • Madrid, Madrid, España Eeze A tiempo completo

    We are seeking an experienced Data Engineer to lead the development of scalable data solutions on Google Cloud Platform (GCP) services. As a key member of our team, you will be responsible for designing and implementing efficient data pipelines, ensuring seamless data flow, and delivering high-quality data insights to support business decisions.About the...


  • Madrid, Madrid, España Otto Group Solution Provider (Osp) Gmbh A tiempo completo

    Job Description: At Otto Group Solution Provider, we are seeking a skilled Data Engineer to join our team. This role is responsible for designing and implementing high-performance data platforms in the Google Cloud.Key Responsibilities:- Develop scalable cloud solutions based on Google or other cloud providers (e.g. Terraform, Google Dataflow, etc.)- Create...

  • Cloud Data Engineer

    hace 4 semanas


    Madrid, Madrid, España Amaris Consulting A tiempo completo

    Are you looking for a challenging role that will help you grow professionally and personally?We are seeking a highly skilled Cloud Data Engineer to join our team at Amaris Consulting. This is a unique opportunity to work with a global company, thrive in an agile environment, and develop your skills further.The ideal candidate will have a strong background in...

  • Big Data Engineer

    hace 3 días


    Madrid, Madrid, España Basf A tiempo completo

    About Basf Digital Hub MadridCompany OverviewWe develop innovative digital solutions for Basf, create new customer experiences and business growth, drive efficiencies in processes, and strengthen our position as the digital leader in the chemical industry.Job DescriptionThe role of a Big Data Engineer involves developing and maintaining scalable data...

  • Cloud Data Engineer

    hace 4 semanas


    Madrid, Madrid, España Devoteam Spain A tiempo completo

    Job Title: Cloud Data Engineer - Design and Deploy ELT PipelinesDescription:We are seeking a skilled Cloud Data Engineer to join our international team. The ideal candidate will have experience with Google Cloud services, particularly BigQuery, Cloud Storage, Cloud Build, Workflows, Pub/Sub, and Cloud Run. Advanced SQL knowledge is also required. The...

  • AWS Cloud Data Engineer

    hace 14 horas


    Madrid, Madrid, España Mantu Group Sa A tiempo completo

    Unlock your full potential as a AWS Cloud Data Engineer with Amaris Consulting. Our team is seeking an experienced professional to join our international group, leveraging their skills in data engineering and AWS services to deliver exceptional results.About Us:Amaris Consulting is an independent technology consulting firm providing guidance and solutions to...