Data Engineer for Big Data Distribution

hace 3 semanas


Madrid, Madrid, España BASF A tiempo completo
About Us

BASF Digital Hub Madrid is a hub for innovation, where we develop cutting-edge digital solutions for BASF, create new and exciting customer experiences, and drive business growth by streamlining processes and strengthening BASF's position as a digital leader in the chemical industry. We believe in the power of creativity, collaboration, and diversity.

Our Team

Our team is responsible for creating and maintaining a robust and scalable Big Data platform built on Azure PaaS components EDL (Enterprise Data Lake). We work closely with stakeholders to understand their needs and design solutions that meet their expectations. Our team is accountable for ensuring the platform's scalability, reliability, and security, as well as staying up-to-date with the latest technologies and trends in the big data domain.

About the Job

We are seeking a highly motivated and detail-oriented candidate to join our team as a Data Distribution Specialist. The successful candidate will be responsible for designing, implementing, and maintaining data distribution strategies using Unity Catalog with Azure Databricks and Azure Data Factory.

Responsibilities
  • Design and implement data distribution strategies using Unity Catalog with Azure Databricks and Azure Data Factory.
  • Maintain and monitor data distribution across various platforms and systems with Azure Data Factory.
  • Develop, extend, and optimize our data distribution pipeline.
  • Implement quality control measures to ensure accuracy and consistency of data.
  • Collaborate with the data science team to understand their data needs and provide suitable solutions.
  • Troubleshoot data-related issues and optimize data distribution for performance and scalability.
  • Develop and maintain technical documentation related to data distribution processes.
  • Test new features and improvements in Unity Catalog, Azure Databricks, Azure Data Factory, and suggest their implementation if beneficial.
Qualifications
  • Bachelor's degree in computer science, information technology, engineering, business, or related fields.
  • Minimum 3-4 years related working experience as a DataOps Engineer using Azure, Python, Azure Data Factory, and Databricks.
  • Experience in Agile way of working with a DevOps mindset.
  • Strong knowledge in automation and scripting with tools like Azure DevOps pipelines, Azure Data Factory, etc.
  • In-depth knowledge of Big Data concepts and Databricks.
  • Strong software engineering and development skills to maintain and extend our Data Distribution Pipeline.
  • Knowledge on Unity Catalog for Data Distribution through Databricks.
Benefits
  • A secure work environment that prioritizes your health, safety, and wellbeing.
  • Flexible work schedule and home-office options.
  • Learning and development opportunities.
  • 23 holiday days per year.
  • 5 additional days (readjustment).
  • 2 cultural days.
  • A collaborative, trustful, and innovative work environment.
  • Being part of an international team and working on global projects.
  • Relocation assistance to Madrid provided.

BASF is a company that values innovation, sustainability, and connected thinking. We are looking for talented individuals who share our values and are passionate about developing the future with us.



  • Madrid, Madrid, España Basf A tiempo completo

    About UsBASF Digital Hub Madrid is a leading digital solutions provider for the chemical industry. We believe in innovation, creativity, and collaboration to drive business growth and efficiency.Job DescriptionWe are seeking a highly motivated Cloud Data Engineer to join our team at BASF Digital Hub Madrid. The successful candidate will be responsible for...

  • Big Data Engineer

    hace 1 semana


    Madrid, Madrid, España BASF SE A tiempo completo

    Job DescriptionAs a Data Distribution Specialist, you will be responsible for designing, implementing and maintaining data distribution strategies using Unity Catalog with Azure Databricks and Azure Data Factory. You will work closely with the data science team to understand their data needs and provide suitable solutions.The ideal candidate has a strong...


  • Madrid, Madrid, España Basf A tiempo completo

    About UsBASF Digital Hub Madrid is a leading digital solutions provider, driving innovation and business growth in the chemical industry.Job SummaryWe are seeking a highly motivated Cloud Data Engineer to join our ambitious team. As a key member of our Big Data Management unit, you will be responsible for designing, implementing, and maintaining data...


  • Madrid, Madrid, España Basf A tiempo completo

    About Basf Digital Hub MadridAt Basf Digital Hub Madrid, we aim to develop innovative digital solutions that drive business growth and efficiencies. Our team believes in creativity, trial and error, and collaboration to achieve success. We are seeking a highly motivated Big Data Engineer to join our team and contribute to the development of scalable cloud...

  • Cloud Data Engineer

    hace 1 mes


    Madrid, Madrid, España Basf Se A tiempo completo

    Job SummaryWe are seeking a highly motivated and detail-oriented Cloud Data Engineer to join our team at BASF Digital Hub Madrid. As a Cloud Data Engineer, you will be responsible for designing, implementing, and maintaining data distribution strategies using Unity Catalog with Azure Databricks and Azure Data Factory.Key ResponsibilitiesDesign and implement...

  • Data Engineer

    hace 4 semanas


    Madrid, Madrid, España Basf Digital Solutions S.L. A tiempo completo

    About UsBASF Digital Solutions S.L. is a leading digital solutions provider that develops innovative digital solutions for BASF. Our team is passionate about creating new exciting customer experiences and business growth, and driving efficiencies in processes to strengthen BASF's position as the digital leader in the chemical industry. We believe in the...


  • Madrid, Madrid, España Inetum Spain A tiempo completo

    Inetum Spain busca a un Data Engineer con experiencia en Big Data y tecnologías de procesamiento en memoria para desarrollar y mantener plataformas de datos y productos de datos.El candidato ideal será responsable de diseñar e implementar pipelines de datos robustos utilizando herramientas como DataStage, asegurando la calidad y fiabilidad de los datos....


  • Madrid, Madrid, España Telefónica Iot & Big Data Tech S.A. A tiempo completo

    Descripción del EmpleoEstamos buscando un estudiante universitario para unirse a nuestro equipo de Ciencia de Datos y Big Data en Telefónica IoT & Big Data Tech S.A. en Madrid.Objetivos del EmpleadoEl objetivo principal de este puesto es formarse en el tratamiento, modelización y explotación de datos a partir de los datos generados por la red móvil de...

  • Data Engineer

    hace 3 semanas


    Madrid, Madrid, España Basf Se A tiempo completo

    About the JobWe are seeking a highly motivated and detail-oriented Data Engineer who will be responsible for the development of new and existing software and data pipelines with Python and Databricks and maintaining/developing CI/CD pipelines and processes in an ambitious team. Key Responsibilities:Develop, analyze, improve and test quality software to meet...

  • Data Engineer

    hace 4 semanas


    Madrid, Madrid, España Inetum A tiempo completo

    About the Role:Inetum is seeking a skilled Data Engineer to join our team. As a Data Engineer, you will be responsible for developing and maintaining new big data platforms and data products. Your mission will be to build data pipelines that ingest, transform, and prepare data from various sources into our datahub platform and data products.Key...


  • Madrid, Madrid, España Data Engineer Jobs A tiempo completo

    Job OverviewWe are seeking an experienced Data Engineer to join our team in Jersey City, NJ. The successful candidate will have a strong background in designing and developing cloud data pipelines using AWS services.About the RoleThe Data Warehouse Dimensional Modeler is a mid-to-senior level position that provides data pipeline expertise and business...

  • Data Engineer

    hace 4 semanas


    Madrid, Madrid, España Inetum Spain A tiempo completo

    About the Role:As a Data Engineer at Inetum Spain, you will be part of our data squads responsible for developing and maintaining new big data platforms and data products. Your mission will be to build data pipelines that ingest, transform, and prepare data from various sources into our datahub platform and data products.Key Responsibilities:• Technical...

  • Data Engineer

    hace 4 semanas


    Madrid, Madrid, España Inetum A tiempo completo

    About the Role:We are seeking a highly skilled Data Engineer to join our team at Inetum. As a Data Engineer, you will be responsible for developing and maintaining new big data platforms and data products.Your Mission:To build data pipelines that ingest, transform, and prepare data from various sources into our datahub platform and data products.Key...

  • Data Engineer

    hace 1 mes


    Madrid, Madrid, España Basf Se A tiempo completo

    About the Job:We are seeking a highly motivated and detail-oriented candidate who will be responsible for the development of new and existing software and data pipelines with Python and Databricks and maintaining/developing CI/CD pipelines and processes in an ambitious team.Responsibilities:Develop, analyze, improve and test quality software to meet both...


  • Madrid, Madrid, España Basf Digital Solutions S.L. A tiempo completo

    We are seeking a highly motivated DataOps Engineer to join our Data Foundation - Big Data Management team at Basf Digital Solutions S.L. In this role, you will be responsible for the development of new and existing software and data pipelines with Python and Databricks, as well as maintaining and developing CI/CD pipelines and processes.About the JobThe...


  • Madrid, Madrid, España Basf A tiempo completo

    About UsBASF Digital Hub Madrid is a team of innovators who develop cutting-edge digital solutions for BASF. We create new customer experiences and drive business growth by streamlining processes and leveraging data insights. Our team is passionate about creativity, collaboration, and learning together. We strive to be a global team that values diversity and...


  • Madrid, Madrid, España Inetum A tiempo completo

    About InetumInetum is a leading company in digital services and technology.Job SummaryWe are seeking a highly skilled Big Data Engineer to join our team. This role will involve developing and maintaining big data platforms and data products using DataStage.Key ResponsibilitiesAnalyze technical requirements and data needs to develop robust data...

  • Data Engineer

    hace 4 semanas


    Madrid, Madrid, España Inetum Spain A tiempo completo

    About the Role:We are seeking a highly skilled Data Engineer to join our team at Inetum Spain. As a Data Engineer, you will be responsible for designing, developing, and maintaining large-scale data pipelines and architectures.Your Mission:To build efficient and scalable data pipelines that ingest, transform, and prepare data from various sources into our...

  • Big Data Engineer

    hace 4 semanas


    Madrid, Madrid, España Ust Global A tiempo completo

    We are seeking a highly skilled Big Data Engineer to contribute to a project with one of our global customers, an international Bank.As a Big Data Engineer, you will help drive the delivery of Accounting E2E reporting within the Finance strategic transformation program.You will work with local and group colleagues across the business and Information...


  • Madrid, Madrid, España Amazon A tiempo completo

    Job SummaryAmazon is seeking a highly skilled Software Development Engineer to join our Open Data Analytics Fundamentals team. As a key member of our team, you will design and build next-generation infrastructure technologies to ensure AWS is the best environment for running large-scale big data processing workloads.About the RoleAs a Software Development...