Big Data Engineer for Infrastructure Optimization

hace 1 semana


Barcelona, Barcelona, España Ndt Global Gmbh. A tiempo completo
About NDT Global GmbH

NDT Global GmbH is a leading provider of diagnostic inspection solutions, data analysis services, and integrity assessment for energy-sector infrastructure assets. Our cutting-edge technologies ensure the safety and longevity of these assets.

Job Description: Big Data Engineer

We are seeking an experienced Big Data Engineer to join our team in Barcelona, Spain. The successful candidate will be responsible for architecting, specifying, and implementing data access, ingestion, processing, and provisioning of large-scale datasets into our Big Data infrastructure.

This role requires following architectural, test-driven, and clean code principles. You will also design, set up, maintain, and upgrade our production environment. Key responsibilities include:

  • Designing and writing software code for various projects in Big Data environments.
  • Mentoring colleagues, Data Scientists, and Engineers in Big Data and Data Engineering topics.
  • Maintaining software code and infrastructure in Big Data environments.
  • Designing and preparing Big Data platforms for machine learning and data processing in production environments.

You will also be responsible for documenting your work, ensuring consistency of information, and implementing software code into production. Additionally, you will perform code reviews to ensure quality of the developed software in the production stage.

Requirements

To succeed in this role, you should have a bachelor's degree in computer science or a related field. A minimum of 6 years of experience in Software Development and Big Data framework is required. You must have at least 3 years of experience working with Big Data frameworks and advanced experience in Scala, Apache Spark, Kafka, and distributed storage systems (S3, Ceph).

Experience with NoSQL Databases (Redis) and SQL Databases (MySQL, Microsoft SQL Server) is essential. Familiarity with cloud platforms (AWS or Azure) and their big data services (e.g., EMR, Databricks) is also required. Experience with Apache Nifi and containerization and orchestration tools (e.g., Docker, Kubernetes) is a plus.

Salary and Benefits

The estimated salary for this position is €80,000 - €120,000 per year, depending on experience. We offer a competitive benefits package, including health insurance, retirement plan, and generous paid time off.

How to Apply

If you meet the requirements and are passionate about Big Data engineering, please submit your application. We look forward to hearing from you.


  • Data Engineer

    hace 1 semana


    Barcelona, Barcelona, España Ndt Global Gmbh. A tiempo completo

    About NDT Global GmbHAs the leading provider of ultra-high-tech diagnostic inspection solutions, advanced data analysis and integrity assessment services for ensuring the safety and longevity of energy-sector infrastructure assets, we at NDT Global strive to be the forerunner in ultrasonic inspection technologies.Job Description: We are looking for a highly...

  • Big Data Engineer

    hace 3 días


    Barcelona, Barcelona, España Microsoft A tiempo completo

    Job DescriptionWe are seeking a highly skilled Big Data Engineer to join our Azure Data engineering team. This is an exciting opportunity to work on building the data platform for the age of AI and powering a new class of data-first applications.Responsibilities:Contribute to the design and architecture of product features, ensuring they meet Microsoft's...


  • Barcelona, Barcelona, España Semrush A tiempo completo

    Are you a seasoned data engineer looking to take your skills to the next level? We have an exciting opportunity for a Senior Cloud Data Infrastructure Engineer to join our team at Semrush. This role will be responsible for designing, developing, and maintaining large-scale cloud-based data systems using Terraform, GitLab, BigQuery, Airflow, and dbt.Job...


  • Barcelona, Barcelona, España Ndt Global Gmbh. A tiempo completo

    Job DescriptionNdt Global Gmbh. is seeking a skilled Senior Data Infrastructure Architect to join our team in Barcelona, Spain.Company OverviewWe are the leading provider of ultra-high-tech diagnostic inspection solutions, advanced data analysis and integrity assessment services for ensuring the safety and longevity of energy-sector infrastructure assets.Job...

  • Senior Data Engineer

    hace 2 semanas


    Barcelona, Barcelona, España Nielseniq A tiempo completo

    We are looking for a Senior Data Engineer to join our team at Nielseniq, the world's leading consumer intelligence company. With unparalleled global reach and a commitment to driving diversity and inclusion, we deliver the Full View of consumer buying behavior and reveal new pathways to growth.About the RoleThis is an exciting opportunity to strengthen our...


  • Barcelona, Barcelona, España Spanish National Bioinformatics Institute (Inb) A tiempo completo

    Unlock the Power of Data in HealthcareBecome a key member of our team at the Spanish National Bioinformatics Institute (Inb) and contribute to pioneering projects in health data research.About the RoleWe are seeking an experienced Research Engineer with expertise in programming, software architecture, and API integration to join our Life Sciences Department....


  • Barcelona, Barcelona, España Barcelona Supercomputing Center (Bsc) A tiempo completo

    About the RoleWe are seeking an ambitious Big Data Developer for Performance Computing Expertise to join our team at the Barcelona Supercomputing Center (BSC-CNS).Job DescriptionAs a Junior Developer in our Data-driven Scientific Computing group, you will design and implement tools to support online and offline analysis of data generated by Big Data...


  • Barcelona, Barcelona, España The White Team Consulting A tiempo completo

    We are seeking a skilled Data Engineer to join our team at The White Team Consulting. This role is ideal for a motivated professional with experience in designing and developing scalable data solutions using SQL and AWS Redshift.About the RoleThis position involves analyzing existing architectures, identifying areas for improvement, and implementing...


  • Barcelona, Barcelona, España Partium A tiempo completo

    Job OverviewWe are seeking an experienced Senior Data Engineer (f/m/x) who is not only driven by the quality of his/her/their work, but is also eager to learn and share knowledge within our team. This role is a unique opportunity to be the driving force in establishing our company's data engineering landscape.About UsPartium is the leading AI-based...

  • Big Data Engineer

    hace 3 días


    Barcelona, Barcelona, España Zurich Australian Insurance Ltd. A tiempo completo

    Compensation Package: €60,000 - €80,000 per year, depending on experience.We are seeking a talented Data Engineer to join our team at Zurich Australian Insurance Ltd. in Barcelona, Poblenou.Job DescriptionAs a Data Engineer, you will play a crucial role in designing and developing cutting-edge solutions for data ingestion, transformation, and processing....


  • Barcelona, Barcelona, España Allianz A tiempo completo

    Job OverviewAs a seasoned data engineer, you will play a pivotal role in designing, implementing, and optimizing data models and database structures that support business-critical processes at Allianz Technology.About the RoleWe are seeking a highly skilled data model engineer to collaborate with technical and business teams to ensure data requirements align...


  • Barcelona, Barcelona, España Bynder Inc. A tiempo completo

    **Bynder Inc. Overview**Bynder goes far beyond managing digital assets, propelling teams towards innovation and excellence in the industry.With a richly integrated ecosystem, we unify and transform the creation and sharing of assets, inspiring teams, delighting customers, and elevating businesses.We enable more than 1.7M users across more than 4,000...


  • Barcelona, Barcelona, España Ntt Data Corporation A tiempo completo

    Job Description:We are seeking an experienced Cloud Infrastructure Engineer to join our team at NTT Data Corporation. This is a great opportunity for a motivated individual who wants to make a real impact in the field of digital transformation.Company Overview:At NTT Data Corporation, we understand that technology plays a vital role in shaping the future....


  • Barcelona, Barcelona, España Moody'S A tiempo completo

    Moody's CRE is a pioneer in data-driven analytics, providing a cutting-edge platform that empowers agile investment teams and asset managers to make informed decisions. Our SaaS solution offers advanced business intelligence, rent roll analysis, and market analytics, leveraging big data and AI technologies to digitize the industry.We are seeking a skilled...


  • Barcelona, Barcelona, España Barcelona Supercomputing Center (Bsc) A tiempo completo

    About Barcelona Supercomputing Center (BSC)The Barcelona Supercomputing Center (BSC) is a leading research institution in Spain, focusing on high-performance computing and data management. Our mission is to facilitate scientific progress by developing and managing information technologies.Job Description:We are seeking a skilled Cloud Infrastructure Engineer...


  • Barcelona, Barcelona, España Fonyou Telecom, S.L. A tiempo completo

    Job DescriptionFonyou Telecom, S.L. is a leading provider of innovative telecommunications solutions to operators worldwide. We are seeking an experienced Cloud Infrastructure Engineer to join our team.About the RoleWe offer a competitive salary of approximately €80,000 per year, depending on experience and qualifications.As a Cloud Infrastructure Engineer...


  • Barcelona, Barcelona, España Spanish National Bioinformatics Institute (Inb) A tiempo completo

    Job Title: Research Engineer for Data Infrastructure in Health ProjectsDescription:We are seeking a Research Engineer to join our team at the Spanish National Bioinformatics Institute (INB) and contribute to the development of cutting-edge infrastructure for ELIXIR projects and health-related research. Our mission is to enable advancements in bioinformatics...


  • Barcelona, Barcelona, España Kantox Limited A tiempo completo

    Company Overview:Kantox Limited is a fast-growing organisation with expanding data needs. Our data architecture requires scalable, reliable, and efficient solutions to support our business.Salary:$120,000 - $180,000 per annum,depending on experience and location.Job Description:We are seeking a highly skilled Data Engineer to join our Data team and...


  • Barcelona, Barcelona, España Bayer A tiempo completo

    About the RoleAt Bayer, we're driven to solve the world's most complex challenges and make a real difference. As a Senior Data Engineer in our Vegetable R&D Engineering and Automation Hub, you'll play a crucial role in developing and maintaining key projects that improve operational efficiency and support predictive breeding.We're seeking a talented...

  • Senior Big Data Engineer

    hace 4 semanas


    Barcelona, Barcelona, España Microsoft A tiempo completo

    Company Overview:Microsoft is a world-renowned technology company that fosters innovation, collaboration, and career growth. Our cloud-enabled environment offers endless possibilities for passionate innovators to envision and achieve their goals.Job Summary:We are seeking an experienced Senior Big Data Engineer to join our Azure Data engineering team,...