High Performance Kafka Engineer

hace 12 horas


Madrid, Madrid, España Datadog A tiempo completo

At Datadog, we rely on Kafka to send data in near real-time between a vast majority of our services. The Streams team operates the Kafka fleet, ensuring that companies and engineers across the world continue to trust Datadog functionality and metrics.

The team's goals are reliability, cost efficiency, and usability for customers. We support dozens of internal customers, with petabytes of storage and hundreds of millions of messages and MiB per second flowing through our infrastructure. At this scale, first-class reliability is an exciting challenge, meaningful cost optimizations can result from clever improvements, and excellent customer satisfaction requires safe automation.

About the Role:

We're looking for a Senior Software Engineer to help us scale and continue improving our Kafka fleet. As a member of the team, you'll regularly interact with users, dig deep into open source solutions to understand behavior, and build necessary tooling and enhancements to keep Kafka running smoothly and efficiently for our customers.

Responsibilities:
  • Operate a bleeding-edge Kafka deployment with exceptional reliability by deeply understanding foundational hardware and cloud-provider building blocks, incident root causes, and performance profiles through detail-oriented investigations.
  • Develop a high-performance streaming platform composed of our control and data planes to ease customer adoption and reduce operational burden.
  • Integrate open source solutions with internal systems to operate Kafka efficiently.
  • Tweak low-level Kafka client behavior to optimize high-throughput application performance.
  • Drive complex projects and be accountable for timely delivery and high-quality results.
  • Mentor junior team members and interns.
About You:

You should be someone who can operate independently, methodically experiment, rapidly iterate, and has no fear of failure. You possess good communication skills, have experience gathering requirements from stakeholders and leading complex technical initiatives. You're also willing to learn new languages and technologies, diving deep into runtimes and ecosystems.

In terms of qualifications, you have experience delivering software solutions and libraries in Java or Golang, and experience building/operating large-scale stateful systems. Familiarity with distributed systems concepts such as consensus, quorums, and replication is a plus.

Bonus Points:
  • Direct experience operating or using large-scale streaming systems like Kafka.
  • Familiarity with C++ or Rust.
What We Offer:

We offer a competitive salary of $175,000 - $225,000 per year, depending on experience, along with new hire stock equity (RSUs) and employee stock purchase plan (ESPP). You'll also enjoy continuous professional development, product training, and career pathing, as well as intradepartmental mentor and buddy programs for in-house networking. Our inclusive company culture allows you to join our Community Guilds (Datadog employee resource groups), and access Inclusion Talks, our internal panel discussions. Finally, you'll receive free, global mental health benefits for employees and dependents age 6+.



  • Madrid, Madrid, España Factor Eleven A tiempo completo

    Senior DevOps EngineerWe're seeking a seasoned Senior DevOps Engineer to join our team at Factor Eleven. As a key member of our engineering organization, you'll play a crucial role in empowering our users with a robust feature set, high availability, and exceptional performance.About the RoleIn this position, you'll work closely with Product Owners and their...


  • Madrid, Madrid, España Datadog A tiempo completo

    About DatadogAt Datadog, we are passionate about building scalable and reliable systems that power our applications.About the RoleWe are seeking an experienced Senior Software Engineer to join our Distributed Systems team in Madrid, Spain. This role offers a competitive salary of $150,000 per year, plus benefits and growth opportunities.Job DescriptionAs a...


  • Madrid, Madrid, España F. Hoffmann-La Roche Ag A tiempo completo

    Job SummaryWe are seeking a highly skilled High Performance Storage Engineer to join our team. As a key member of our product team, you will be responsible for designing, implementing, and maintaining high-performance storage platforms for our scientific computing applications.Key ResponsibilitiesContribute to the development of high-performance storage...


  • Madrid, Madrid, España Federal Management System, Inc. A tiempo completo

    Job Title: Software Engineer for High-Performance ComputingJob Summary:We are seeking a highly skilled Software Engineer for High-Performance Computing to join our team at the Barcelona Supercomputing Center (BSC). As a key member of our Autosubmit development team, you will be responsible for improving the efficiency and productivity of numerical weather...


  • Madrid, Madrid, España Datadog A tiempo completo

    Senior Software Engineer - Kafka Fleet OperationsKafka plays a crucial role at Datadog, enabling the real-time exchange of data between various services. The Streams team is responsible for ensuring the reliability, efficiency, and usability of our Kafka infrastructure, which supports numerous internal customers and handles vast amounts of data. Our goal is...


  • Madrid, Madrid, España Datadog A tiempo completo

    Datadog is a leading technology company that provides real-time monitoring and analytics for modern infrastructure, applications, and cloud services.Our mission is to deliver world-class products and services that empower our customers to optimize their performance, security, and reliability.We're seeking a Senior Software Engineer to join our Streams team,...


  • Madrid, Madrid, España Grow-Talent A tiempo completo

    Grow-Talent.com is partnering with a renowned liquidity and technology solutions company to onboard a Streaming Software Engineer. This role is based remotely, with the GMT 4 timezone (-1 hour) time difference. OverviewThis award-winning company is seeking an experienced Streaming Software Engineer to join their team. As a key member, you will be responsible...


  • Madrid, Madrid, España F. Hoffmann-La Roche Ag A tiempo completo

    Job SummaryWe are seeking a highly skilled DevOps Engineer to join our team in Madrid, Spain. As a key member of our product team, you will be responsible for delivering high-performance scientific computing platforms in partnership with our scientists and service providers.Key Responsibilities:Contribute to the development of cutting-edge features and...


  • Madrid, Madrid, España F. Hoffmann-La Roche Gruppe A tiempo completo

    Job DescriptionThe position of HPC DevOps Engineer at Roche Informatics is a unique opportunity to join our High Performance Computing section and contribute to the development of computing and data IT infrastructure solutions. As a key member of our team, you will be responsible for implementing, operating, and evolving computing and data IT infrastructure...


  • Madrid, Madrid, España Roche A tiempo completo

    We are seeking a highly skilled High Performance Computing Engineer to join our team at Roche in the role of HPC Solution Architect / Engineer. The ideal candidate will have a solid background in computer science and hands-on experience with HPC scheduler knowledge such as LSF, Slurm, or Gridengine.The HPC Solution Architect / Engineer will be responsible...


  • Madrid, Madrid, España Dune A tiempo completo

    About DuneDune is a collaborative multi-chain analytics platform used by thousands of developers, analysts, and investors to understand the on-chain world and the frontiers of finance. Our team of ~50 employees works together across Europe and eastern US timezones, driven by our mission to make crypto data accessible.We're backed by some of the world's best...


  • Madrid, Madrid, España Backbase Inc. A tiempo completo

    Role DescriptionAs a Performance Engineer at Backbase, you will play a crucial role in ensuring the optimal performance of our software products. You will work closely with development teams to design and implement best practices for performance engineering processes.Key ResponsibilitiesDefine Performance Metrics: Identify and define SMART KPIs for...


  • Madrid, Madrid, España ebury A tiempo completo

    About UsEbury is a fast-paced FinTech firm that offers a range of innovative products, including FX risk management, trade finance, currency accounts, international payments, and API integration.We are seeking an experienced High-Performance Infrastructure Engineer to join our team in Madrid. As a key member of our platform engineering team, you will work...


  • Madrid, Madrid, España Centric Software A tiempo completo

    Centric Market Intelligence is a leading AI-driven competitive assortment benchmarking and market trend insights solution for fashion, beauty, and home goods brands and retailers. Our team is seeking a highly skilled Database Back End Engineer to join our Backend Team and develop high-performance systems that support our applications. Key...


  • Madrid, Madrid, España F. Hoffmann-La Roche Gruppe A tiempo completo

    Role OverviewAt F. Hoffmann-La Roche Gruppe, we're committed to fostering a culture of diversity, equity, and inclusion. As a DevOps Engineer for Scientific Computing Platforms, you'll play a crucial role in delivering cutting-edge scientific computing platforms that meet the needs of our scientists and service providers.Key ResponsibilitiesCollaborate with...


  • Madrid, Madrid, España Dune A tiempo completo

    About the Role:Dune is pushing the boundaries of real-time data processing, and we're looking for a talented Rust Engineer to help us build and optimize the next generation of APIs. As an engineer on DevX at Dune, you'll play a crucial role in developing high-performance, real-time systems that power crypto applications. You'll work on some of the most...


  • Madrid, Madrid, España Roche A tiempo completo

    Unlock the Power of High Performance ComputingAt Roche, we're pushing the boundaries of scientific discovery and innovation. As an HPC Solution Architect / Engineer, you'll play a critical role in driving the evolution and delivery of cutting-edge projects in the Science & Research space.Our shared HPC environment provides compute on demand resources for...


  • Madrid, Madrid, España Sdg Group A tiempo completo

    ¿Quieres formar parte de un equipo de líderes en la industria del dato?En Sdg Group, estamos buscando a un Ingeniero de Tiempo Real en Kafka para unirse a nuestro equipo de expertos en Big Data y Analytics. Como Ingeniero de Tiempo Real en Kafka, serás responsable de diseñar y desarrollar soluciones de datos en tiempo real para ayudarnos a tomar...


  • Madrid, Madrid, España F. Hoffmann-La Roche Gruppe A tiempo completo

    About the RoleThe F. Hoffmann-La Roche Gruppe is seeking a talented DevOps Engineer for Scientific Computing Platforms to join their team in Madrid, Spain. This is a hybrid role that involves working with product teams to deliver cutting-edge scientific computing platforms.In this position, you will have the opportunity to work with parallel file systems,...


  • Madrid, Madrid, España Adyen A tiempo completo

    Company Overview:Adyen is a leading financial technology platform providing payments, data, and financial products to customers like Facebook, Uber, H&M, and Microsoft.We offer an environment with opportunities for our people to succeed, backed by the culture and support to ensure they are enabled to truly own their careers.About the Job:As a Software...