Big Data Engineer

hace 2 semanas

Madrid, España Solera Corporation A tiempo completo

We are seeking a highly skilled and experienced Big Data Engineer to join our international team. Working on Big Data environments and projects such as Global Data Lake, and including our focus on Sustainable Estimatics offerings.
Sustainable Estimatics is a leading suite within the company, renowned for its significant impact on the industry. With our cutting-edge and certified algorithm, we offer a unique advantage to our customers, enabling them to achieve substantial cost savings by effectively reducing waste and optimizing resource usage. By integrating sustainability principles into our Estimatics practices, we actively contribute to the industry's collective effort to minimize environmental impact. Our commitment to sustainability extends beyond individual projects, as we strive to drive industry-wide innovation. Through the continuous development of new technologies and practices, we aim to create a positive ripple effect that benefits not only the environment but also society as a whole.
As a Big Data Engineer, you will play a critical role in our organization's data processing initiatives, working with large amounts of data from diverse sources across the globe.
In this position, you will be responsible for developing and maintaining scalable and efficient data processing pipelines, leveraging Cloudera and Spark technologies among others, and implementing global APIs. Your expertise in designing and implementing data ingestion, transformation, and storage processes will ensure the availability, integrity, and quality of our data across our international locations. Your participation on different projects will assure the high level quality deliverables.

Working closely with cross-functional teams, you will collaborate to understand business requirements and translate them into technical specifications for data processing and analysis. As part of our international team, you will be at the forefront of optimizing Spark jobs, ensuring they meet strict latency and throughput requirements while processing massive datasets. Your ability to collaborate effectively with data scientists and analysts will enable timely and reliable delivery of data sets for advanced analytics and machine learning models.

If you are a passionate and skilled Big Data Engineer, with a strong background in Spark technologies, Spring Boot experience, and thrive in an international, data-intensive environment, we invite you to join our team. Together, we will unlock the potential of our global data and drive meaningful insights that contribute to our organization's success.
What You'll Do Develop and maintain scalable and efficient data processing pipelines using Cloudera and Spark technologies among others to handle large volumes of data from diverse sources. Design and implement data ingestion, transformation, and storage processes to ensure data quality, integrity, and availability across international locations. Develop and maintain scalable API solutions to provide data to both: external and internal customers. Collaborate with cross-functional teams to understand business requirements and translate them into technical specifications for data processing and analysis. Optimize data workflows and performance of Spark jobs to meet strict latency and throughput requirements in processing massive datasets. Implement data governance practices and ensure compliance with data privacy and security regulations across different regions. Conduct troubleshooting and performance tuning of the Cloud or On-premises infrastructure to identify and resolve performance bottlenecks and optimize resource utilization. Collaborate with data scientists and analysts to provide timely and reliable data sets for advanced analytics and machine learning models. Stay updated with emerging technologies and industry trends related to big data processing, Cloudera, and Spark, and propose innovative solutions to enhance data processing capabilities. Provide technical leadership, mentorship, and guidance to junior members of the team, fostering a collaborative and learning culture within the international team. Participate in agile development practices, including sprint planning, code reviews, and continuous integration and deployment, to ensure high-quality software delivery. What You'll Need Bachelor's or Master's degree in Computer Science, Data Science, or a related field. Proven at least 5 years of working experience as a Big Data Engineer, preferably working with Cloudera and Spark technologies. Strong proficiency in programming languages such as Scala, Python, or Java. Strong knowledge of API architectures preferably developed with Spring Boot. In-depth knowledge of distributed computing principles and frameworks, including Hadoop and Spark. Hands-on experience with Cloudera distribution and related tools like HDFS, Hive, Impala, and HBase. Proficiency in designing, implementing, and optimizing data pipelines using Spark Streaming, Spark SQL, or other Spark modules. Expertise in data modeling and database design principles, including schema design, partitioning, and indexing. Familiarity with data ingestion techniques and tools, such as Kafka, Flume, Sqoop, or Nifi. Strong understanding of SQL and NoSQL databases, data warehousing concepts, and ETL processes. Experience with cloud platforms like AWS or Azure, and knowledge of containerization technologies like Docker or Kubernetes is a plus. Understanding of data governance, data privacy, and security practices, particularly in an international context. Excellent problem-solving and analytical skills, with the ability to troubleshoot and optimize data processing workflows. Strong communication and collaboration skills to work effectively in an international team setting. Ability to prioritize tasks, manage multiple projects simultaneously, and deliver high-quality results within defined timelines. Continuous learning mindset to stay updated with emerging technologies, industry trends, and best practices in big data engineering. The ability to thrive in a fast-paced environment and take on new responsibilities quickly. About Us Solera is the global leader in vehicle lifecycle management software-as-a-service, data, and services. Through four lines of business – vehicle claims, vehicle repairs, vehicle solutions and fleet solutions – Solera is home to many leading brands in the vehicle lifecycle ecosystem, including Identifix, Audatex, DealerSocket, Omnitracs, LoJack, Spireon, eDriving/Mentor, Explore, CAP HPI, Autodata, and others. Solera empowers its customers to succeed in the digital age by providing them with a "one-stop shop" solution that streamlines operations, offers data-driven analytics, and enhances customer engagement, which Solera believes helps customers drive sales, promote customer retention, and improve profit margins. Solera serves over 300,000 global customers and partners in 100+ countries. For more information, visit  www.solera.com .
#J-18808-Ljbffr

  • Big Data Engineer

    hace 2 días

    Madrid, España Infoplus Technologies UK Ltd A tiempo completo

    **Big Data Engineer (Big Data, Spark, Scala)** **Experience** - Experience in creation of pipeline and data evaluation in the big data environment. - Experience in developing (code) software in governed technology systems. - Unit testing experience and the ability to resolve incidents quickly. Understand the testing and governance required in the investment...

  • Big Data Engineer

    hace 1 día

    Madrid, España Juno Projects A tiempo completo

    **Descripción**: **OUR CLIENT** Spanish technology consultancy with more than 28 years of experience in the market and more than 2,300 employees working on national and international projects. Track record: Extensive technological background carrying out complete digital transformations, from customer experience to employee experience, including...

  • Data Engineer. Big Data/aws. Ingles

    hace 4 días

    Madrid, España Grupo NS A tiempo completo

    **Volver a la página principal** **Data Engineer. Big Data/AWS. Ingles** Madrid - Publicada el 6 de marzo de 2023 Descripción Desde Grupo NS precisamos incorporar un Data Enginner con al menos dos años de experiência en Big Data Activo y en Data Analytics en AWS Modalidad del trabajo: Hibido (Madrid. Zona Ciudad Lineal) **Requisitos** Nível...

  • Big Data Engineer, 100% En Remoto

    hace 1 semana

    Madrid, España Cas Training A tiempo completo

    Big data engineer en 100% En remoto. CAS Training selecciona a dos desarrolladores con al menos dos años de experiencia en entornos big data Se requiere experiencia en Conocimientos y experencia contrastada en arquitecturas big data Conocimientos consolidados en procesamiento de datos con spark batch y streaming Conocimientos kafka. Arquitectura cloud AWS...

  • Data Engineer Big Data/aws, Madrid

    hace 4 días

    Madrid, España Grupo NS A tiempo completo

    Data Engineer Big Data/AWS en Madrid. Desde Grupo NS precisamos incorporar un Data Enginner con al menos dos años de experiência en Big Data Activo y en Data Analytics en AWS Modalidad del trabajo Hibido (Madrid. Zona Ciudad Lineal) **Requisitos**: Nível avanzado de inglés Experiência en proyectos internacionales trabajando en todas las etapas del...

  • Big data engineer, 100% En remoto

    Encontrado en: Talent ES 2A C2 - hace 5 días

    Madrid, España CAS TRAINING A tiempo completo

    Big data engineer en 100% En remoto.CAS Training selecciona a dos desarrolladores con al menos dos años de experiencia en entornos big dataSe requiere experiencia en Conocimientos y experencia contrastada en arquitecturas big dataConocimientos consolidados en procesamiento de datos con spark batch y streamingConocimientos kafka.Arquitectura cloud AWS EMR ...

  • Big Data Engineer

    hace 12 horas

    Madrid, España We Bring A tiempo completo

    Desde **We Bring** estamos seleccionando para su incorporación en una empresa tecnológica que desarrolla proyectos de Inteligencia Artificial end to end un/a **Big** **Data Engineer en modalidad 100% remoto**. **¿CUÁLES SERÁN TUS FUNCIONES?** - Desarrollarás, construirás, probarás y mantendrás arquitecturas, bases de datos y sistemas de...

  • Big Data Engineer

    hace 3 días

    Madrid, España Mensoft Consultores, S.L A tiempo completo

    ¿Interesado/a en nuevos retos? En Mensoft Consultores estamos buscando un/a Big Data Engineer que quiera entrar a formar parte de un proyecto estable, en uno de nuestros clientes más importantes.REQUISITOS:- Experiencia de entre 2 y 4 años como Big Data Engineer - Experencia contrastada en arquitecturas Big Data- Experiencia en procesamiento de datos con...

  • Big Data Engineer

    hace 3 días

    Madrid, España Vodafone A tiempo completo

    **About the role**: Our team will be responsible for migrating legacy analytic & business intelligence solutions and their associated data pipelines across Vodafone onto the Google Cloud Platform, enhancing and developing new solutions. The Data Engineer delivers data pipelines and data service components. This will be at an industrial scale delivering 20+...

  • Big Data Engineer

    hace 2 días

    Madrid, España FHIOS Smart Knowledge A tiempo completo

    En FHIOS estamos buscando un/a **Big Data Engineer** para el departamento de desarrollo de un importante cliente del sector automovilístico. **¿Quién es FHIOS?** Es muy probable que antes de aplicar a la oferta de trabajo te preguntes: ¿FHIOS? ¿Qué es? **¡Conócenos!** Somos una consultoría tecnológica que lleva más de 25 años ayudando a las...

  • Global Big Data Engineer

    hace 2 días

    Madrid, España Sapindex A tiempo completo

    Somos una compañía especializada en recruitment de profesionales SAP altamente cualificados. Ofrecemos servicios de outsourcing, consultoría y soluciones IT en una amplia gama de sectores del mercado informático, tales como SAP (HANA), Business Object, BigData, Salesforce, Desarrollo de aplicaciones Móviles y Desarrollo Web. **Requisitos**: Ingeniero...

  • Big Data Engineer

    hace 1 semana

    Madrid, España Onhunters A tiempo completo

    Desde onhunters estamos buscando BIG DATA ENGINEER para trabajar en importante compañía internacional, en Madrid. Las funciones a realizar serán: Optimización de modelos de datos en stack de Cloudera: Impala, Spark, ETL en Hive, HDFS Migración de datos y analítica avanzada a Cloudera. Integración de flujos y orquestación de procesos. Conocimientos en...

  • Big Data Engineer

    hace 1 semana

    Madrid, España onhunters A tiempo completo

    Descripción Desde** onhunters** estamos buscando **BIG DATA ENGINEER **para trabajar en importante compañía internacional, en **Madrid.** Las funciones a realizar serán: - Optimización de modelos de datos en stack de Cloudera: Impala, Spark, ETL en Hive, HDFS - Migración de datos y analítica avanzada a Cloudera. - Integración de flujos y...

  • Data Engineer

    hace 2 días

    Tres Cantos, Madrid provincia, España beonit A tiempo completo

    Desde Beonit trabajamos para una de las empresas bancarias referentes en España que se dedica al diseño, desarrollo y gestión de soluciones y servicios IT, tanto para las entidades financieras de su grupo como de otros clientes._ **Data Engineer** - Estamos buscando un Data Engineer que forme parte del departamento de Data Analytics, trabajando de la...

  • Big Data Architect

    hace 3 días

    Madrid, España SCI – Serviclients Informatica A tiempo completo

    Madrid - Posted on 02/07/2023 **No. of vacancies**: **1** **functions** - Development of new projects - Collaborate and coordinate adequately with the team that makes up the area **Requirements**: - **2 years experience in similar position**: - Big data experience - Data Lake Experience - Knowledge of HCIS **Assessable requirements** -...

  • Data Engineer

    hace 4 días

    Tres Cantos, Madrid provincia, España beonit A tiempo completo

    Desde Beonit trabajamos para una de las empresas bancarias referentes en España que se dedica al diseño, desarrollo y gestión de soluciones y servicios IT, tanto para las entidades financieras de su grupo como de otros clientes._ **Data Engineer** - Estamos buscando un Data Engineer que forme parte del departamento de Data Analytics, trabajando de la...

  • Data Engineer

    hace 2 días

    Madrid, España SANDAV A tiempo completo

    **DESCRIPCIÓN** Se requiere un data engineer para unirse al grupo de Ingeniería de Global Big Data, responsable de una plataforma que proporciona a los analistas información y KPIs relevantes para una mejor toma de decisiones de negocio. **RESPONSABILIDADES** Participará en el desarrollo y mantenimiento de procesos de extracción, transformación y...

  • Big Data Engineer – Data Lake

    Encontrado en: Talent ES C2 - hace 3 semanas

    Madrid, España Santander A tiempo completo

    Big Data Engineer – Data Lake (Openbank)Country: Spain WHAT YOU WILL BE DOING Openbank is in the middle of a digital transformation, working in startup format and innovating in core banking product development from within. We want to convey a whole new digital experience to our customers and for that reason we need the most skilled professionals to come...

  • Data Engineer

    Encontrado en: beBee S ES - hace 2 semanas

    Madrid, España LHH A tiempo completo

    Eres Data Engineer y te apasiona la innovación y el desarrollo de software? ? Desde LHH nos encontramos colaborando con una empresa del sector healthcare dedicada al desarrollo de soluciones inteligentes para la prevención y el cuidado de la salud. Buscan incorporar a su equipo de datos a un Data Engineer para sus oficinas en ? Madrid ¿Qué necesitas para...

  • Desarrollador Big Data. Teletrabajo, Madrid

    hace 1 semana

    Madrid, España Second Window A tiempo completo

    Desarrollador Big Data. TELETRABAJO en Madrid. ¡Te ofrecemos crecer con nosotros en nuestra unidad tecnológica Data Analytics como Data Engineer ! Si tienes 2 años de experiencia como Desarrollador Big Data con tecnologías como Scala Spark así como iniciativa empatía y ganas de llegar lejos danos la oportunidad de conocerte. ¿Qué necesitamos? -...