Data Engineer

hace 2 semanas

Barcelona, Barcelona, España Axiom Software Solutions Limited A tiempo completo

Job Title: Data Engineer

Job Description:

Seeking a skilled Data Engineer with a robust background in PySpark and extensive experience with AWS services, including Athena and EMR. The ideal candidate will be responsible for designing, developing, and optimizing large-scale data processing systems, ensuring efficient and reliable data flow and transformation.

Key Responsibilities:


• Data Pipeline Development: Design, develop, and maintain scalable data pipelines using PySpark to process and transform large datasets.


• AWS Integration: Utilize AWS services, including Athena and EMR, to manage and optimize data workflows and storage solutions.


• Data Management: Implement data quality, data governance, and data security best practices to ensure the integrity and confidentiality of data.


• Performance Optimization: Optimize and troubleshoot data processing workflows for performance, reliability, and scalability.


• Collaboration: Work closely with data scientists, analysts, and other stakeholders to understand data requirements and deliver solutions that meet business needs.


• Documentation: Create and maintain comprehensive documentation of data pipelines, ETL processes, and data architecture.

Required Skills and Qualifications:


• Education: Bachelor's or Master's degree in Computer Science, Engineering, or a related field.


• Experience: 5+ years of experience as a Data Engineer or in a similar role, with a strong emphasis on PySpark.


• Technical Expertise:

o Proficient in PySpark for data processing and transformation.

o Extensive experience with AWS services, specifically Athena and EMR.

o Strong knowledge of SQL and database technologies.

o Experience with Apache Airflow is a plus

o Familiarity with other AWS services such as S3, Lambda, and Redshift.


• Programming: Proficiency in Python; experience with other programming languages is a plus.


• Problem-Solving: Excellent analytical and problem-solving skills with attention to detail.


• Communication: Strong verbal and written communication skills to effectively collaborate with team members and stakeholders.


• Agility: Ability to work in a fast-paced, dynamic environment and adapt to changing priorities.

Preferred Qualifications:


• Experience with data warehousing solutions and BI tools.


• Knowledge of other big data technologies such as Hadoop, Hive, and Kafka.


• Understanding of data modeling, ETL processes, and data warehousing concepts.


• Experience with DevOps practices and tools for CI/CD.

  • Data Engineer – Databricks

    hace 2 semanas

    Barcelona, Barcelona, España NTT DATA, Inc. A tiempo completo

    NTT DATA is all of us. A team of more than 190,000 professionals, as diverse as the more than 50 countries in which we operate and the different sectors in which we operate: telecommunications, financial institutions, industry, utilities, energy, public administration, and healthcare.Our mission? To offer technological, business, strategy, application...

  • Data Engineer

    hace 2 semanas

    Barcelona, Barcelona, España isolutions A tiempo completo

    Are you an experienced Data Engineer with a passion for Microsoft technologies?If so, join us in our journey of innovation and shape the future of data engineering We look forward to welcoming you to our team.As a Data Engineer, you will thrive within our collaborative team, participating in high-end projects alongside skilled professionals. Your role will...

  • Data Engineer

    hace 1 semana

    Barcelona, Barcelona, España BASE life science A tiempo completo

    The life sciences industry plays a significant role in advancing healthcare and improving the quality of life - in many cases, saving millions of lives globally. At BASE life science, this is the motivation for everything we do, aimed at pushing boundaries, seeking excellence, and making a lasting difference through consulting expertise and data-driven...

  • Data Engineer

    hace 5 días

    Barcelona, Barcelona, España BASE life science A tiempo completo

    The life sciences industry plays a significant role in advancing healthcare and improving the quality of life - in many cases, saving millions of lives globally. At BASE life science, this is the motivation for everything we do, aimed at pushing boundaries, seeking excellence, and making a lasting difference through consulting expertise and data-driven...

  • Junior Data Engineer

    hace 2 semanas

    Barcelona, Barcelona, España Cobser Consulting A tiempo completo

    Estamos contratandoUbicación:BarcelonaPuesto:Data EngineerSalario: €/año Modalidad:Presencial (no guardias)Idiomas:Español (catalán deseable)¿Qué buscamos?Un/a Data Engineer con al menos 2 años de experiencia, que combine habilidades técnicas con capacidad de comunicación para colaborar con usuarios del área de Asuntos Sociales.Requisitos...

  • Cloud Data Engineer

    hace 2 semanas

    Barcelona, Barcelona, España dataguru A tiempo completo

    Qué estamos buscando?En dataguru, estamos buscando un Cloud Data Engineer apasionado por construir y optimizar la infraestructura de datos que impulse nuestras capacidades de análisis e inteligencia artificial. Si te apasiona diseñar, construir y mantener pipelines de datos robustos, escalables y eficientes en entornos cloud, te entusiasma trabajar con...

  • Senior Data Engineer

    hace 5 días

    Barcelona, Barcelona, España Visium SA A tiempo completo

    Title: Senior Data EngineerType: Full-timeLocation: Valencia or BarcelonaAbout usAt Visium, we enable enterprise executives in defining their AI & Data strategy, execute large scale transformations and implement AI across operations, ensuring their organization becomes future-proof.With expertise in strategy, architecture, cloud engineering, analytics,...

  • Senior Data Engineer

    hace 5 días

    Barcelona, Barcelona, España Visium A tiempo completo

    Title: Senior Data EngineerType: Full-timeLocation: Valencia or BarcelonaAbout UsAt Visium, we enable enterprise executives in defining their AI & Data strategy, execute large scale transformations and implement AI across operations, ensuring their organization becomes future-proof.With expertise in strategy, architecture, cloud engineering, analytics,...

  • Senior Data Engineer

    hace 7 días

    Barcelona, Barcelona, España Visium SA A tiempo completo

    Title: Senior Data EngineerType: Full-timeLocation: Valencia or BarcelonaAbout usAt Visium, we enable enterprise executives in defining their AI & Data strategy, execute large scale transformations and implement AI across operations, ensuring their organization becomes future-proof.With expertise in strategy, architecture, cloud engineering, analytics,...

  • Data Engineer

    hace 2 semanas

    Barcelona, Barcelona, España Prenomics SL A tiempo completo

    ¿Eres una persona con experiencia en el diseño e industrialización de soluciones end to end basadas en datos? ¿Te apasiona la innovación, la inteligencia artificial y las tecnologías de datos? Si la respuesta es sí, te estamos buscando Como Data Engineer en Prenomics, trabajarás en un entorno innovador y dinámica, contribuyendo a la puesta en...