Data Engineer

hace 2 semanas

Barcelona, Barcelona, España Axiom Software Solutions Limited A tiempo completo

Job Title: Data Engineer

Job Description:

Seeking a skilled Data Engineer with a robust background in PySpark and extensive experience with AWS services, including Athena and EMR. The ideal candidate will be responsible for designing, developing, and optimizing large-scale data processing systems, ensuring efficient and reliable data flow and transformation.

Key Responsibilities:


• Data Pipeline Development: Design, develop, and maintain scalable data pipelines using PySpark to process and transform large datasets.


• AWS Integration: Utilize AWS services, including Athena and EMR, to manage and optimize data workflows and storage solutions.


• Data Management: Implement data quality, data governance, and data security best practices to ensure the integrity and confidentiality of data.


• Performance Optimization: Optimize and troubleshoot data processing workflows for performance, reliability, and scalability.


• Collaboration: Work closely with data scientists, analysts, and other stakeholders to understand data requirements and deliver solutions that meet business needs.


• Documentation: Create and maintain comprehensive documentation of data pipelines, ETL processes, and data architecture.

Required Skills and Qualifications:


• Education: Bachelor's or Master's degree in Computer Science, Engineering, or a related field.


• Experience: 5+ years of experience as a Data Engineer or in a similar role, with a strong emphasis on PySpark.


• Technical Expertise:

o Proficient in PySpark for data processing and transformation.

o Extensive experience with AWS services, specifically Athena and EMR.

o Strong knowledge of SQL and database technologies.

o Experience with Apache Airflow is a plus

o Familiarity with other AWS services such as S3, Lambda, and Redshift.


• Programming: Proficiency in Python; experience with other programming languages is a plus.


• Problem-Solving: Excellent analytical and problem-solving skills with attention to detail.


• Communication: Strong verbal and written communication skills to effectively collaborate with team members and stakeholders.


• Agility: Ability to work in a fast-paced, dynamic environment and adapt to changing priorities.

Preferred Qualifications:


• Experience with data warehousing solutions and BI tools.


• Knowledge of other big data technologies such as Hadoop, Hive, and Kafka.


• Understanding of data modeling, ETL processes, and data warehousing concepts.


• Experience with DevOps practices and tools for CI/CD.

  • Data Engineer

    hace 5 días

    Barcelona, Barcelona, España Switch Tech Talent A tiempo completo

    Role:Data EngineerSalary:Up to €75,000 + equityType:PermanentLocation:Remote / Hybrid (Barcelona)Key Skills:SQL, Python, AWS, Data Pipelines, Redshift/Snowflake, Airflow, dbtWe're on the hunt for a talented Data Engineer to join our growing Data Team. In this role, you'll drive the scaling and optimisation of data platforms and integrations, ensuring teams...

  • Data Engineer

    hace 7 días

    Barcelona, Barcelona, España Xcede A tiempo completo

    Data Engineer – BarcelonaAbout the companyJoin a fast-growing global tech unicorn transforming how teams connect and work in real life. With a collaborative, in-person-first culture, you'll be part of a forward-thinking organisation that values innovation, autonomy, and impact.About the roleWe're looking for aData Engineerto scale our Data Team and enhance...

  • Data Engineer

    hace 5 días

    Barcelona, Barcelona, España psd group A tiempo completo

    Data EngineerSummary Location:Barcelona (Hybrid)Candidates must be willing to work 3 days per week in the office in Barcelona, 2 days remoteDay Rate:NegotiableDuration:6 MonthsAvailability:ASAPAbout the ClientMy client is the air transport industry's IT provider, delivering solutions for airlines, airports, aircraft and governments. Their technology powers...

  • Data Engineer – Databricks

    hace 2 semanas

    Barcelona, Barcelona, España NTT DATA, Inc. A tiempo completo

    NTT DATA is all of us. A team of more than 190,000 professionals, as diverse as the more than 50 countries in which we operate and the different sectors in which we operate: telecommunications, financial institutions, industry, utilities, energy, public administration, and healthcare.Our mission? To offer technological, business, strategy, application...

  • Data Engineer

    hace 5 días

    Barcelona, Barcelona, España Insight International (UK) Ltd A tiempo completo

    Role: Data EngineerLocation: Barcelona, SpainJob Type: PermanentMission:A Data Engineer is responsible for designing, building, and maintaining large-scale data systems that enable data-driven decision-making.They work with various stakeholders to understand data requirements, data architectures, and implement data pipelines to support business intelligence,...

  • Data Engineer

    hace 2 semanas

    Barcelona, Barcelona, España isolutions A tiempo completo

    Are you an experienced Data Engineer with a passion for Microsoft technologies?If so, join us in our journey of innovation and shape the future of data engineering We look forward to welcoming you to our team.As a Data Engineer, you will thrive within our collaborative team, participating in high-end projects alongside skilled professionals. Your role will...

  • Data Engineer

    hace 5 días

    Barcelona, Barcelona, España DNS Info Ltd A tiempo completo

    Fulltime role - 2-3 days onsite.Mission:A Data Engineer is responsible for designing, building, and maintaining large-scale data systems that enable data-driven decision-making.They work with various stakeholders to understand data requirements, data architectures, and implement data pipelines to support business intelligence, analytics, and data science...

  • Data Engineer

    hace 5 días

    Barcelona, Barcelona, España BASE life science A tiempo completo

    The life sciences industry plays a significant role in advancing healthcare and improving the quality of life - in many cases, saving millions of lives globally. At BASE life science, this is the motivation for everything we do, aimed at pushing boundaries, seeking excellence, and making a lasting difference through consulting expertise and data-driven...

  • Cloud Data Engineer

    hace 2 semanas

    Barcelona, Barcelona, España dataguru A tiempo completo

    Qué estamos buscando?En dataguru, estamos buscando un Cloud Data Engineer apasionado por construir y optimizar la infraestructura de datos que impulse nuestras capacidades de análisis e inteligencia artificial. Si te apasiona diseñar, construir y mantener pipelines de datos robustos, escalables y eficientes en entornos cloud, te entusiasma trabajar con...

  • Senior Data Engineer

    hace 2 semanas

    Barcelona, Barcelona, España Piper Maddox A tiempo completo

    Data EngineerAsset Performance & Renewable Energy AnalyticsThe opportunityAn established renewable energy and digital solutions business is expanding itsAsset Performance Management (APM)technology team and is hiring an experiencedData Engineerto support large-scale operational renewable assets.This role sits within a product-focused engineering group...