Data Pipeline Engineer

hace 4 días

Barcelona, España Near Space Labs A tiempo completo

At Near Space Labs we design, build and operate a fleet of stratospheric robots to image the earth. We launch our proprietary, balloon-based imaging robots to heights between 40,000 and 60,000 feet. From this vantage point, we capture petabytes of imagery for a variety of use cases.

We seek a proactive Data Engineer to drive the evolution of our petabyte-scale geospatial imagery pipeline. This role will be instrumental in ensuring the seamless flow of data, collaborating closely with data and software engineers You will provide ongoing support by actively monitoring and troubleshooting data pipeline performance and reliability. Your contributions will directly impact our ability to deliver high-quality imagery to customers with optimal speed and reliability.

**Your responsibilities will include**:

- Build and maintain our proprietary data pipeline for batch geospatial data processing.
- Monitor and support data pipeline performance and reliability.
- Develop new data processing tasks to support evolving data requirements.
- Design and implement data storage solutions using cloud-based storage services.
- Develop and optimize data workflows using distributed systems.
- Rapid generation of functional prototypes.

**Location**

This is a position for EU nationals or work permit holders based in Barcelona. You can work from home or the office as needed.

**Our Stack**:

- Python & Typescript
- Distributed system, gRPC services, Protobuf, REST
- Kubernetes, Istio
- Google Cloud Platform, Docker, Pub/sub
- PostgreSQL, MongoDb, Google Cloud Storage
- GDAL, Shapely, Spatio-Temporal Asset Catalogs

**Your Skills**:

- ** Data Pipeline Expertise**: Proven experience designing, building, and maintaining scalable data pipelines using distributed processing frameworks (e.g., Airflow, Spark, etc).
- ** Cloud Storage and Data Warehousing**: Experience working with cloud-based storage solutions (e.g., AWS S3, Google Cloud Storage, Azure Blob Storage) and data warehousing technologies.
- ** Understanding of distributed systems**: Knowledge of how distributed systems work and the challenges that they present.
- ** Problem-Solving and Troubleshooting**: Strong ability to diagnose and resolve complex data infrastructure and pipeline issues.
- ** Self-Starter and Collaborative**: Ability to work independently and collaboratively in a fast-paced environment, managing projects and deliverables effectively.

**What We Offer**:

- An exciting startup culture where you will have the opportunity to play a critical role in building and scaling a one-of-a-kind technology and organization.
- You will be part of an enthusiastic, international and motivated team of professionals who are committed to building unique technologies, being rigorous, and finding novel solutions to interesting problems.
- A diverse and inclusive workplace where we welcome people of different backgrounds, experiences and perspectives.
- A commitment that you will never be bored.

**Equal employment opportunity**:
Near Space Labs is committed to diversity in our organization and building an equitable and inclusive environment for people of all backgrounds and experiences. Near Space Labs provides equal employment opportunities to all employees and applicants for employment without regard to race, color, religion, sex, national origin, age, disability or genetics.

J-18808-Ljbffr

  • Data Engineer: Scale Data Pipelines

    hace 1 semana

    Barcelona, España Perk A tiempo completo

    A multinational travel and spend management company in Barcelona is seeking a Data Engineer to improve Data Governance and Integration processes. This role involves maintaining data infrastructure, developing data pipelines, and collaborating with analysts to optimize analytics. Ideal candidates have 3+ years of relevant experience, are SQL masters, and...

  • Senior PySpark Data Engineer — AWS Data Pipelines

    hace 1 semana

    Barcelona, España Axiom Software Solutions Limited A tiempo completo

    A leading software solutions provider in Barcelona is seeking an experienced Data Engineer proficient in PySpark and AWS services such as Athena and EMR. The ideal candidate will design and optimize data processing systems while ensuring data integrity and performance. Responsibilities include developing data pipelines and collaborating with team members to...

  • Remote Data Engineer — AWS

    hace 1 semana

    Barcelona, España Antal International Network A tiempo completo

    A rapidly growing organization in Conversational AI seeks a Data Engineer to design and maintain their data ecosystem. The ideal candidate has 3+ years of experience, strong SQL and Python skills, and is capable of remote work in an English-first, async-friendly team. The role involves building data pipelines, creating dashboards, and collaborating across...

  • Data Engineer

    hace 1 semana

    Barcelona, España WIZELINE A tiempo completo

    A global AI-native technology solutions provider in Barcelona is seeking a Data Engineer with 3+ years of experience in building scalable data pipelines. You will be responsible for data migration, modeling, and optimization, requiring strong skills in SQL, Python, and Apache Airflow. This role offers a high-impact environment and emphasizes professional...

  • Senior Data Engineer — Hybrid Fleet Data Pipelines

    hace 2 semanas

    Barcelona, España ZF Group A tiempo completo

    A leading technology company in Barcelona is looking for a Senior Data Engineer to enhance their innovative fleet management ecosystem. This role involves designing efficient data pipelines and ensuring high data quality while collaborating with cross-functional teams across Europe. The ideal candidate possesses strong skills in Python and extensive...

  • Senior Data Engineer: Scale Fleet Data Pipelines

    hace 1 semana

    Barcelona, España ZF Group A tiempo completo

    A leading automotive technology company is seeking a Senior Data Engineer in Barcelona to enhance their fleet management ecosystem. The role involves building and optimizing data pipeline architectures and ensuring high data quality. Successful candidates will have significant experience in data engineering, proficiency in AWS services, and strong...

  • Data Engineer: Healthcare DataOps

    hace 15 horas

    Barcelona, España MIGx AG A tiempo completo

    A leading global consulting firm is seeking a Data Engineer to join their Data and AI Engineering team in Spain. This role involves building cloud-native data solutions and managing ETL/ELT pipelines utilizing tools like Databricks and SQL. Candidates should have strong experience with Python and CI/CD automation. The company offers a hybrid work model,...

  • Remote Junior Data Engineer: Build Scalable Data Pipelines

    hace 4 días

    Barcelona, España Reverse Tech A tiempo completo

    A health and wellness digital firm is seeking a Junior Data Engineer to evolve their data stack and ensure data-driven decisions. You will work closely with data engineers and analysts to build reliable data pipelines, requiring knowledge of SQL and Python. The role offers remote flexibility, unlimited growth prospects, and a fun atmosphere while focusing on...

  • Senior Data Engineer: Oracle ERP Cloud

    hace 16 horas

    barcelona, España Next Ventures A tiempo completo

    A leading tech solutions firm in Barcelona is looking for an experienced Data Engineer to join their Data & Analytics team. The successful candidate will design and maintain data pipelines for Oracle Cloud and AWS Data Lake environments, enabling advanced analytics and enterprise reporting. Applicants should have a strong background in data engineering,...

  • Data Engineer: Build Pipelines

    hace 20 horas

    Barcelona, España AMH HEADHUNTING A tiempo completo

    Una empresa de selección de personal busca un Data Engineer con más de 7 años de experiencia para unirse a un proyecto estratégico de construcción de una plataforma de datos. El rol incluye construir y mantener pipelines de datos, desarrollar procesos ETL/ELT y colaborar con el equipo en iniciativas de IA generativa. Se requiere dominio de Python y...