Hibrido BCN

Capitole is growing fast, and we want you to be part of the journey

Asegúrese de leer la descripción completa a continuación y, si confía en que cumple todos los requisitos, envíe su solicitud de inmediato.

We are looking for a Senior Data Platform Engineer with +6 years of experience to help drive the evolution of our Azure-based Data Platform and Lakehouse architecture. This role is focused on the platform itself. The professional will own and evolve the foundations that data product teams build on: Apache Airflow orchestration (migrating from ADF), Azure resource management (Key Vault, identity and access, networking, storage), Azure DevOps CI/CD and Infrastructure as Code, Databricks and Unity Catalog governance, observability and cost efficiency, while remaining hands‑on and close to the teams that use the platform.

Responsibilities:
Orchestration Platform (Apache Airflow & ADF)

• 
  
• Own the Apache Airflow platform: environments, deployment, upgrades, configuration, connections, variables, pools and DAG standards.
  
• Work on the migration from Azure Data Factory to Airflow and the evolution of the Metadata-Driven Orchestration Framework (DAG generation, reusable operators, dependency and scheduling patterns).
  
• Ensure Airflow is reliable and observable: SLAs, alerting, retries, backfills, health monitoring and clear failure ownership.
  

Azure Resource Management

• 
  
• Manage Azure resources with Infrastructure as Code (Terraform): Databricks workspaces, Data Lake Storage, Data Factory, Key Vault, networking and supporting services.
  
• Own secrets and identity management: Key Vault, service principals and managed identities, secret scopes and rotation.
  
• Own access management: Azure RBAC, Entra ID groups, Unity Catalog grants and ownership, following least privilege, plus access requests and periodic reviews.
  
• Own Azure DevOps: repositories, CI/CD pipelines, branch policies and multi‑environment promotion, including deployment validation and Terraform state.
  
• Track cost and governance of Azure resources: tagging, budgets, right‑sizing and cost attribution (e.g. query tagging).
  

Hands‑on Data Platform evolution

• 
  
• Contribute to the implementation of the Data Catalog and Data Quality Framework and it’s lineage across the platform.
  
• Drive observability: pipelines that are observable, debuggable and alert‑driven, with clear ownership of failures.
  
• Operate the Databricks platform: Unity Catalog structure, cluster and job policies, Auto Loader ingestion patterns, and performance and cost optimization of workloads.
  

Technical Leadership

• 
  
• Participate in architecture reviews and technical decision‑making, stating trade offs (performance, cost, reliability, complexity) with data.
  
• Define and promote engineering standards: naming conventions, patterns and guidelines, through code reviews and design reviews.
  
• Collaborate with Architects, Product Owners and data engineers in technical maturation of data products.
  
• Build and maintain developer tooling and automation, including AI‑assisted engineering workflows (shared agent instructions, skills, automated PR review).
  

Support & Operations (Run Team Collaboration)

• 
  
• Provide second‑line support to the Run/Operations team for complex production topics.
  
• Troubleshoot and resolve platform incidents, failures and performance issues; lead root cause analysis and post‑mortems.
  
• Ensure proper handover, documentation and operational readiness of platform changes.
  
• Continuously improve runbooks, monitoring and operational processes.
  

Requirements:
Technical Skills

• 
  
• 6+ years of experience in data engineering, with significant time on platform or infrastructure topics
  
• Strong technical expertise with Python and SQL (advanced), including PySpark and Delta Lake semantics
  
• Deep hands‑on experience with Apache Airflow in production: DAG design, deployment, operations, upgrades, and orchestration of Databricks and ADF workloads
  
• Hands‑on experience with Azure resource management:
  
  • 
    
  • Key Vault, managed identities and service principals, secret management
    
  • Access management: Azure RBAC, Entra ID, Unity Catalog permissions
    
  • Azure DevOps (repos, pipelines, service connec