Hibrido BCN

We are looking for a Senior Data Platform Engineer with +6 years of experience to help drive the evolution of our Azure-based Data Platform and Lakehouse architecture. The professional will own and evolve the foundations that data product teams build on: Apache Airflow orchestration (migrating from ADF), Azure resource management (Key Vault, identity and access, networking, storage), Azure DevOps CI/CD and Infrastructure as Code, Databricks and Unity Catalog governance, observability and cost efficiency, while remaining hands‐on and close to the teams that use the platform. Orchestration Platform (Apache Airflow & ADF) Own the Apache Airflow platform: environments, deployment, upgrades, configuration, connections, variables, pools and DAG standards. Work on the migration from Azure Data Factory to Airflow and the evolution of the Metadata-Driven Orchestration Framework (DAG generation, reusable operators, dependency and scheduling patterns). Ensure Airflow is reliable and observable : SLAs, alerting, retries, backfills, health monitoring and clear failure ownership. Azure Resource Management Manage Azure resources with Infrastructure as Code (Terraform) : Databricks workspaces, Data Lake Storage, Data Factory, Key Vault, networking and supporting services. Key Vault, service principals and managed identities, secret scopes and rotation. Azure RBAC, Entra ID groups, Unity Catalog grants and ownership, following least privilege, plus access requests and periodic reviews. Own Azure DevOps : repositories, CI/CD pipelines, branch policies and multi‐environment promotion, including deployment validation and Terraform state. Track cost and governance of Azure resources: tagging, budgets, right‐sizing and cost attribution (e.g. query tagging). Hands‐on Data Platform evolution Contribute to the implementation of the Data Catalog and Data Quality Framework and it's lineage across the platform. Unity Catalog structure, cluster and job policies, Auto Loader ingestion patterns, and performance and cost optimization of workloads. Participate in architecture reviews and technical decision‐making, stating trade offs (performance, cost, reliability, complexity) with data. Collaborate with Architects, Product Owners and data engineers in technical maturation of data products. Provide second‐line support to the Run/Operations team for complex production topics. Troubleshoot and resolve platform incidents, failures and performance issues; lead root cause analysis and post‐mortems. Continuously improve runbooks, monitoring and operational processes. 6+ years of experience in data engineering, with significant time on platform or infrastructure topics ~ Strong technical expertise with Python and SQL (advanced), including PySpark and Delta Lake semantics ~ Deep hands‐on experience with Apache Airflow in production: DAG design, deployment, operations, upgrades, and orchestration of Databricks and ADF workloads ~ Hands‐on experience with Azure resource management : ~ Key Vault, managed identities and service principals, secret management ~ Azure RBAC, Entra ID, Unity Catalog permissions ~ Azure DevOps (repos, pipelines, service connections, environments) and CI/CD for data platforms ~ ADLS, networking and security basics, Azure Data Factory Infrastructure as Code (Terraform) and multi‐environment promotion Hands‐on experience with Databricks and Unity Catalog (compute, jobs, governance) Observability and monitoring, alerting and incident management Solid understanding of Data Lakehouse architecture, Medallion architecture and metadata‐driven architecture ; awareness of cost management and performance Led an Airflow implementation or migration (ideally from ADF), or run it at scale. Built metadata‐driven orchestration or reusable ETL frameworks. Applied FinOps practices on Azure and Databricks: cost attribution, budgets, forecasting and measurable savings Experience running Airflow on Kubernetes or operating containerized workloads on AKS 1,200/year for your professional development (trainings, events, certifications...) ~ Private health insurance fully covered by Capitole ~ Discounts with top brands through Capitole Club capitole-consulting. Will have access to confidential information related to Capitole and the project they are working on. Must comply with the security and internal policies of both the company and the client.