Hibrido Bcn

Capitole is growing fast, and we want you to be part of the journey
We are looking for a
Senior Data Platform Engineer
with
+6 years of experience
to help drive the evolution of our
Azure-based Data Platform
and
Lakehouse architecture
.
This role is focused on the
platform itself
.
The professional will own and evolve the
foundations
that data product teams build on:
Apache Airflow orchestration (migrating from ADF)
,
Azure resource management
(Key Vault, identity and access, networking, storage),
Azure DevOps CI/CD and Infrastructure as Code
,
Databricks and Unity Catalog governance
,
observability
and
cost efficiency
, while remaining
hands?on
and close to the teams that use the platform.
Responsibilities:
Orchestration Platform (Apache Airflow & ADF)
Own the
Apache Airflow
platform: environments, deployment, upgrades, configuration, connections, variables, pools and
DAG standards
.
Work on the migration from
Azure Data Factory to Airflow
and the evolution of the
Metadata-Driven Orchestration Framework
(DAG generation, reusable operators, dependency and scheduling patterns).
Ensure Airflow is
reliable and observable
: SLAs, alerting, retries, backfills, health monitoring and clear failure ownership.
Azure Resource Management
Manage Azure resources with
Infrastructure as Code (Terraform)
: Databricks workspaces,
Data Lake Storage
, Data Factory,
Key Vault
, networking and supporting services.
Own
secrets and identity management
: Key Vault, service principals and managed identities, secret scopes and rotation.
Own
access management
: Azure RBAC, Entra ID groups, Unity Catalog grants and ownership, following
least privilege
, plus access requests and periodic reviews.
Own
Azure DevOps
: repositories,
CI/CD pipelines
, branch policies and
multi?environment promotion
, including deployment validation and Terraform state.
Track
cost and governance
of Azure resources: tagging, budgets, right?sizing and cost attribution (e.g. query tagging).
Hands?on Data Platform evolution
Contribute to the implementation of the
Data Catalog
and
Data Quality Framework
and it's
lineage
across the platform.
Drive
observability
: pipelines that are observable, debuggable and alert?driven, with clear ownership of failures.
Operate the
Databricks
platform:
Unity Catalog
structure, cluster and job policies,
Auto Loader
ingestion patterns, and performance and cost optimization of workloads.
Technical Leadership
Participate in
architecture reviews
and technical decision?making, stating
trade offs
(performance, cost, reliability, complexity) with data.
Define and promote
engineering standards
: naming conventions, patterns and guidelines, through code reviews and design reviews.
Collaborate with Architects, Product Owners and data engineers in
technical maturation
of data products.
Build and maintain
developer tooling and automation
, including
AI?assisted engineering
workflows (shared agent instructions, skills, automated PR review).
Support & Operations (Run Team Collaboration)
Provide
second?line support
to the Run/Operations team for complex production topics.
Troubleshoot and resolve
platform incidents
, failures and performance issues; lead
root cause analysis
and post?mortems.
Ensure proper
handover, documentation and operational readiness
of platform changes.
Continuously improve
runbooks, monitoring and operational processes
.
Requirements:
Technical Skills
6+ years of experience in data engineering, with significant time on
platform or infrastructure
topics
Strong technical expertise with Python and SQL (advanced), including PySpark and
Delta Lake
semantics
Deep hands?on experience with
Apache Airflow
in production: DAG design, deployment, operations, upgrades, and orchestration of Databricks and ADF workloads
Hands?on experience with
Azure resource management
:
Key Vault
, managed identities and service principals, secret management
Access management
: Azure RBAC, Entra ID, Unity Catalog permissions
Azure DevOps
(repos, pipelines, service connections, environments) and CI/CD for data platforms
ADLS, networking and security basics, Azure Data Factory
Infrastructure as Code
(Terraform) and multi?environment promotion
Hands?on experience with
Databricks
and
Unity Catalog
(compute, jobs, governance)
Observability and monitoring
, alerting and incident management
Solid understanding of Data Lakehouse architecture,
Medallion archit