Data Analytics Engineer SQL
Hace 3 días
Madrid, Area Metropolitana (comarca); Comunidad de Madrid, España
Hn Services España
Jornada completa
Gratis con email o Google
Guarda esta oferta y sigue tu búsqueda
Crea una cuenta gratis para guardar empleos, crear alertas y volver a esta oferta desde tu panel.
Gratis con email o Google
Al continuar, aceptas nuestros Términos & Política de Privacidad.
We are an international technology services company founded in ***
* and currently have over 2,000 employees in 5 countries: A Data Engineer to join a stable international project, based in Madrid.
Responsibilities:
Contribute to the production support and maintenance of the DataHub, correct incidents and anomalies, resolve data-related issues, and implement functional and technical developments to ensure the stability of production processes and ensure the DataHub is available to users with minimal disruption during agreed uptime requirements.
Modify existing code according to business requirements and continuously improve it for improved performance and maintainability.
Ensure the performance and security of the data infrastructure and follow data engineering best practices.
Data modeling and development of efficient data pipelines to enrich and transform large volumes of data with complex business rules, automate data pipelines, and optimize data ingestion. Design and implement scalable and secure data processing pipelines using Scala, Spark, cloud object storage, Hadoop, and COS.
Integrate data from multiple sources and formats into the raw layer of the DataHub.
Implement data transformation and quality to ensure data consistency and accuracy. Use programming languages such as Scala and SQL and tools such as Spark for data transformation and enrichment operations.
Configure CI/CD pipelines to automate deployments, unit testing, and development management.
Write and perform unit and validation tests to ensure the accuracy and integrity of the developed code.
Implement various orchestrators and scheduling processes to automate data pipeline execution (Airflow as a Service). Migrate existing Hadoop infrastructure to cloud infrastructure on Kubernetes Engine, COS, Spark as a Service, and Airflow as a Service.
Requirements:
Spark on ScalaHDFS and structured databases (SQL)HadoopS3 storage/COSShell scriptSoftware Development Lifecycle (SDLC) awarenessAgile principles and ceremoniesDremio as tool to virtualize dataOptionally/ as a plusElasticsearch and KibanaHVaultEnglish B2 or higher.
Work Model:Hibrid.
Flexible hours, Monday to Friday.xqysrnhIf you have experience in data and are looking to grow technically and professionally, don't hesitate to apply for this position.
* and currently have over 2,000 employees in 5 countries: A Data Engineer to join a stable international project, based in Madrid.
Responsibilities:
Contribute to the production support and maintenance of the DataHub, correct incidents and anomalies, resolve data-related issues, and implement functional and technical developments to ensure the stability of production processes and ensure the DataHub is available to users with minimal disruption during agreed uptime requirements.
Modify existing code according to business requirements and continuously improve it for improved performance and maintainability.
Ensure the performance and security of the data infrastructure and follow data engineering best practices.
Data modeling and development of efficient data pipelines to enrich and transform large volumes of data with complex business rules, automate data pipelines, and optimize data ingestion. Design and implement scalable and secure data processing pipelines using Scala, Spark, cloud object storage, Hadoop, and COS.
Integrate data from multiple sources and formats into the raw layer of the DataHub.
Implement data transformation and quality to ensure data consistency and accuracy. Use programming languages such as Scala and SQL and tools such as Spark for data transformation and enrichment operations.
Configure CI/CD pipelines to automate deployments, unit testing, and development management.
Write and perform unit and validation tests to ensure the accuracy and integrity of the developed code.
Implement various orchestrators and scheduling processes to automate data pipeline execution (Airflow as a Service). Migrate existing Hadoop infrastructure to cloud infrastructure on Kubernetes Engine, COS, Spark as a Service, and Airflow as a Service.
Requirements:
Spark on ScalaHDFS and structured databases (SQL)HadoopS3 storage/COSShell scriptSoftware Development Lifecycle (SDLC) awarenessAgile principles and ceremoniesDremio as tool to virtualize dataOptionally/ as a plusElasticsearch and KibanaHVaultEnglish B2 or higher.
Work Model:Hibrid.
Flexible hours, Monday to Friday.xqysrnhIf you have experience in data and are looking to grow technically and professionally, don't hesitate to apply for this position.