Principal Data Engineer

Trident Search have partnered with one of Europe’s fastest growing organisations in Cyber Threat Intelligence. They are looking for a Principal Data Engineer to join their ever growing team with the opportunity of leading the entire data team after six months in the position.
- Good understanding of how to satisfy intelligence requirements by looking at the ideal data sources.
- Strong ETL experience
- Define data validation QA processes and implement them.
- Analyse the collected data to ensure proper classification and processing.
- Define and maintain KPIs to which degree our Information Collection Plans are satisfied.
- Research, define and improve data acquisition and processing.
- Maintain, improve and operate our scalable high performance data collection platform. Identify and resolve performance bottlenecks, ensuring the system can handle increasing data volume and user load.
- Monitor system performance and proactively optimize data collection processes to ensure high availability, reliability, and efficiency.
- Conduct thorough testing and debugging of our data collection platform. Engineer scalable data storage and retrieval solutions matching requirements of our customers.

**Qualifications**
- Ideally around 7 years experience in Data Collection Management
- Masters degree in Data Science/Engineering or additional 4 years equivalent experience
- Very experienced in Python
- Strong Linux/Unix experience
- Experience in system design for microservice architectures and data flow pipelines
- Experience with SQL/NOSQL databases
- Fluent in English
- Excellent knowledge of IT security principles