Senior Python Data Engineer

Introduction to the Organization HCLTech is a global leader in technology and IT services, recognized for delivering innovative solutions that empower organizations to transform digitally. With a presence in over 50 countries and a diverse workforce exceeding 220,000 employees, HCLTech is committed to fostering a culture of innovation, collaboration, and excellence. The company has achieved industry acclaim for its client-centric approach, cutting-edge technologies, and a strong focus on sustainability and social responsibility. Overview of the Role As a Senior Python Data Engineer, you will play a pivotal role in architecting and delivering robust enterprise-grade data solutions that drive critical business insights and efficiencies. You will lead the development of scalable data pipelines and frameworks, ensuring data quality, reliability, and performance for mission-critical applications. Your expertise will directly impact the digital transformation journeys of our clients, supporting HCLTech’s mission to deliver operational excellence and innovation at scale. Detailed Responsibilities
• Design, develop, and implement enterprise-grade data solutions using Python, with advanced utilization of Pandas and NumPy for large-scale data processing.
• Build, optimize, and maintain scalable ETL/ELT pipelines for data ingestion, preprocessing, cleansing, transformation, validation, and enrichment.
• Develop reusable frameworks and components to support efficient and maintainable data engineering operations.
• Optimize data pipelines for enhanced performance, reliability, and operational efficiency in production environments.
• Apply software engineering best practices, including clean code principles, modular architecture, automated testing, and thorough code reviews.
• Transform analytical prototypes into production-ready, industrialized solutions.
• Design, implement, and maintain CI/CD pipelines to automate build, test, deployment, and release processes.
• Manage Python deployment artifacts, packages, and libraries for production use.
• Implement and maintain monitoring, logging, alerting, and observability capabilities for data services.
• Provide production support, troubleshooting, and proactive resolution for deployed applications.
• Collaborate effectively with multidisciplinary teams and stakeholders to translate business requirements into technical solutions.
• Ensure solutions adhere to the highest standards of scalability, robustness, security, and operational excellence.
• Champion and contribute to Agile development methodologies. Skill Requirements
• Bachelor’s or Master’s degree in Computer Science, Information Technology, Engineering, or a related field.
• Extensive hands-on experience in Python programming, with advanced expertise in data engineering libraries such as Pandas and NumPy.
• Proven track record in building and optimizing ETL/ELT pipelines for complex, large-scale datasets.
• Strong foundation in data engineering processes: ingestion, preprocessing, cleansing, transformation, validation, and enrichment.
• Deep understanding of software engineering best practices, including DevOps principles and the software delivery lifecycle.
• Experience in designing and maintaining CI/CD pipelines for automated deployments.
• Proficiency in creating and managing Python packages and deployment artifacts in production environments.
• Familiarity with cloud platforms, particularly Azure or similar cloud environments.
• Experience with monitoring, logging, and alerting tools for production data services.
• Demonstrated ability to collaborate with multidisciplinary teams and manage stakeholder expectations effectively.
• Excellent problem-solving skills and results-oriented mindset. Other Requirements (Optional)
• Professional certifications in cloud platforms (e.g., Azure Certified Data Engineer, AWS Data Analytics) are a plus.
• Experience with additional data engineering tools and frameworks (Spark, Airflow, Kafka, etc.) is advantageous.
• Exposure to machine learning model deployment and MLOps practices is a bonus.
• Strong verbal and written communication skills for technical documentation and stakeholder interactions. Career Development Opportunities At HCLTech, we are committed to nurturing talent and fostering long-term career growth. You will have access to:
• Continuous learning through global training programs, certifications, and workshops on the latest technologies and methodologies.
• Clear career progression paths and leadership development initiatives.
• Opportunities to work on cutting-edge projects for Fortune 500 clients, expanding your expertise and industry exposure.
• Internal mobility across geographies and business units, supporting your professional aspirations.
• A supportive and inclusive culture that values innovation, diversity, and work-life balance.