Python Jobs in Gurgaon

Job Title: Lead / Architect – Data Engineering (AWS | Databricks)

📍 Location: Bangalore / Chandigarh

💼 Work Mode: Hybrid / Work from Office

🧑‍💼 Experience: 8+ Years

Share Cv at hr@talentaspire.com or Contact +91-7376738923

🔹 Role Overview

We are seeking a Lead / Architect – Data Engineering with 8+ years of experience in building scalable data platforms, preferably within banking, lending, or financial services domains.

This is a high-impact, hands-on leadership role, where you will drive end-to-end data architecture, modernize BI, AI, and ML pipelines, and deliver production-grade, audit-ready data systems. You will collaborate with global stakeholders and play a key role in enabling data-driven decision-making and business growth.

🔹 Key Responsibilities

• Lead the architecture and development of end-to-end analytics platforms
• Design and build scalable BI, analytics, and reporting data pipelines
• Define and implement data architecture, standards, and best practices
• Set up and manage AWS Glue for data ingestion and data lake architecture
• Deploy and govern Databricks workspaces with secure access and controls
• Refactor legacy systems (SAS) into modern Spark-based architectures
• Build PySpark / Spark SQL ELT pipelines using Delta Tables (ACID compliance)
• Orchestrate workflows using Databricks Workflows / Airflow (if applicable)
• Develop and optimize batch processing pipelines for large-scale data
• Drive data lineage, data quality, and validation frameworks
• Design and deliver interactive dashboards aligned with business needs
• Ensure secure deployment with row-level security and governance controls
• Transition analytics solutions into production-grade, scalable systems
• Lead and mentor data engineering and BI teams
• Work closely with stakeholders to translate business requirements into data solutions

🔹 Required Skills

• Strong expertise in Python, PySpark, and Spark SQL
• Advanced proficiency in SQL and large-scale data processing
• Hands-on experience with Databricks (must-have)
• Strong experience with AWS Glue and cloud-based data pipelines
• Experience with Tableau or similar BI tools
• Knowledge of SAS (migration/refactoring preferred)
• Strong understanding of data governance, lineage, and quality frameworks
• Experience in building scalable data pipelines and analytics platforms
• Exposure to AI/ML data pipelines and modern data architectures

🔹 Leadership & Ownership Expectations

• Drive end-to-end ownership of data platforms and architecture
• Make key technical decisions aligned with scalability and performance
• Mentor teams while remaining hands-on in coding and design
• Align data solutions with business outcomes and growth objectives
• Collaborate with global clients and cross-functional teams

🔹 Preferred Qualifications

• AWS Certified Data Engineer – Associate
• Experience in banking, lending, or credit risk domains
• Proven experience leading data engineering / BI teams

🔹 Skills Summary

Must Have: Python, PySpark, Advanced SQL, Databricks, AWS Glue, Tableau

Good to Have: SAS, Spark SQL, Data Pipeline Orchestration, Data Governance

Bonus: Exposure to AI/ML pipelines, Advanced Prompt Engineering

🔹 Why Join?

• Work on high-impact financial data transformation projects
• Exposure to AI/ML and modern data platforms
• Opportunity to work with global stakeholders
• Strong ownership-driven and growth-focused environment
• Learning and leadership opportunities

🔹 Benefits

• Health insurance
• Paid time off
• Learning & development programs