Software Engineering Manager, Data Platform
Autodesk · Bengaluru · 12+ yrs experience · Posted 2026-07-18
Tech stack: AWS, Kubernetes, Python, SQL
Apply on the company site · Get a referral for this role
Autodesk salary & ratings · Autodesk interview process · More live openings
About the role
We are looking for an experienced Engineering Manager to lead our Batch Processing platform team — responsible for building and scaling a high-throughput, governed data platform that powers critical business decisions across the organization, including Autodesk's AI products such as Autodesk Assistant. This role sits at the intersection of high-scale platform engineering, data architecture, and organizational leadership. You will lead a team of strong engineers designing and operating large-scale distributed data processing systems on a modern lakehouse architecture, enabling reliable, secure, and cost-efficient data movement at petabyte scale.
Responsibilities:
- Team leadership & delivery
- Lead, mentor, and grow a team of strong engineers working on batch processing and data platform capabilities
- Be an AI-native leader — leverage AI tools to drive rapid iteration, experimentation, and continuous product evolution
- Drive roadmap execution with a focus on reliability, scalability, ease of use, and time-to-data
- Establish a high-ownership culture with strong engineering fundamentals: design reviews, RCA, testing, and observability
- Partner with TPMs and stakeholders to ensure predictable, high-quality delivery
- Platform architecture & engineering
- Design and evolve a scalable, AI-friendly batch processing platform using Apache Spark (PySpark), distributed compute frameworks, workflow orchestration (Airflow / Temporal), Kubernetes controllers, and AI agents
- Build systems capable of processing billions of records at petabyte scale
- Enable self-service and AI-native pipeline capabilities for internal teams through clean, extensible APIs and platform abstractions
- Optimize continuously for performance, cost efficiency, and reliability
- Lakehouse & data ecosystem
- Drive adoption of a modern lakehouse architecture (S3 + Apache Iceberg or equivalent)
- Build self-service batch processing capabilities that enable internal teams to run Spark, Python, Ray, and MLOps jobs with minimal friction and strong platform guardrails
- Enable downstream consumption via SQL engines and warehouse integrations
- Governance, security & compliance
- Ensure platform adherence to data governance and security standards
- Build self-service controls and abstractions that allow teams to manage access, enforce data policies, and meet compliance requirements without deep platform expertise
- Partner with security and compliance teams to meet enterprise and regulatory requirements
- Observability & reliability
- Build end-to-end observability: metrics, logging, alerting, SLA tracking, and reporting
- Drive incident management, root cause analysis, and continuous improvement
- Improve platform resilience through automation and resilience patterns
- Stakeholder & cross-team collaboration
- Partner with data engineering, analytics, and platform teams to drive adoption
- Work closely with infra, security, and partner teams for platform integration
- Influence architecture decisions at the org level
- Communicate effectively with senior leadership and business stakeholders
Qualifications:
- 8–12+ years in software or data engineering
- 3–5+ years managing engineering teams
- Experience leading platform teams or large-scale data systems
- Strong hands-on experience with distributed data processing (Spark, Flink, etc.), batch pipelines at scale, and AWS (preferred)
- Deep understanding of data lake / lakehouse architectures, data modeling, and workflow orchestration (Airflow, Temporal, etc.)
- Proven ability to hire, mentor, and grow high-performing teams
- Strong communication and cross-functional influence skills; ability to balance technical depth with strategic thinking
- Experience with Apache Iceberg or Delta Lake
- Familiarity with data observability tools
- Experience with metadata and catalog systems (e.g., Atlan or similar)
- Exposure to data governance frameworks and compliance standards
- Experience with Apache Iceberg or Delta Lake
- Familiarity with data observability tools
- Experience with metadata and catalog systems (e.g., Atlan or similar)
- Exposure to data governance frameworks and compliance standards
Qualifications
- 8–12+ years in software or data engineering
- 3–5+ years managing engineering teams
- Experience leading platform teams or large-scale data systems
- Strong hands-on experience with distributed data processing (Spark, Flink, etc.), batch pipelines at scale, and AWS (preferred)
- Deep understanding of data lake / lakehouse architectures, data modeling, and workflow orchestration (Airflow, Temporal, etc.)
- Proven ability to hire, mentor, and grow high-performing teams
- Strong communication and cross-functional influence skills; ability to balance technical depth with strategic thinking
- Experience with Apache Iceberg or Delta Lake
- Familiarity with data observability tools
- Experience with metadata and catalog systems (e.g., Atlan or similar)
- Exposure to data governance frameworks and compliance standards
Responsibilities
- Team leadership & delivery
- Lead, mentor, and grow a team of strong engineers working on batch processing and data platform capabilities
- Be an AI-native leader — leverage AI tools to drive rapid iteration, experimentation, and continuous product evolution
- Drive roadmap execution with a focus on reliability, scalability, ease of use, and time-to-data
- Establish a high-ownership culture with strong engineering fundamentals: design reviews, RCA, testing, and observability
- Partner with TPMs and stakeholders to ensure predictable, high-quality delivery
- Platform architecture & engineering
- Design and evolve a scalable, AI-friendly batch processing platform using Apache Spark (PySpark), distributed compute frameworks, workflow orchestration (Airflow / Temporal), Kubernetes controllers, and AI agents
- Build systems capable of processing billions of records at petabyte scale
- Enable self-service and AI-native pipeline capabilities for internal teams through clean, extensible APIs and platform abstractions
- Optimize continuously for performance, cost efficiency, and reliability
- Lakehouse & data ecosystem
- Drive adoption of a modern lakehouse architecture (S3 + Apache Iceberg or equivalent)
- Build self-service batch processing capabilities that enable internal teams to run Spark, Python, Ray, and MLOps jobs with minimal friction and strong platform guardrails
- Enable downstream consumption via SQL engines and warehouse integrations
- Governance, security & compliance
- Ensure platform adherence to data governance and security standards
- Build self-service controls and abstractions that allow teams to manage access, enforce data policies, and meet compliance requirements without deep platform expertise
- Partner with security and compliance teams to meet enterprise and regulatory requirements
- Observability & reliability
- Build end-to-end observability: metrics, logging, alerting, SLA tracking, and reporting
- Drive incident management, root cause analysis, and continuous improvement
- Improve platform resilience through automation and resilience patterns
- Stakeholder & cross-team collaboration
- Partner with data engineering, analytics, and platform teams to drive adoption
- Work closely with infra, security, and partner teams for platform integration
- Influence architecture decisions at the org level
- Communicate effectively with senior leadership and business stakeholders