Technical Lead - Data Engineering
Paytm · Noida · 6+ yrs experience · Posted 2026-05-18
Tech stack: AWS, Cassandra, Java, Kafka, Python, Redis, SQL, Scala, Apache Spark, Apache Flink, Hadoop, Hive
Apply on the company site · Get a referral for this role
Paytm salary & ratings · Paytm interview process · More live openings
About the role
About the role:
Build systems for collection & transformation of complex data sets for use in production systems Collaborate with engineers on building & maintaining back-end services Implement data schema and data management improvements for scale and performance Provide insights into key performance indicators for the product and customer usage Serve as team's authority on data infrastructure, privacy controls and data security.
Collaborate with appropriate stakeholders to understand user requirements Support efforts for continuous improvement, metrics and test automation Maintain operations of live service as issues arise on a rotational, on-call basis Verify whether data architecture meets security and compliance requirements and expectations .Should be able to fast learn and quickly adapt at rapid pace. java/scala, SQL,
Minimum Qualifications:
● Minimum 6+ years of experience in Big Data technologies.
● The position grows our analytics capabilities with faster, more reliable tools, handling petabytes of data every day.
● Brainstorm and create new platforms that can help in our quest to make available to cluster users in all shapes and forms, with low latency and horizontal sca
Responsibilities:
- Build systems for collection & transformation of complex data sets for use in production systems Collaborate with engineers on building & maintaining back-end services Implement data schema and data management improvements for scale and performance Provide insights into key performance indicators for the product and customer usage Serve as team's authority on data infrastructure, privacy controls and data security.
- Collaborate with appropriate stakeholders to understand user requirements Support efforts for continuous improvement, metrics and test automation Maintain operations of live service as issues arise on a rotational, on-call basis Verify whether data architecture meets security and compliance requirements and expectations.Should be able to fast learn and quickly adapt at rapid pace. java/scala, SQL,
Qualifications:
- Minimum 6+ years of experience in Big Data technologies.
- The position grows our analytics capabilities with faster, more reliable tools, handling petabytes of data every day.
- Brainstorm and create new platforms that can help in our quest to make available to cluster users in all shapes and forms, with low latency and horizontal scalability.
- Make changes to our diagnosing any problems across the entire technical stack.
- Design and develop a real-time events pipeline for Data ingestion for real-time dash- boarding.
- Develop complex and efficient functions to transform raw data sources into powerful, reliable components of our data lake.
- Design & implement new components and various emerging technologies in Hadoop Eco- System, and successful execution of various projects.
- Fluent with Strong hands-on experience with Hadoop, MapReduce, Hive, Spark, PySpark etc.
- Excellent programming/debugging skills in Python/Java/Scala.
- Experience with any scripting language such as Python, Bash etc.
- Good to have experience of working with noSQL databases like Hbase, Cassandra.
- Hands-on programming experience with multithreaded applications.
- Good to have experience in Database, SQL, messaging queues like Kafka.
- Good to have experience in developing streaming applications e.g.
- Spark Streaming, Flink, Storm, etc.
- Good to have experience with AWS and cloud technologies such as S3Experience with caching architectures like Redis etc.
Qualifications
- Minimum 6+ years of experience in Big Data technologies.
- The position grows our analytics capabilities with faster, more reliable tools, handling petabytes of data every day.
- Brainstorm and create new platforms that can help in our quest to make available to cluster users in all shapes and forms, with low latency and horizontal scalability.
- Make changes to our diagnosing any problems across the entire technical stack.
- Design and develop a real-time events pipeline for Data ingestion for real-time dash- boarding.
- Develop complex and efficient functions to transform raw data sources into powerful, reliable components of our data lake.
- Design & implement new components and various emerging technologies in Hadoop Eco- System, and successful execution of various projects.
- Fluent with Strong hands-on experience with Hadoop, MapReduce, Hive, Spark, PySpark etc.
- Excellent programming/debugging skills in Python/Java/Scala.
- Experience with any scripting language such as Python, Bash etc.
- Good to have experience of working with noSQL databases like Hbase, Cassandra.
- Hands-on programming experience with multithreaded applications.
- Good to have experience in Database, SQL, messaging queues like Kafka.
- Good to have experience in developing streaming applications e.g.
- Spark Streaming, Flink, Storm, etc.
- Good to have experience with AWS and cloud technologies such as S3Experience with caching architectures like Redis etc.
Responsibilities
- Build systems for collection & transformation of complex data sets for use in production systems
- Collaborate with engineers on building & maintaining back-end services Implement data schema and data management improvements for scale and performance
- Provide insights into key performance indicators for the product and customer usage Serve as team's authority on data infrastructure, privacy controls and data security.
- Collaborate with appropriate stakeholders to understand user requirements
- Support efforts for continuous improvement, metrics and test automation
- Maintain operations of live service as issues arise on a rotational, on-call basis Verify whether data architecture meets security and compliance requirements and expectations.
- Should be able to fast learn and quickly adapt at rapid pace. java/scala, SQL