Optimizing Spark Performance
Read More →
Prateek Rajput
Senior Data Engineer & Backend Leader with 8+ years of experience building scalable systems for media and e-commerce platforms.
What I Do
Data Engineering
Built high-performance data pipelines processing 13TB daily, reducing processing time from 24 hours to 4.5 hours. Expert in Apache Spark, Kafka, Hadoop, and scaling systems from 10K to 22K events/sec.
Recommendation Systems
Designed and migrated recommendation engines from Hadoop to Spark for e-commerce platforms. Developed "HawkEye" computation layer to drastically reduce HDFS read times and improve user experience.
AI & LLM Solutions
Built AI-powered chatbots using Azure OpenAI Studio and Langchain for media content analysis. Integrated advanced summarization and translation capabilities for daily articles using LLMs.
Backend Development
Architect scalable backend systems using Python Flask, Java, and Scala. Built RESTful APIs and migrated legacy systems to modern cloud-native solutions with focus on reliability.
Cloud Architecture
Deploy solutions on Azure and GCP with Docker, Kubernetes, and custom Spark environments. Established cost-effective alternatives to Databricks while optimizing system efficiency.
Technical Leadership
Currently Senior Manager at Mediacorp Singapore, leading platform modernizations and mentoring teams. Authored design documents and drove NFR planning for major sales events at Flipkart.
Hackathon Winner & Multiple Awards
Top 1% in National MCA Rankings
Based in Singapore (Employment Pass)