Large-Scale Architecture for Retail Platforms Using Cloud-Native Big Data Systems

Authors

• Karthik Perikala Senior Principal Software Engineer

Abstract

Modern retail platforms operate at extreme scale, serving millions of customers while continuously integratingproduct,pricing,inventory,fulfillment, andpromotionaldatafromheterogeneoussystems. Traditional relational and document-oriented data models struggle to represent the highly connected and rapidly evolving nature of retail ecosystems under stringent latency and availability requirements.

Thispaperpresentsthedesignprinciples,system architecture, and operational characteristics of a large-scale retail data platform built using

cloud-native big data technologies.The system sustains read workloads of approximately 10,000 requests per second while supporting write ingestion rates between 300,000 and 500,000 updates per second, enabling near real-time freshness for business-critical signals such as pricing and inventory.

We describe how batch and streaming ingestion pipelines are unified within a single data architecture, how horizontal scalability and fault tolerance are achieved using distributed storage systems, and how the resulting platform supports efficient search, navigation, and recommendation workloads.The paper concludes with lessons learned from production operation andimplications for future generative AI and personalization systems.