Serverless Computing for Big Data Analytics: Performance and Cost Analysis of AWS Lambda and Google Cloud Functions

Abstract

Serverless computing has emerged as a crucial paradigm for big data analytics, offering automated resource provisioning and event-driven execution environments that aim to reduce overhead and maintenance complexities. In this work, a detailed performance and cost analysis of two prominent serverless platforms, AWS Lambda and Google Cloud Functions, is performed in the context of big data workloads. By investigating the intricacies of invocation patterns, parallel execution, cold start latencies, and resource allocation models, this research identifies essential factors that influence efficiency and scalability. The discussion presents a comprehensive framework for analyzing task throughput, response times, memory usage, and concurrency limits under varying workloads. The interplay between performance metrics and cost structures is emphasized, highlighting how function duration, memory configurations, and request frequencies contribute to diverse pricing outcomes. To further refine these outcomes, mathematical models capturing task arrival rates, provisioning delays, and cost functions are introduced to elucidate optimal resource allocation decisions. The results presented in this work underscore the potential of serverless platforms to handle large-scale datasets effectively, while also illustrating critical trade-offs in cost, performance consistency, and architectural design. This analysis serves as a reference for practitioners seeking to implement big data processing pipelines, as well as for researchers aiming to extend the theoretical underpinnings of serverless computing in cloud-based analytics.