Serverless Computing: Event-Driven Architecture and Scaling
Serverless Function Execution Model
Serverless abstracts servers: developers write functions, cloud provider manages infrastructure (provisioning, scaling, patching). AWS Lambda: write function (handler exported), deploy as ZIP, Lambda automatically invokes on trigger (API call, S3 upload, timer). Execution environment: function runs in container (container created on first invocation, reused for subsequent calls within same second). Cold start: first invocation takes 1-5 seconds (container initialization, dependency loading). Warm invocation: function already in-memory, responds <100ms. Warmth maintained for 15 minutes (typical), then container destroyed. Function lifecycle: (1) create container (~2s), (2) load code and dependencies (~1s), (3) execute handler (~50ms typical), (4) return result. Concurrency model: 1000 concurrent functions per AWS account (default limit, can request increase). Reserved concurrency: guarantee 100 functions always available (pays even if unused). Provisioned concurrency: pre-warm containers before traffic spike (eliminates cold starts).
Pricing and Cost Optimization
AWS Lambda pricing: $0.0000002 per request (first 1 million free monthly) + $0.0000166 per GB-second (128MB ≈ $0.0000021 per second). Example: 1 million requests, each 128MB, 1 second duration: 1M * $0.0000002 + (1M * 1 * 0.125GB * $0.0000166) = $200 + $2,075 = $2,275/month. Comparison to EC2: always-on m5.large (2GB RAM, $30/month), scaling with Lambda cheaper unless sustained >10K requests/second. Reserved Capacity: pre-purchase compute (AWS Compute Savings Plans, 40% discount, 1-3 year commitment). Cost reduction strategies: (1) reduce memory (larger memory = faster, but costs more), 256MB = ~0.5K requests/second vs 3GB = ~50K requests/second (sweet spot varies), (2) optimize duration (dependency size, code inefficiency), (3) batch requests (fewer invocations, same work). Monitoring: Lambda insights show duration, billed duration (rounded to 100ms), memory used (may be over-provisioned).
Event Sources and Integrations
Synchronous triggers: API Gateway (HTTP calls), ALB (load balancer), direct invocation. Response sent immediately after function completes. Asynchronous triggers: S3 (file upload), SNS (notification), SQS (queue message), Kinesis (stream). Lambda processes events, returns immediately (success = event accepted, may fail silently). Dead-letter queue (DLQ): failed events stored for retry analysis. Scheduled triggers: EventBridge cron expression (rate: '5 minutes', cron '0 12 * * ? *' daily at noon). Streams: DynamoDB Streams (on table changes), Kinesis (real-time data), SQS (batch window). Batch size: 10-100 records per Lambda invocation (tunable, higher batch = fewer invocations, higher latency). Concurrency control: reserved concurrency prevents runaway costs (function won't exceed limit, excess events queued).
Limitations and Architectural Patterns
Hard limits: 15-minute timeout (long-running processes not suitable), 10GB ephemeral storage (/tmp directory), 512MB RAM minimum, 69KB request/response payload limit (larger data use S3 pre-signed URLs). Soft limits: Lambda quotas (concurrency, storage) can be increased. Stateless design: functions must be stateless (containers destroyed after invocation, state persisted in external store). State options: DynamoDB (millisecond latency, $1.25/GB/month), S3 (cheap but slower), ElastiCache (in-memory, expensive). Architectural patterns: (1) Lambda + API Gateway = REST API, (2) Lambda + DynamoDB = NoSQL database, (3) Lambda + SQS = async queue processing, (4) Lambda + Step Functions = complex workflows (parallel tasks, error handling, retries). Step Functions: state machine orchestrates multiple Lambdas, handles retries, timeouts, branching logic. Example: Lambda 1 validates input → Lambda 2 processes → Lambda 3 sends notification (coordinated by Step Functions).
Development and Deployment Tools
AWS SAM (Serverless Application Model): CloudFormation extension simplifying Lambda deployment. Template.yaml defines functions, triggers, permissions (IaC approach). Example: globals: Handler, Runtime, Environment; Resources: MyFunction, Properties: Handler index.handler, Runtime python3.11, Events: ApiEvent (Type AWS::ApiEvent, Properties: Path /api, Method get). sam build compiles dependencies, sam local start-api runs locally, sam deploy pushes to AWS. Serverless Framework: open-source tool supporting AWS/Azure/GCP. serverless.yml defines functions, plugins (auto-scales, layers). serverless deploy packages and deploys. AWS Lambda Layers: shared code/libraries (.zip file, imported by multiple functions). Example: create layer with common libraries (requests, pandas), multiple functions reference same layer (reduce deployment size). Local testing: SAM local/Serverless offline plugin allows local Lambda testing (simulate API calls, events). CI/CD: GitHub Actions → build → test → deploy to staging → smoke tests → deploy production (automated).
Observability and Performance Tuning
CloudWatch Logs: function logs printed to stdout/stderr. Logs Insights: query logs with SQL-like syntax (find errors, latency distribution, error rate trends). Example: fields @duration | stats pct(@duration, 95) as p95 (95th percentile duration). X-Ray tracing: distributed tracing across Lambda, RDS, HTTP calls. Shows: function duration breakdown, downstream service latency, errors. ServiceLens: visualizes traces and dependencies. Performance optimization: (1) code efficiency (reduce cold start), (2) memory sizing (test various memory levels, measure cost vs speed), (3) connection reuse (keep database connections warm between invocations), (4) layer caching (frequently used data cached, reduce database hits). Benchmarking: measure function duration with different memory configurations (128MB-3GB range). Sweet spot: 1GB memory often optimal (good latency/cost ratio). Excessive memory: diminishing returns (add extra 1GB, duration decreases 10%, but cost increases 11%).
Security and Best Practices
IAM execution role: Lambda assumes role, permissions granted (least privilege). Example: function needs S3 read, DynamoDB write. Grant s3:GetObject for bucket/arn and dynamodb:PutItem for table/arn (not blanket permissions). Secrets management: AWS Secrets Manager stores DB passwords, API keys. Lambda retrieves secret (cached for 1 hour). Avoid hardcoding secrets in code (security risk, visible in CloudWatch logs). VPC deployment: Lambda runs in VPC (private subnet, can access RDS/internal resources). Caveat: NAT gateway required for internet access (adds latency, cost). Cold start reduction: (1) minimize dependency size (exclude unused libraries), (2) compiled languages (Go ~1MB, Node 20MB+), (3) provisioned concurrency (expensive but eliminates cold starts), (4) connections pooling (reuse DB connections). Development best practices: (1) use layers for common code, (2) parameterize environment-specific values (dev/staging/prod), (3) structured logging (JSON format for parsing), (4) monitoring alerts (error rate, duration outliers).