AWS GuardDuty

Overview

Building and scaling AWS GuardDuty's threat detection infrastructure, processing millions of security events daily across thousands of AWS Organizations. My work focuses on automated protection deployment, coverage visibility, high-throughput data pipelines, and metrics at scale.

Key Contributions

• 01Data Source Enabler - Built automated security feature deployment system using Lambda, enabling administrators to configure protection across Organization member accounts with a single click. Engineered a solution managing 11K+ AWS Organizations (up to 50K members each), reducing deployment time from hours to minutes.
• 02Coverage Reporter - Spearheaded Organization Coverage Reporter using AWS Fargate, providing delegated administrators visibility across 50K+ member accounts. Developed an ECS/Fargate-based system processing daily coverage metrics for 11K+ organizations with sub-5-minute report generation.
• 03ETL Pipeline- Created a high-performance Spark/EMR ETL pipeline processing 20M+ daily findings (800K/hour) with sub-minute latency for GuardDuty's Extended Threat Detection (re:Invent 2024 launch).
• 04Usage Metrics - Developed CloudWatch Metrics integration for 8M+ active accounts, publishing 240M+ metrics/hour at 99.9% reliability with hierarchical organization-level aggregation.