Work Experience

AWS EMR spark optimization team: Designed and implemented multiple improvements to spark and EMRFS, including improved exchange planning, join optimizations, blob storage friendly access patterns. runtime stats generation, and other performance improvements.

Azulene: Lead developer and architect of a blob storage base upsert engine with ACID like guarantees for high performance OLAP queries.

Horizon: Lead software developer and Architect of the Horizon Platform, an EMR and redshift powered data processing platform that drives Amazon's financial data processing system. Horizon has processed more than 1 million jobs per month, processed more than quadrillion records in 2017 while processing more than a trillion dollars in transactions. Horizon is the primary data processing platform for Amazon and AWS financial data including accounting events, tax events as well as other primary financial datasets used for reporting and BI.