Cloud ETL Pipeline

Processed 1.6M+ records at ~70,000 rows/sec with peak memory under 300MB via chunked reading and automatic encoding detection. Idempotent batch upserts and aggregation-based deduplication give the pipeline fault tolerance, cutting full load time from 4 minutes to 45 seconds.

M-Pesa Airflow Transaction Pipeline

DAG-orchestrated M-Pesa transaction processing with a Faker-driven synthetic data generator, a rule-engine-based multi-stage cleaning pipeline, and a Docker Compose stack reproducible with a single command. CI runs at a 100% test pass rate gating all merges.

Real-Time Transaction Streaming System

Kafka producer/consumer model for live transaction ingestion, with consumer-group design for horizontal scalability and exactly-once semantics to prevent double-counting. Plus live Plotly visualisations rendered straight from the consumer layer in Jupyter.

End-to-End SQL Pipeline Portfolio

Multi-tier data lake pattern from CSV staging through OLTP models into a star schema warehouse, with SCD Type 2 on customer dimensions and recursive CTE queries surfacing revenue shifts, fraud signals, and retention metrics.

Streaming Pipeline

A real-time data streaming pipeline for scalable ingestion, processing, transformation, and analytics of continuous data streams.

Early Portfolio Site

My first public portfolio project, built under an earlier GitHub handle before consolidating my work under this account.