For Remembrance Day, I spent 72 hours building theywerehere.co.uk – a searchable database of every Commonwealth soldier who died in WWI and WWII.
The Data
- Source: Commonwealth War Graves Commission
- Records: 1,750,608 soldiers
- Fields: Name, rank, regiment, date, cemetery, age
The Tech
- Scraped with TypeScript + Puppeteer
- Postgres on Supabase
- Next.js frontend
- Infinite scroll with virtual windowing
Why I built it
My great-grandfather's name is somewhere in those 1.75M. So I built this so no soldier is just a statistic.
Happy to answer technical questions about the scraping/database/UI choices.
by CameraNovel5762