Data pipelines · Parked
Street-level imagery, and the quota that stopped it
End-to-end collection, organisation and management of Street View imagery, built to feed a vision model.
Python ·
What it found
The quota, not the code, set the ceiling. The architecture was built for a throughput the data source would never permit, which is a design mistake worth making only once.
The work
What was actually built
Collecting street-level imagery is a solved problem right up until you want a lot of it. The code is straightforward: request, store, organise, deduplicate, resume.
The ceiling turned out to have nothing to do with the code. The API quota set it, and no amount of engineering moved it. The pipeline works and is public; it simply cannot be pointed at a city.
Parked, with the reason written down, because the next person to want this should know the constraint before they design for scale.
See it
Open it, or read the code.
Read the code, opens in a new tab on an external site
github.com. Opens in a new tab on an external site.