I recently wrapped up a project with the Newparts team that I am genuinely proud of, and this is the engineering story behind it.
The problem
In the automotive parts industry, determining whether two components are compatible with a vehicle and its variants is surprisingly complex. The team was doing this manually: 40 to 50 minutes per kit with just two parts, across 50 to 55 compatibility parameters. A single project could consume an entire week of human effort.
The solution: the Kit Builder
I designed and built an end-to-end automated pipeline that accepts hierarchical input (part types, not just individual parts) and outputs a fully compatible kit with matched vehicles and product attributes, with no manual intervention.
The architecture
- FastAPI backend with PostgreSQL and legacy SQL Server integration
- Celery and Redis for asynchronous, distributed task processing
- Next.js frontend
- Fully Dockerised for portability and reliability
The engineering highlight
The core of the system is a recursive search for compatible part matches. Instead of writing results directly to PostgreSQL, which risks memory bloat and I/O bottlenecks at scale, compatible matches are streamed into Redis first, acting as a fast, resilient buffer, then flushed to PostgreSQL. This keeps the system stable under heavy load and prevents data loss mid-pipeline.
The pipeline stages: extract, transform, find fitment, generate product attributes, export.
The outcome
The Kit Builder now produces more accurate results than the manual process did. The team told me the accuracy levels are beyond what they have seen before, and that it feels like it knows the catalogue better than they do. The proof of concept was delivered in week one; the full system in 47 days.
If you are working on complex data pipelines, legacy system integrations or automation problems, get in touch.