Ian Greenough

Founding Product Engineer
← Back to projects

Async AI Content Pipeline

I built this to explore the job queue pattern for AI workloads that are too slow for synchronous HTTP responses. The API accepts content requests and immediately enqueues them via BullMQ. Workers pick up jobs, run Claude, and update job status in PostgreSQL. The frontend polls or listens for completion.

This decoupling is critical for production AI apps: it prevents request timeouts on slow generations, supports retries and dead-letter queues for failed jobs, and allows horizontal scaling of workers independently from the API.

The project uses the same monorepo structure (server, worker, web client) as the rest of the portfolio, making it easy to see exactly what the async layer adds over a synchronous architecture.

Technologies Used

TypeScriptNext.jsExpressPostgreSQLRedisBullMQClaude APIZod