Streaming an LLM response through Rails and React
Eleven seconds of spinner became 400ms to first token with no speedup at all. Server-sent events over ActionCable, the SSE frame-splitting bug everyone ships, and the capacity arithmetic that breaks when a request holds a thread for eleven seconds.
- Rails
- React
- LLM
- Streaming