AI Chat with streaming

Laratic provides a nice AI chat implementation with livewire and a background job that can stream responses in real-time. Similar to AI agents like Chat GPT, Claude, etc.

How it works

  1. User submits a prompt in the chat interface.
  2. The Livewire component (App\\Livewire\\AiChat) processes the prompt, creates the message history, prepares it for the API, and dispatches the streaming job.
  3. A background job (App\\Jobs\\StreamAiReply) is created and queued. It sends the request to OpenAI using Prism and saves the response chunks in cache as they arrive.
  4. The Livewire component polls the cache every 250ms using wire:poll to get the updated buffer, enabling real-time streaming in the UI.
  5. Once generation completes, the AI helper calculates and saves the usage data (tokens, model, user ID) for tracking and cost analysis.

Setting Up the Queue

This implementation uses a background queue to handle AI streaming. The job is dispatched to the ai queue, which runs asynchronously.

To start processing the queue, run:

php artisan queue:work --queue=ai

Changing the AI Model

To change the AI model, edit the $model variable in the StreamAiReply job's handle() method

$provider = 'openai';
$model = 'gpt-4o-mini';

Make sure that you have the API key for the provider in your .env file. Update the pricing in app/Helpers/ai_helpers.php to match your chosen model for accurate cost tracking.

Message History

The chat keeps the entire conversation history. Each message in the $messages array is sent to the API, so the AI has full context of the conversation.

The message history is stored in the $this->messages array in the Livewire component. Each time you send a message, it's added to this array along with the assistant's response.

You can use this to store the conversation history in a database or a file.

array (
    0 => 
    array (
    'role' => 'user',
    'content' => 'tell me a joke',
    ),
    1 => 
    array (
    'id' => '01K8KE7GABGJHE2XFVK9GFXDEF',
    'role' => 'assistant',
    'content' => 'Why don\'t skeletons fight each other? 

Because they don\'t have the guts!',
    )
)

Stopping the Stream

Users can stop streaming at any time by clicking the "Stop" button. This works by setting two cache flags:

Cache::put("ai:cancel:{$this->streamingId}", true, now()->addMinutes(10));
Cache::put("ai:done:{$this->streamingId}", true, now()->addMinutes(10));

The job checks for the cancel flag on each chunk and exits early if set. The UI stops polling when the done flag is detected.

Formatting the AI Response

The AI response is rendered using Markdown. The view uses the renderMarkdown() function from chat.js which:

Related Routes

Method Path Route name
GET /ai-chat ai.chat

Next Steps