Endpoints for managing conversations across distributed LLM hosts.
Start a new conversation with a host supporting the specified model.
{
"modelName": "mistral-7b"
}
Guid of the new conversationStream responses from the LLM for a given prompt. Results are returned as a sequence of text chunks.
{
"conversationId": "f2a1c4e9-9d8e-4a88-bb3b-73c71f65a1df",
"text": "Write a haiku about AI."
}
End an active conversation and release resources.
"f2a1c4e9-9d8e-4a88-bb3b-73c71f65a1df"
{"message":"Conversation ended successfully"}