"assistant"
. The "content"
of each response can either be a string
or a list
of chunks with different chunk types for extended features.
A Basic Chat Request Example
Sample Response Example
Finish reason can be ofstop
means the generation endeded, ortool
means tool was called instead, orcancelled
if the generation was aborted. You can use this to see the reason behind the event occured during the generation, forexample when request has atool calling
you can use it to detect and process that request.
Chat Messages
Chat messages (messages) are a collection of prompts or messages, with each message having a specific role assigned to it, such as system, user, assistant, or tool. A system message is an optional message that sets the behavior and context for an AI assistant in a conversation, such as modifying its personality or providing specific instructions. A system message can include task instructions, personality traits, contextual information, creativity constraints, and other relevant guidelines to help the AI better understand and respond to the user’s input.See the API reference for explanations on how to set up a custom system prompt. A user message is a message sent from the perspective of the human in a conversation with an AI assistant. It typically provides a request, question, or comment that the AI assistant should respond to. User prompts allow the human to initiate and guide the conversation, and they can be used to request information, ask for help, provide feedback, or engage in other types of interaction with the AI. An assistant message is a message sent by the AI assistant back to the user. It is usually meant to reply to a previous user message by following its instructions, but you can also find it at the beginning of a conversation, for example to greet the user. A tool message only appears in the context of function calling. It is used at the final response formulation step when the model has to format the tool call’s output for the user.
To learn more about function calling, see the guide.
Message Role Orders Required
Our message role ordering follows the alternate structure — you cannot send two consecutive messages with the same role.If you need to add additional context to a role, include it inside the
content
list instead of repeating the role.
You can mix system, user, or assistant roles in alternate order for your conversation context.
Example Multiple Messages
Example Invalid Messages
Example Invalid Messages
Correct Way to Add Context to the Same Role
Stop Generation
Unlike to other providers, our Chat request API has built-in aborting generation mechanism in nature that you can send signal in your front-end to abort the generation anytime. Please check examples front-end code in typescript usingAbortSignal
.