CohereChatRequestV2

class oci.generative_ai_inference.models.CohereChatRequestV2(**kwargs)

Bases: BaseChatRequest

Details for the chat request for Cohere models.

Attributes:

API_FORMAT_COHERE

A constant which can be used with the api_format property of a BaseChatRequest.

API_FORMAT_COHEREV2

A constant which can be used with the api_format property of a BaseChatRequest.

API_FORMAT_GENERIC

A constant which can be used with the api_format property of a BaseChatRequest.

SAFETY_MODE_CONTEXTUAL

A constant which can be used with the safety_mode property of a CohereChatRequestV2.

SAFETY_MODE_OFF

A constant which can be used with the safety_mode property of a CohereChatRequestV2.

SAFETY_MODE_STRICT

A constant which can be used with the safety_mode property of a CohereChatRequestV2.

TOOLS_CHOICE_NONE

A constant which can be used with the tools_choice property of a CohereChatRequestV2.

TOOLS_CHOICE_REQUIRED

A constant which can be used with the tools_choice property of a CohereChatRequestV2.

api_format

[Required] Gets the api_format of this BaseChatRequest.

citation_options

Gets the citation_options of this CohereChatRequestV2.

documents

Gets the documents of this CohereChatRequestV2.

frequency_penalty

Gets the frequency_penalty of this CohereChatRequestV2.

is_log_probs_enabled

Gets the is_log_probs_enabled of this CohereChatRequestV2.

is_raw_prompting

Gets the is_raw_prompting of this CohereChatRequestV2.

is_search_queries_only

Gets the is_search_queries_only of this CohereChatRequestV2.

is_stream

Gets the is_stream of this CohereChatRequestV2.

is_strict_tools_enabled

Gets the is_strict_tools_enabled of this CohereChatRequestV2.

max_tokens

Gets the max_tokens of this CohereChatRequestV2.

messages

[Required] Gets the messages of this CohereChatRequestV2.

presence_penalty

Gets the presence_penalty of this CohereChatRequestV2.

priority

Gets the priority of this CohereChatRequestV2.

response_format

Gets the response_format of this CohereChatRequestV2.

safety_mode

Gets the safety_mode of this CohereChatRequestV2.

seed

Gets the seed of this CohereChatRequestV2.

stop_sequences

Gets the stop_sequences of this CohereChatRequestV2.

stream_options

Gets the stream_options of this CohereChatRequestV2.

temperature

Gets the temperature of this CohereChatRequestV2.

thinking

Gets the thinking of this CohereChatRequestV2.

tools

Gets the tools of this CohereChatRequestV2.

tools_choice

Gets the tools_choice of this CohereChatRequestV2.

top_k

Gets the top_k of this CohereChatRequestV2.

top_p

Gets the top_p of this CohereChatRequestV2.

Methods:

__init__(**kwargs)

Initializes a new CohereChatRequestV2 object with values from keyword arguments.

get_subtype(object_dictionary)

Given the hash representation of a subtype of this class, use the info in the hash to return the class of the subtype.

API_FORMAT_COHERE = 'COHERE'

A constant which can be used with the api_format property of a BaseChatRequest. This constant has a value of “COHERE”

API_FORMAT_COHEREV2 = 'COHEREV2'

A constant which can be used with the api_format property of a BaseChatRequest. This constant has a value of “COHEREV2”

API_FORMAT_GENERIC = 'GENERIC'

A constant which can be used with the api_format property of a BaseChatRequest. This constant has a value of “GENERIC”

SAFETY_MODE_CONTEXTUAL = 'CONTEXTUAL'

A constant which can be used with the safety_mode property of a CohereChatRequestV2. This constant has a value of “CONTEXTUAL”

SAFETY_MODE_OFF = 'OFF'

A constant which can be used with the safety_mode property of a CohereChatRequestV2. This constant has a value of “OFF”

SAFETY_MODE_STRICT = 'STRICT'

A constant which can be used with the safety_mode property of a CohereChatRequestV2. This constant has a value of “STRICT”

TOOLS_CHOICE_NONE = 'NONE'

A constant which can be used with the tools_choice property of a CohereChatRequestV2. This constant has a value of “NONE”

TOOLS_CHOICE_REQUIRED = 'REQUIRED'

A constant which can be used with the tools_choice property of a CohereChatRequestV2. This constant has a value of “REQUIRED”

__init__(**kwargs)

Initializes a new CohereChatRequestV2 object with values from keyword arguments. The default value of the api_format attribute of this class is COHEREV2 and it should not be changed. The following keyword arguments are supported (corresponding to the getters/setters of this class):

Parameters
  • api_format (str) – The value to assign to the api_format property of this CohereChatRequestV2. Allowed values for this property are: “COHERE”, “COHEREV2”, “GENERIC”

  • messages (list[oci.generative_ai_inference.models.CohereMessageV2]) – The value to assign to the messages property of this CohereChatRequestV2.

  • documents (list[object]) – The value to assign to the documents property of this CohereChatRequestV2.

  • citation_options (oci.generative_ai_inference.models.CitationOptionsV2) – The value to assign to the citation_options property of this CohereChatRequestV2.

  • tools_choice (str) – The value to assign to the tools_choice property of this CohereChatRequestV2. Allowed values for this property are: “REQUIRED”, “NONE”

  • tools (list[oci.generative_ai_inference.models.CohereToolV2]) – The value to assign to the tools property of this CohereChatRequestV2.

  • is_strict_tools_enabled (bool) – The value to assign to the is_strict_tools_enabled property of this CohereChatRequestV2.

  • is_log_probs_enabled (bool) – The value to assign to the is_log_probs_enabled property of this CohereChatRequestV2.

  • thinking (oci.generative_ai_inference.models.CohereThinkingV2) – The value to assign to the thinking property of this CohereChatRequestV2.

  • response_format (oci.generative_ai_inference.models.CohereResponseFormat) – The value to assign to the response_format property of this CohereChatRequestV2.

  • is_search_queries_only (bool) – The value to assign to the is_search_queries_only property of this CohereChatRequestV2.

  • stream_options (oci.generative_ai_inference.models.StreamOptions) – The value to assign to the stream_options property of this CohereChatRequestV2.

  • is_stream (bool) – The value to assign to the is_stream property of this CohereChatRequestV2.

  • max_tokens (int) – The value to assign to the max_tokens property of this CohereChatRequestV2.

  • temperature (float) – The value to assign to the temperature property of this CohereChatRequestV2.

  • top_k (int) – The value to assign to the top_k property of this CohereChatRequestV2.

  • top_p (float) – The value to assign to the top_p property of this CohereChatRequestV2.

  • frequency_penalty (float) – The value to assign to the frequency_penalty property of this CohereChatRequestV2.

  • presence_penalty (float) – The value to assign to the presence_penalty property of this CohereChatRequestV2.

  • seed (int) – The value to assign to the seed property of this CohereChatRequestV2.

  • stop_sequences (list[str]) – The value to assign to the stop_sequences property of this CohereChatRequestV2.

  • priority (int) – The value to assign to the priority property of this CohereChatRequestV2.

  • is_raw_prompting (bool) – The value to assign to the is_raw_prompting property of this CohereChatRequestV2.

  • safety_mode (str) – The value to assign to the safety_mode property of this CohereChatRequestV2. Allowed values for this property are: “CONTEXTUAL”, “STRICT”, “OFF”

property api_format

[Required] Gets the api_format of this BaseChatRequest. The API format for the model’s family group. COHERE is for the Cohere family models such as the cohere.command-r-16k and cohere.command-r-plus models. GENERIC is for other model families such as the meta.llama-3-70b-instruct model.

Allowed values for this property are: “COHERE”, “COHEREV2”, “GENERIC”

Returns

The api_format of this BaseChatRequest.

Return type

str

property citation_options

Gets the citation_options of this CohereChatRequestV2.

Returns

The citation_options of this CohereChatRequestV2.

Return type

oci.generative_ai_inference.models.CitationOptionsV2

property documents

Gets the documents of this CohereChatRequestV2. A list of relevant documents that the model can refer to for generating grounded responses to the user’s requests. Some example keys that you can add to the dictionary are “text”, “author”, and “date”. Keep the total word count of the strings in the dictionary to 300 words or less.

Returns

The documents of this CohereChatRequestV2.

Return type

list[object]

property frequency_penalty

Gets the frequency_penalty of this CohereChatRequestV2. To reduce repetitiveness of generated tokens, this number penalizes new tokens based on their frequency in the generated text so far. Greater numbers encourage the model to use new tokens, while lower numbers encourage the model to repeat the tokens. Set to 0 to disable.

Returns

The frequency_penalty of this CohereChatRequestV2.

Return type

float

static get_subtype(object_dictionary)

Given the hash representation of a subtype of this class, use the info in the hash to return the class of the subtype.

property is_log_probs_enabled

Gets the is_log_probs_enabled of this CohereChatRequestV2. The log probabilities of the generated tokens will be included in the response.

Returns

The is_log_probs_enabled of this CohereChatRequestV2.

Return type

bool

property is_raw_prompting

Gets the is_raw_prompting of this CohereChatRequestV2. When enabled, the user’s message will be sent to the model without any preprocessing.

Returns

The is_raw_prompting of this CohereChatRequestV2.

Return type

bool

property is_search_queries_only

Gets the is_search_queries_only of this CohereChatRequestV2. When set to true, the response contains only a list of generated search queries without the search results and the model will not respond to the user’s message.

Returns

The is_search_queries_only of this CohereChatRequestV2.

Return type

bool

property is_stream

Gets the is_stream of this CohereChatRequestV2. Whether to stream the partial progress of the model’s response. When set to true, as tokens become available, they are sent as data-only server-sent events.

Returns

The is_stream of this CohereChatRequestV2.

Return type

bool

property is_strict_tools_enabled

Gets the is_strict_tools_enabled of this CohereChatRequestV2. When set to true, tool calls in the Assistant message will be forced to follow the tool definition strictly. Note:The first few requests with a new set of tools will take longer to process.

Returns

The is_strict_tools_enabled of this CohereChatRequestV2.

Return type

bool

property max_tokens

Gets the max_tokens of this CohereChatRequestV2. The maximum number of output tokens that the model will generate for the response. The token count of your prompt plus maxTokens must not exceed the model’s context length. For on-demand inferencing, the response length is capped at 4,000 tokens for each run.

Returns

The max_tokens of this CohereChatRequestV2.

Return type

int

property messages

[Required] Gets the messages of this CohereChatRequestV2. A list of chat messages in chronological order, representing a conversation between the user and the model.

Returns

The messages of this CohereChatRequestV2.

Return type

list[oci.generative_ai_inference.models.CohereMessageV2]

property presence_penalty

Gets the presence_penalty of this CohereChatRequestV2. To reduce repetitiveness of generated tokens, this number penalizes new tokens based on whether they’ve appeared in the generated text so far. Greater numbers encourage the model to use new tokens, while lower numbers encourage the model to repeat the tokens.

Similar to frequency penalty, a penalty is applied to previously present tokens, except that this penalty is applied equally to all tokens that have already appeared, regardless of how many times they’ve appeared. Set to 0 to disable.

Returns

The presence_penalty of this CohereChatRequestV2.

Return type

float

property priority

Gets the priority of this CohereChatRequestV2. The priority of the request (lower means earlier handling; default 0 highest priority). Higher priority requests are handled first, and dropped last when the system is under load.

Returns

The priority of this CohereChatRequestV2.

Return type

int

property response_format

Gets the response_format of this CohereChatRequestV2.

Returns

The response_format of this CohereChatRequestV2.

Return type

oci.generative_ai_inference.models.CohereResponseFormat

property safety_mode

Gets the safety_mode of this CohereChatRequestV2. Safety mode: Adds a safety instruction for the model to use when generating responses. Contextual: (Default) Puts fewer constraints on the output. It maintains core protections by aiming to reject harmful or illegal suggestions, but it allows profanity and some toxic content, sexually explicit and violent content, and content that contains medical, financial, or legal information. Contextual mode is suited for entertainment, creative, or academic use. Strict: Aims to avoid sensitive topics, such as violent or sexual acts and profanity. This mode aims to provide a safer experience by prohibiting responses or recommendations that it finds inappropriate. Strict mode is suited for corporate use, such as for corporate communications and customer service. Off: No safety mode is applied. Note: This parameter is only compatible with models cohere.command-r-08-2024, cohere.command-r-plus-08-2024 and Cohere models released after these models. See release dates.

Allowed values for this property are: “CONTEXTUAL”, “STRICT”, “OFF”

Returns

The safety_mode of this CohereChatRequestV2.

Return type

str

property seed

Gets the seed of this CohereChatRequestV2. If specified, the backend will make a best effort to sample tokens deterministically, so that repeated requests with the same seed and parameters yield the same result. However, determinism cannot be fully guaranteed.

Returns

The seed of this CohereChatRequestV2.

Return type

int

property stop_sequences

Gets the stop_sequences of this CohereChatRequestV2. Stop the model generation when it reaches a stop sequence defined in this parameter.

Returns

The stop_sequences of this CohereChatRequestV2.

Return type

list[str]

property stream_options

Gets the stream_options of this CohereChatRequestV2.

Returns

The stream_options of this CohereChatRequestV2.

Return type

oci.generative_ai_inference.models.StreamOptions

property temperature

Gets the temperature of this CohereChatRequestV2. A number that sets the randomness of the generated output. A lower temperature means less random generations. Use lower numbers for tasks such as question answering or summarizing. High temperatures can generate hallucinations or factually incorrect information. Start with temperatures lower than 1.0 and increase the temperature for more creative outputs, as you regenerate the prompts to refine the outputs.

Returns

The temperature of this CohereChatRequestV2.

Return type

float

property thinking

Gets the thinking of this CohereChatRequestV2.

Returns

The thinking of this CohereChatRequestV2.

Return type

oci.generative_ai_inference.models.CohereThinkingV2

property tools

Gets the tools of this CohereChatRequestV2. A list of available tools (functions) that the model may suggest invoking before producing a text response.

Returns

The tools of this CohereChatRequestV2.

Return type

list[oci.generative_ai_inference.models.CohereToolV2]

property tools_choice

Gets the tools_choice of this CohereChatRequestV2. Used to control whether or not the model will be forced to use a tool when answering. When REQUIRED is specified, the model will be forced to use at least one of the user-defined tools, and the tools parameter must be passed in the request. When NONE is specified, the model will be forced not to use one of the specified tools, and give a direct response. If tool_choice isn’t specified, then the model is free to choose whether to use the specified tools or not. Note:This parameter is only compatible with models Command-r7b and newer.

Allowed values for this property are: “REQUIRED”, “NONE”

Returns

The tools_choice of this CohereChatRequestV2.

Return type

str

property top_k

Gets the top_k of this CohereChatRequestV2. A sampling method in which the model chooses the next token randomly from the top k most likely tokens. A higher value for k generates more random output, which makes the output text sound more natural. The default value for k is 0 which disables this method and considers all tokens. To set a number for the likely tokens, choose an integer between 1 and 500.

If also using top p, then the model considers only the top tokens whose probabilities add up to p percent and ignores the rest of the k tokens. For example, if k is 20 but only the probabilities of the top 10 add up to the value of p, then only the top 10 tokens are chosen.

Returns

The top_k of this CohereChatRequestV2.

Return type

int

property top_p

Gets the top_p of this CohereChatRequestV2. If set to a probability 0.0 < p < 1.0, it ensures that only the most likely tokens, with total probability mass of p, are considered for generation at each step.

To eliminate tokens with low likelihood, assign p a minimum percentage for the next token’s likelihood. For example, when p is set to 0.75, the model eliminates the bottom 25 percent for the next token. Set to 1.0 to consider all tokens and set to 0 to disable. If both k and p are enabled, p acts after k.

Returns

The top_p of this CohereChatRequestV2.

Return type

float