Values.CountTokensRequestSourceReturns the token count for a given inference request. This operation helps you estimate token usage before sending requests to foundation models by returning the token count that would be used if the same input were sent to the model in an inference request. Token counting is model-specific because different models use different tokenization strategies. The token count returned by this operation will match the token count that would be charged if the same input were sent to the model in an InvokeModel or Converse request. You can use this operation to: Estimate costs before sending inference requests. Optimize prompts to fit within token limits. Plan for token usage in your applications. This operation accepts the same input formats as InvokeModel and Converse, allowing you to count tokens for both raw text inputs and structured conversation formats. The following operations are related to CountTokens: InvokeModel - Sends inference requests to foundation models Converse - Sends conversation-based inference requests to foundation models
type nonrec t = {modelId : FoundationModelVersionIdentifier.t;The unique identifier or ARN of the foundation model to use for token counting. Each model processes tokens differently, so the token count is specific to the model you specify.
*)input : CountTokensInput.t;The input for which to count tokens. The structure of this parameter depends on whether you're counting tokens for an InvokeModel or Converse request: For InvokeModel requests, provide the request body in the invokeModel field For Converse requests, provide the messages and system content in the converse field The input format must be compatible with the model specified in the modelId parameter.
*)}val make :
modelId:FoundationModelVersionIdentifier.t ->
input:CountTokensInput.t ->
unit ->
tval to_value :
t ->
[> `Structure of
(string
* [> `String of FoundationModelVersionIdentifier.t
| `Structure of
(string
* [> `Structure of
(string
* [> `Blob of Body.t
| `List of
[> `Structure of
(string
* [> `Enum of string
| `List of
[> `Structure of
(string
* [> `String of String_.t
| `Structure of
(string
* [> `Blob of Blob.t
| `Enum of string
| `List of
[> `Structure of
(string
* [> `List of
[> `Structure of
(string
*
[>
`String of
String_.t ])
list ]
list
| `String of
String_.t
| `Structure of
(string
* [> `Enum of
string
| `List of
[> `Structure of
(string
*
[>
`String of
String_.t ])
list ]
list
| `String of
DocumentBlockNameString.t
| `Structure of
(string
* [>
`Blob of
ImageSourceBytesBlob.t
| `Boolean of
Boolean.t
| `Integer of
DocumentCharLocationDocumentIndexInteger.t
| `List of
[>
`Structure of
(string
*
[>
`String of
String_.t ])
list ]
list
| `String of
String_.t
| `Structure of
(string
*
[>
`String of
S3Uri.t ])
list ])
list ])
list ])
list ]
list
| `String of
DocumentBlockNameString.t
| `Structure of
(string
* [> `Blob of
ImageSourceBytesBlob.t
| `Boolean of Boolean.t
| `Enum of string
| `List of
[> `Enum of string
| `Structure of
(string
* [> `String of
String_.t ])
list ]
list
| `String of String_.t
| `Structure of
(string
* [> `Blob of
GuardrailConverseImageSourceBytesBlob.t
| `String of S3Uri.t ])
list ])
list ])
list ])
list ]
list
| `String of NonEmptyString.t
| `Structure of
(string
* [> `Enum of string
| `Structure of
(string
* [> `Enum of string
| `List of [> `Enum of string ] list
| `String of String_.t
| `Structure of
(string
* [> `Blob of
GuardrailConverseImageSourceBytesBlob.t ])
list ])
list ])
list ])
list ]
list
| `Structure of
(string
* [> `List of
[> `Structure of
(string
* [> `Structure of
(string
* [> `Boolean of Boolean.t
| `Enum of string
| `String of ToolName.t
| `Structure of
(string
* [> `Structure of 'a list ])
list ])
list ])
list ]
list
| `Structure of
(string
* [> `Structure of
(string * [> `String of ToolName.t ])
list ])
list ])
list ])
list ])
list ])
list ]