Values.AutomatedEvaluationConfigSourceThe configuration details of an automated evaluation job. The EvaluationDatasetMetricConfig object is used to specify the prompt datasets, task type, and metric names.
type nonrec t = {datasetMetricConfigs : EvaluationDatasetMetricConfigs.t;Configuration details of the prompt datasets and metrics you want to use for your evaluation job.
*)evaluatorModelConfig : EvaluatorModelConfig.t option;Contains the evaluator model configuration details. EvaluatorModelConfig is required for evaluation jobs that use a knowledge base or in model evaluation job that use a model as judge. This model computes all evaluation related metrics.
*)customMetricConfig : AutomatedEvaluationCustomMetricConfig.t option;Defines the configuration of custom metrics to be used in an evaluation job.
*)}val make :
?evaluatorModelConfig:??? ->
?customMetricConfig:??? ->
datasetMetricConfigs:EvaluationDatasetMetricConfigs.t ->
unit ->
tval to_value :
t ->
[> `Structure of
(string
* [> `List of
[> `Structure of
(string
* [> `Enum of string
| `List of [> `String of EvaluationMetricName.t ] list
| `Structure of
(string
* [> `String of EvaluationDatasetName.t
| `Structure of
(string * [> `String of S3Uri.t ]) list ])
list ])
list ]
list
| `Structure of
(string
* [> `List of
[> `Structure of
(string
* [> `String of EvaluatorModelIdentifier.t
| `Structure of
(string
* [> `List of
[> `Structure of
(string
* [> `String of
RatingScaleItemDefinition.t
| `Structure of
(string
* [> `Float of Float_.t
| `String of
RatingScaleItemValueStringValueString.t ])
list ])
list ]
list
| `String of MetricName.t ])
list ])
list ]
list
| `Structure of
(string
* [> `List of
[> `Structure of
(string
* [> `String of EvaluatorModelIdentifier.t ])
list ]
list ])
list ])
list ])
list ]