Lm.DSourcemodule Graph : sig ... endmodule Optimise = Graph.Neuron.Optimisemodule Algodiff = Graph.Neuron.Optimise.Algodiffmodule Init = Graph.Neuron.Initmodule Activation = Graph.Neuron.Activationmodule Params = Graph.Neuron.Optimise.Paramsmodule Batch = Graph.Neuron.Optimise.Batchmodule Learning_Rate = Graph.Neuron.Optimise.Learning_Ratemodule Loss = Graph.Neuron.Optimise.Lossmodule Gradient = Graph.Neuron.Optimise.Gradientmodule Momentum = Graph.Neuron.Optimise.Momentummodule Regularisation = Graph.Neuron.Optimise.Regularisationmodule Clipping = Graph.Neuron.Optimise.Clippingmodule Stopping = Graph.Neuron.Optimise.Stoppingmodule Checkpoint = Graph.Neuron.Optimise.Checkpointmodule Stepper = Graph.Neuron.Optimise.Steppermodule E : sig ... endmodule B : sig ... endmodule Distribution : sig ... endval scalar : float -> Algodiff.tval column : float array -> Algodiff.tval batch : Algodiff.A.arr array -> Algodiff.tval one_hot : int -> int array -> Algodiff.tval values : Algodiff.A.arr -> float arrayval predict : Graph.network -> Algodiff.A.arr -> float arrayval n_actions : (Algodiff.A.arr, int) Owl_rllib.Ndenv.t -> intval adam : float -> Params.typval update : Graph.network -> Stepper.state -> Algodiff.t -> unitval mlp :
seed:int ->
shape:int array ->
hidden:int list ->
outputs:int ->
Graph.networkval orthogonal : float -> Init.typval mlp_with :
activation:Activation.typ ->
init:init ->
seed:int ->
shape:int array ->
hidden:int list ->
outputs:int ->
Graph.networkval gaussian_mlp :
activation:Activation.typ ->
init:init ->
seed:int ->
shape:int array ->
hidden:int list ->
actions:int ->
Graph.networkval cnn :
seed:int ->
shape:int array ->
convs:(int * int * int) list ->
hidden:int list ->
outputs:int ->
Graph.networkContext length: the longest prompt plus every answer token but the last.
A context as its left-padded one-hot window, row-major [position; token].
Encode a batch of contexts.
Hidden layers default to [128; 128]; outputs (default vocab) can be 1 for a critic.
log pi(action | context) for every row, differentiable in the logits.
type sample = Make(Owl.Neural.D.Graph).sample = {prompt : int array;tokens : int array;generated tokens, ending with eos unless cut off
log_probs : float array;log pi(token) under the generating network
}val generate :
?greedy:??? ->
?max_len:??? ->
rng:Random.State.t ->
Graph.network ->
int array array ->
sample arrayGenerate up to max_len tokens for every prompt (default Owl_rllib_llm.Lm_task.max_answer), one batched forward pass per position; sampling by default, the most likely token with ~greedy:true.
For (prompt, tokens) pairs, one row per token: the context before it, the token, and the index of its pair.
The fraction of problems answered right, with greedy decoding or one sample per problem; the sampling stream is fixed, so evaluations repeat.
val sft :
?batch_size:??? ->
?learning_rate:??? ->
?report:??? ->
?task:??? ->
steps:int ->
rng:Random.State.t ->
Graph.network ->
Lm_task.problem array ->
unitSupervised fine-tuning with teacher forcing: minimise the cross-entropy of every target token given the tokens before it. report step loss is called after every step.