Lm.MakeSourceinclude module type of struct include Owl_rllib.Neural.Make(Graph) endinclude module type of struct include Owl_neural_generic.Flatten(Graph) endmodule Graph = Owl_rllib.Neural.Make(Graph).Graphmodule Optimise = Owl_rllib.Neural.Make(Graph).Optimisemodule Algodiff = Owl_rllib.Neural.Make(Graph).Algodiffmodule Init = Owl_rllib.Neural.Make(Graph).Initmodule Activation = Owl_rllib.Neural.Make(Graph).Activationmodule Params = Owl_rllib.Neural.Make(Graph).Paramsmodule Batch = Owl_rllib.Neural.Make(Graph).Batchmodule Learning_Rate = Owl_rllib.Neural.Make(Graph).Learning_Ratemodule Loss = Owl_rllib.Neural.Make(Graph).Lossmodule Gradient = Owl_rllib.Neural.Make(Graph).Gradientmodule Momentum = Owl_rllib.Neural.Make(Graph).Momentummodule Regularisation = Owl_rllib.Neural.Make(Graph).Regularisationmodule Clipping = Owl_rllib.Neural.Make(Graph).Clippingmodule Stopping = Owl_rllib.Neural.Make(Graph).Stoppingmodule Checkpoint = Owl_rllib.Neural.Make(Graph).Checkpointmodule Stepper = Owl_rllib.Neural.Make(Graph).Steppermodule E = Owl_rllib.Neural.Make(Graph).Emodule B = Owl_rllib.Neural.Make(Graph).Bmodule Distribution = Owl_rllib.Neural.Make(Graph).Distributionval scalar : float -> Algodiff.tval column : float array -> Algodiff.tA [n; 1] matrix.
val batch : Algodiff.A.arr array -> Algodiff.tStack observations along a new batch axis.
val one_hot : int -> int array -> Algodiff.tone_hot n actions is a [batch; n] matrix.
val values : Algodiff.A.arr -> float arrayAll elements, in row-major order.
val predict : Graph.network -> Algodiff.A.arr -> float arrayThe network's output for one observation.
val n_actions : (Algodiff.A.arr, int) Owl_rllib.Ndenv.t -> intRaises Invalid_argument unless actions are discrete.
val adam : float -> Params.typAdam with the given learning rate, betas 0.9 and 0.999.
val update : Graph.network -> Stepper.state -> Algodiff.t -> unitOne optimiser step on a loss computed from Graph.forward.
Every constructor seeds Owl's generator with seed before initialising the weights, so a seed always gives the same network.
val mlp :
seed:int ->
shape:int array ->
hidden:int list ->
outputs:int ->
Graph.networkReLU hidden layers and a linear output, with Owl's default initialisation.
type init = Owl_rllib.Neural.Make(Graph).init = val orthogonal : float -> Init.typOrthogonal weights scaled by a gain, like PyTorch's orthogonal_.
val mlp_with :
activation:Activation.typ ->
init:init ->
seed:int ->
shape:int array ->
hidden:int list ->
outputs:int ->
Graph.networkval gaussian_mlp :
activation:Activation.typ ->
init:init ->
seed:int ->
shape:int array ->
hidden:int list ->
actions:int ->
Graph.networkA network for Owl_rllib.Policy.Make.gaussian whose log_std does not depend on the state: one learned vector, initially 0.
val cnn :
seed:int ->
shape:int array ->
convs:(int * int * int) list ->
hidden:int list ->
outputs:int ->
Graph.networkFor NHWC images [height; width; channels]: each (kernel, stride, channels) is a VALID ReLU convolution, followed by ReLU hidden layers on the flattened features and a linear output.
Context length: the longest prompt plus every answer token but the last.
A context as its left-padded one-hot window, row-major [position; token].
Encode a batch of contexts.
Hidden layers default to [128; 128]; outputs (default vocab) can be 1 for a critic.
log pi(action | context) for every row, differentiable in the logits.
type sample = {prompt : int array;tokens : int array;generated tokens, ending with eos unless cut off
log_probs : float array;log pi(token) under the generating network
}val generate :
?greedy:bool ->
?max_len:int ->
rng:Random.State.t ->
Graph.network ->
int array array ->
sample arrayGenerate up to max_len tokens for every prompt (default Owl_rllib_llm.Lm_task.max_answer), one batched forward pass per position; sampling by default, the most likely token with ~greedy:true.
For (prompt, tokens) pairs, one row per token: the context before it, the token, and the index of its pair.
val accuracy :
?sample:bool ->
?task:Lm_task.task ->
Graph.network ->
Lm_task.problem array ->
floatThe fraction of problems answered right, with greedy decoding or one sample per problem; the sampling stream is fixed, so evaluations repeat.
val sft :
?batch_size:int ->
?learning_rate:float ->
?report:(int -> float -> unit) ->
?task:Lm_task.task ->
steps:int ->
rng:Random.State.t ->
Graph.network ->
Lm_task.problem array ->
unitSupervised fine-tuning with teacher forcing: minimise the cross-entropy of every target token given the tokens before it. report step loss is called after every step.