Module Plebeia.StorageSource

Generic cell based storage

Sourcetype mode =
  1. | Private
  2. | Reader
  3. | Writer

Reader: Read only mode. Multiple readers can exist.

Writer: Read write mode. Only 1 writer can exist for a storage.

Private : Modifications are private. See MAP_PRIVATE of the document of mmap()

Sourceval is_shared : mode -> bool
Sourcetype config = {
  1. head_string : string;
    (*

    exactly 20 bytes

    *)
  2. version : int;
  3. bytes_per_cell : int;
  4. max_index : Index.t;
}
Sourcetype storage
Sourcetype t = storage

The type

2 Open, close, and commit

Sourceval create : ?length:Index.t -> ?resize_step_bytes:int -> config:config -> string -> t Lwt.t

Create a new storage

* length: The initial size of the storage in cells, excluding pos. * resize_step: How many cells allocated for each resize * version: Default is Verison.version * string : The path name

Sourceval open_ : ?resize_step_bytes:int -> mode:mode -> string -> (config * t) option Lwt.t

Open an existing storage

Sourceval open_or_create : ?length:Index.t -> ?resize_step_bytes:int -> config:config -> string -> t Lwt.t
Sourceval null : t

Dummy storage

Sourceval is_null : t -> bool
Sourceval reopen : t -> unit Lwt.t
Sourceval truncate : ?length:int -> t -> unit Lwt.t

Truncate the data file and reinitialize it. All the contents are lost. length is the initial reserved size of the reinitialized file.

Sourceval update_reader : t -> unit Lwt.t

For reader to update the storage to catch up the update by the writer process. For Writer and Private, it does nothing.

Sourceval close : t -> unit Lwt.t

Close the storage. For non Reader, it calls commit before closing.

Sourceval commit : t -> unit Lwt.t

Write the current state of the storage. Once called, the state is shared with the reader processes with process sync enabled.

Note that commit does NOT assure the persistence of the data after a system crash. Use flush to make it persistent even after a crash.

Sourceval flush : t -> unit Lwt.t

Flushes all the added data to the disk. The data added to the storage before the call of flush are persisted even after a system crash. Data added after the last call of flush may be lost after a system crash.

flush may trigger huge disk writes. Too frequent call of flush may degrates the performance.

Sourceval enable_process_sync : t -> unit

For readers. Enable the process synchronization between the writer process. Once enabled, update_reader checks the header for the process sync instead of the disk sync.

Do not call this function before the writer calls commit. Otherwise readers can misuse an invalid header value.

2 Accessor

Sourceval filename : t -> string

Return the file name

Sourceval mode : t -> mode

Return the opening mode

Sourceval get_last_root_index : t -> Index.t option
Sourceval get_current_length : t -> Index.t

Get the status of the storage

For Reader, it only returns the lastest information it knows in memory. Writer may already update this information on the disk.

Sourceval size : t -> int64

Used cells in bytes

Sourceval version : t -> int

Storage version

Sourceval start_index : t -> Index.t
Sourceval override_version : t -> int -> unit

Override the header version of the file

Sourceval set_last_root_index : t -> Index.t option -> unit

Set the last index of root hash. Note that the data are only saved to the file when Header.commit is called.

2 Read and write

Sourceval get_cell : t -> Index.t -> Mmap.Buffer.t

Get the content of the cell specified by the index

Sourceval get_cell2 : t -> Index.t -> Mmap.Buffer.t

make a 2-cell-wide writable buffer from the beginning of the cell of the given index

Sourceval get_bytes : t -> Index.t -> int -> Mmap.Buffer.t

Get the contiguous bytes from the head of the index

Sourcetype Error.t +=
  1. | Index_overflow of t

Error when Index.t exceeds the max_index or overflows

Sourceval new_index : t -> (Index.t, Error.t) result

Allocate a cell and returns its index

Sourceval new_indices : t -> int -> (Index.t, Error.t) result

Allocate cells and return the first index

Sourcemodule Chunk : sig ... end

Bigger data than a cell

Sourcemodule Internal : sig ... end