When setting up an Irmin database in OCaml you will need to consider, at least, the content type and storage backend. This is because Irmin has the ability to adapt to existing data structures using a convenient type combinator (Irmin.Type), which is used to define contents for your datastore. Irmin provides implementations for [String][irmin.contents.string], [Cstruct][irmin.contents.cstruct], Json and Json_value contents, but it is also very easy to make your own!

Irmin gives you a few options when it comes to storage:

• an in-memory store (irmin-mem)
• a filesystem store (irmin-fs)
• git-compatible filesystem/in-memory stores (irmin-git)

These packages define the way that the data should be organized, but not any I/O routines (with the exception of irmin-mem, which does no I/O). Luckily, irmin-unix implements the I/O routines needed to make Irmin work on Unix-like platforms. Additionally, the irmin-mirage, irmin-mirage-git and irmin-mirage-graphql packages provide Mirage-compatible interfaces.

It's also possible to implement your own storage backend if you'd like -- nearly everything in Irmin is configurable thanks to the power of functors in OCaml! This includes the hash function, branch, key and metadata types. Because of this flexibility there are a lot of different options to pick from; I will do my best to explain the most basic usage in this section and begin introducing more advanced concepts in subsequent sections.

It is important to note that most Irmin functions return Lwt.t values, which means that you will need to use Lwt_main.run to execute them. If you're not familiar with Lwt then I suggest this tutorial.

Creating a store

An in-memory store with string contents:

module Mem_store = Irmin_mem.KV(Irmin.Contents.String)

An on-disk git store with JSON contents:

module Git_store = Irmin_unix.Git.FS.KV(Irmin.Contents.Json)

These examples are using Irmin.KV, which is a specialization of Irmin.S with string list keys, string branches and no metadata.

The following example is the same as the first, using Irmin_mem.Make instead of Irmin_mem.KV:

module Mem_Store =
    Irmin_mem.Make
        (Irmin.Metadata.None)
        (Irmin.Contents.Json)
        (Irmin.Path.String_list)
        (Irmin.Branch.String)
        (Irmin.Hash.SHA1)

Configuring and creating a repo

Different store types require different configuration options -- an on-disk store needs to know where it should be stored in the filesystem, however an in-memory store doesn't. This means that each storage backend implements its own configuration methods based on Irmin.Private.Conf -- for the examples above there are Irmin_mem.config, Irmin_fs.config and Irmin_git.config, each taking slightly different parameters.

let git_config = Irmin_git.config ~bare:true "/tmp/irmin"

let config = Irmin_mem.config ()

With this configuration it's very easy to create an Irmin.Repo using Repo.v:

let git_repo = Git_store.Repo.v git_config

let repo = Mem_store.Repo.v config

Using the repo to obtain access to a branch

Once a repo has been created, you can access a branch and start to modify it.

To get access to the master branch:

open Lwt.Infix

let master config =
    Mem_store.Repo.v config >>= Mem_store.master

To get access to a named branch:

let branch config name =
    Mem_store.Repo.v config >>= fun repo ->
    Mem_store.of_branch repo name

Modifying the store

Now you can begin to interact with the store using get and set.

let info message = Irmin_unix.info ~author:"Example" "%s" message

let main =
    Mem_store.Repo.v config >>= Mem_store.master >>= fun t ->
    (* Set a/b/c to "Hello, Irmin!" *)
    Mem_store.set_exn t ["a"; "b"; "c"] "Hello, Irmin!" ~info:(info "my first commit") >>= fun () ->
    (* Get a/b/c *)
    Mem_store.get t ["a"; "b"; "c"] >|= fun s ->
    assert (s = "Hello, Irmin!")
let () = Lwt_main.run main

Transactions

Transactions allow you to make many modifications using an in-memory tree then apply them all at once. This is done using with_tree:

let transaction_example =
Mem_store.Repo.v config >>= Mem_store.master >>= fun t ->
let info = Irmin_unix.info "example transaction" in
Mem_store.with_tree_exn t [] ~info ~strategy:`Set (fun tree ->
    let tree = match tree with Some t -> t | None -> Mem_store.Tree.empty in
    Mem_store.Tree.remove tree ["foo"; "bar"] >>= fun tree ->
    Mem_store.Tree.add tree ["a"; "b"; "c"] "123" >>= fun tree ->
    Mem_store.Tree.add tree ["d"; "e"; "f"] "456" >>= Lwt.return_some)
let () = Lwt_main.run transaction_example

A tree can be modified using the functions in Irmin.S.Tree, and when it is returned by the with_tree callback, it will be applied using the transaction's strategy (`Set in the code above) at the given key ([] in the code above).

Here is an example move function to move files from one path to another:

let move t ~src ~dest =
    Mem_store.with_tree_exn t Mem_store.Key.empty ~strategy:`Set (fun tree ->
        match tree with
        | Some tr ->
            Mem_store.Tree.get_tree tr src >>= fun v ->
            Mem_store.Tree.remove tr src >>= fun _ ->
            Mem_store.Tree.add_tree tr dest v >>= Lwt.return_some
        | None -> Lwt.return_none
    )
let main =
    Mem_store.Repo.v config >>= Mem_store.master >>= fun t ->
    let info = Irmin_unix.info "move a -> foo" in
    move t ~src:["a"] ~dest:["foo"] ~info
let () = Lwt_main.run main

Sync

Irmin.Sync implements the functions needed to interact with remote stores.

• fetch populates a local store with objects from a remote store
• pull updates a local store with objects from a remote store
• push updates a remote store with objects from a local store

Each of these also has an _exn variant which may raise an exception instead of returning result value.

For example, you can pull a repo and list the files in the root of the project:

open Irmin_unix
module Git_mem_store = Git.Mem.KV(Irmin.Contents.String)
module Sync = Irmin.Sync(Git_mem_store)
let remote = Git_mem_store.remote "git://github.com/mirage/irmin.git"
let main =
    Git_mem_store.Repo.v config >>= Git_mem_store.master >>= fun t ->
    Sync.pull_exn t remote `Set >>= fun _ ->
    Git_mem_store.list t [] >|= List.iter (fun (step, kind) ->
        match kind with
        | `Contents -> Printf.printf "FILE %s\n" step
        | `Node -> Printf.printf "DIR %s\n" step
    )
let () = Lwt_main.run main

JSON Contents

Most examples in this tutorial use string contents, so I will provide some further information about using JSON values.

There are two types of JSON contents: Json_value and Json, where Json can only store JSON objects and Json_value works with any JSON value.

Setting up the store is exactly the same as when working with strings:

module Mem_store_json = Irmin_mem.KV(Irmin.Contents.Json)
module Mem_store_json_value = Irmin_mem.KV(Irmin.Contents.Json_value)

For example, using Men_store_json_value we can assign {"x": 1, "y": 2, "z": 3} to the key a/b/c:

let main =
    let module Store = Mem_store_json_value in
    Store.Repo.v config >>= Store.master >>= fun t ->
    let value = `O ["x", `Float 1.; "y", `Float 2.; "z", `Float 3.] in
    Store.set_exn t ["a"; "b"; "c"] value ~info:(info "set a/b/c") >>= fun () ->
    Store.get t ["a"; "b"; "c"] >|= fun x ->
    assert (Irmin.Type.equal Store.contents_t value x)
let () = Lwt_main.run main

An interesting thing about Json_value stores is the ability to use Json_tree to recursively project values onto a key. This means that using Json_tree.set, to assign {"test": {"foo": "bar"}, "x": 1, "y": 2, "z": 3} to the key a/b/c will set a/b/c/x to 1, a/b/c/y to 2, a/b/c/z to 3 and a/b/c/test/foo to "bar".

This allows for large JSON objects to be modified in pieces without having the encode/decode the entire thing to access specific fields. Using Json_tree.get we can also retrieve a tree as a JSON value. So if we call Json_tree.get with the key a/b we will get the following object back: {"c": {"test": {"foo": "bar"}, "x": 1, "y": 2, "z": 3}}.

let main =
    let module Store = Mem_store_json_value in
    let module Proj = Irmin.Json_tree(Store) in
    Store.Repo.v config >>= Store.master >>= fun t ->
    let value = `O ["test", `O ["foo", `String "bar"]; "x", `Float 1.; "y", `Float 2.; "z", `Float 3.] in
    Proj.set t ["a"; "b"; "c"] value ~info:(info "set a/b/c") >>= fun () ->
    Store.get t ["a"; "b"; "c"; "x"] >>= fun x ->
    assert (Irmin.Type.equal Store.contents_t (`Float 1.) x);
    Store.get t ["a"; "b"; "c"; "test"; "foo"] >>= fun x ->
    assert (Irmin.Type.equal Store.contents_t (`String "bar") x);
    Proj.get t ["a"; "b"] >|= fun x ->
    assert (Irmin.Type.equal Store.contents_t (`O ["c", value]) x)
let () = Lwt_main.run main