# Hacking on Dune

This section is for people who want to work on Dune itself.

## Setup

### Create an Opam switch with the required dependencies

From the project root, execute

```sh
$ make dev-switch
# This takes some time
```

## Building

### Common usage

To build and install the development version, run the following while in the
project's root directory:

```sh
$ make dev
```

Note that this will build the `dune` binary twice: once using the
bootstrap build procedure and then one normal dune build using the
freshly produced `dune binary`.

For a short synopsis on the usual `make` commands used in development, run

```sh
$ make
# or
$ make help
```

### A note on bootstrapping

In order to build itself, Dune uses a micro dune written as a single
[boot/duneboot.ml](boot/duneboot.ml) file. This micro build system
cannot read `dune` files and instead has the configuration hard-coded
in [boot/libs.ml](boot/libs.ml). This latter file is automatically
updated during development when we modify the `dune` files in the
repository. [boot/duneboot.ml](boot/duneboot.ml) itself is built with
a single invocation of `ocamlopt` or `ocamlc` via the
[bootstrap.ml](bootstrap.ml) ocaml script.

[boot/duneboot.ml](boot/duneboot.ml) builds a `dune.exe` binary at the
root of the source tree and uses this binary to build everything else.

`make dev` takes care of bootstrapping if needed, but if you want to just run
the bootstrapping step itself, build the `dune.exe` target with

```sh
make dune.exe
```

## OCaml compatibility test

Install opam switches for all the entries in the
[dune-workspace.dev](dune-workspace.dev) file and run:

```sh
$ make all-supported-ocaml-versions
```

## Repository organization

- `vendor/` contains dependencies of Dune, that have been vendored
- `plugin/` contains the API given to `dune` files that are OCaml
  scripts
- `src/` contains the core of Dune
- `bin/` contains the command line interface
- `doc/` contains the manual and rules to generate the manual pages

## Design

Dune (nee "JBuilder") was initially designed to sort out the public release of
Jane Street packages which became incredibly complicated over time. It is still
successfully used for this purpose.

One necessary feature to achieve this is the ability to precisely
report the external dependencies necessary to build a given set of
targets without running any command, just by looking at the source
tree. This is used to automatically generate the `<package>.opam`
files for all Jane Street packages.

To implement this, the build rules are described using a build data type,
which is defined in [src/dune/build.mli](src/dune/build.mli). In the end it
makes the development of the internal rules of Dune very composable and
quite pleasant.

To deal with process multiplexing, Dune uses a simplified
Lwt/Async-like monad, implemented in [src/fiber/fiber.mli](src/fiber/fiber.mli).

## Tests

Dune uses [cram style](https://blog.janestreet.com/testing-with-expectations/)
tests for its test suite. The test suite is contained in
[test/blackbox-tests](test/blackbox-tests). A single test consists of a test
directory in the test-cases/ sub directory which contains a run.t file defining
the test.

An example `run.t` file:

```
A description of the test. The command running the tests is preceeded by
two spaces and a $. The expected output of the command is also indented by
two spaces and is right below the command (note the absence of a $)

  $ echo "the expected output is below"
  the expected output is below
```

Running the entire test suite is done with `$ make test`. A particular test can
be executed with `$ dune build @<test-name>`.

Running the test will execute the command after the `$` and its output will be
compared against the expected output right below the command. Any differences
will result in a test failure.

### Adding a Test

Simply add a new directory in test/blackbox-tests/test-cases and then `$ make`
generate the rules for the test , followed by `$ make promote` to accept the new
rules.

### Accepting Corrections

A failing expect test will generate a diff between the expected and actual
output. If the new output generated by the command is satisfactory, it can be
*promoted* with the `$ make promote` command.

## Documentation

### User documentation

User documentation lives in the [./doc](./doc) directory.

In order to build the user documentation, you must install

- [`python-sphinx`][python-sphinx], and
- [`sphinx_rtd_theme`][sphinx_rtd_theme]

Build the documentation with

```sh
make doc
```

For automatically updated builds, you can [install
`sphinx-autobuild`][sphinx-autobuild], and run

```sh
make livedoc
```

[python-sphinx]: http://www.sphinx-doc.org/en/master/usage/installation.html
[sphinx_rtd_theme]: https://sphinx-rtd-theme.readthedocs.io/en/stable/
[sphinx-autobuild]: https://pypi.org/project/sphinx-autobuild/

## Code flow

- [src/dune/dune_file.mli](src/dune/dune_file.mli) contains the internal
  representation of `dune` files and the parsing code
- [src/dune/dune_load.mli](src/dune/dune_load.mli) contains the code to scan
  a source tree and build the internal database by reading
  the `dune` files
- [src/dune/gen_rules.mli](src/dune/gen_rules.mli) contains all the build rules
  of Dune
- [src/dune/build_system.mli](src/dune/build_system.mli) contains a trivial
  implementation of a Build system. This is what Jenga will provide
  when implementing the bridge
