Source instructions
===================

# Introduction

The main orig tarball for this package matches upstream `crowdsec` releases, and
can be downloaded via `uscan`. Upstream code includes a `debian/` directory
(used for their pragmatic packages), that gets overwritten during merges.

In addition to that main tarball, two additional tarballs are used:

 - one for extra data;
 - one for the offline hub.

The idea behind those tarballs is making `crowdsec` useful out of the box,
without having to download anything from the internet (which it can and would do
on its own if that approach hadn't been implemented).


# Tarball creation

## `data` tarball

The `data` tarball ships files that are found in various other locations:

 - text and regex files found in https://github.com/crowdsecurity/sec-lists/
 - IPs found at https://www.cloudflare.com/ips-v4
 - IPs found at https://www.cloudflare.com/ips-v6

Since those updates are infrequent, the tarball must be crafted manually. It can
be carried over from the previous `crowdsec` upstream release if there were no
updates; in that case, it's only a matter of adjusting the version number in the
filename.


## `hub` tarball

The `hub` tarball ships files found in the
[crowdsecurity/hub](https://github.com/crowdsecurity/hub) repository. It can be
built from the `master` when working on release candidates, or from the branch
matching the upstream release once the final release is out (e.g. `v1.4.2`).

There are a few variables and rules in `debian/rules` to automate generating
this tarball as this is much more frequent than updating the `data` tarball:

 - `HUB_ID` is an integer that is used both in the tarball's filename and in the
   hub directory name where its contents are extracted. Use `1` when packaging a
   new upstream release of `crowdsec`, and increment it if the hub needs to be
   updated while `crowdsec` itself stays at the same version. As of December
   2022, there has been no need for such increments, so documentation and/or
   code might be lacking.
 - `HUB_BRANCH` is the name of the branch to generate the export from. As
   mentioned above, `origin/master` is to be used for `crowdsec` release
   candidates, and `origin/vX.Y.Z` can be used instead once the `crowdsec` final
   release is out and the matching branch is available in the `hub` repository.
 - `HUB_DIR` is an absolute or relative path to the `hub` checkout. That
   checkout only needs to be kept current (e.g. via `git fetch`), no local
   branches are needed if the `origin/` prefix is specified in `HUB_BRANCH`.

Once all those variables have been set appropriately, the tarball can be
generated with:

    ./debian/rules generate_hub_tarball

It's best practice to include the output in a `debian/changelog` entry, to keep
track of the upstream commit that was used to generate that export.


Example:

    $ ./debian/rules generate_hub_tarball
    […]
    Generated hub tarball from branch origin/v1.4.2, at commit 73d2edaaf1

can be documented this way:

      * Include a snapshot of hub files from the v1.4.2 branch, at commit
        73d2edaaf1.


# Tarball usage

Given a clean git repository, the following commands can be used to extract data
and hub tarballs into place, creating `dataX/` and `hubY/` directories:

    ./debian/rules extract_data_tarball
    ./debian/rules extract_hub_tarball

It's important to have them in place before trying to apply patches, as some of
them modify hub files; that's needed at least to disable `geoip-enrich` since
the MMDB files can't be distributed.
