Get started with ShardPost

ShardPost is one layer of a mail system: it needs Ceph underneath, an authentication backend and an MTA in front. This page puts the first steps in order and links to the manual for each of them. It describes ShardPost 0.52.0.

What you need

• A Ceph cluster. ShardPost talks to RADOS through librados; each server host needs librados2 in the version of the cluster and network access to the monitors and every OSD. The image builds against Ceph 20 (Tentacle).
• Linux. In containers or natively. ShardPost needs a Ceph cluster, not Kubernetes and not Rook; the Kubernetes API is used only to read the cephx key and the monitor list from Rook when no key is configured.
• Authentication. In the operational profile Nauthilus handles logins and resolves addresses to the canonical account, over gRPC or its HTTP JSON API. A static backend with a user list exists for tests.
• An MTA in front. ShardPost accepts no internet SMTP. An MTA such as Postfix delivers to it over LMTP, and a configured SMTP relay sends what ShardPost generates (Sieve redirect and vacation, JMAP submission).
• Optional: nauthilus-director as the public front door, a classifier such as Rspamd for learning, an OpenTelemetry collector and Prometheus.

Source and release image

The source, the deployment files and the documentation are in one repository:

git clone https://github.com/croessner/shardpost.git

Release images are published as docker.io/chrroessner/shardpost for linux/amd64, tagged with the version (for example 0.52.0). The image contains the server shardpost, the admin client shardpostctl and the external programs.

To build an image yourself and check only its configuration:

docker build --platform linux/amd64 -t shardpost:dev .
docker run --rm -e LOG_LEVEL=info shardpost:dev config check

config check validates the configuration and the secret files, but connects neither to Ceph nor to Nauthilus. Building the server natively needs Go as named in go.mod, gcc and the librados development files; shardpostctl and shardpost-load build without cgo. The developer guide has the details.

Prepare the Ceph pools

The index pool must be replicated, because the mailbox state lives in omap. The data pool holds write-once blobs and can be erasure coded. The system pool is optional and replicated. The example from the operations guide:

ceph osd pool create shardpost-index replicated
ceph osd pool set shardpost-index size 3
ceph osd erasure-code-profile set shardpost-ec21 k=2 m=1 crush-failure-domain=host
ceph osd pool create shardpost-data erasure shardpost-ec21
ceph osd pool create shardpost-system replicated
ceph osd pool set shardpost-system size 3
for p in shardpost-index shardpost-data shardpost-system; do
  ceph osd pool application enable "$p" shardpost
  ceph osd pool set "$p" nodelete true
done

The cephx identity needs read access to the monitors and rwx on the pools; x is required for object classes:

ceph auth get-or-create client.shardpost \
  mon 'allow r' \
  osd 'allow rwx pool=shardpost-index, allow rwx pool=shardpost-data, allow rwx pool=shardpost-system'

A new cluster with a system pool is prepared with shardpost system init; see shardpost-system(1). The storage guide explains why there are three pools, and Pools how to size them.

Kubernetes with Rook

The repository carries Kustomize manifests under deploy/, among them a base with the server deployment, service, PodDisruptionBudget, scrub CronJob and the Rook pools and cephx client, components for Nauthilus, admin mTLS, the import sidecar, JMAP push workers and job runners, and the overlays of the reference installation.

Read Operations and releases before the first rollout: dependencies, a read-only first diagnosis, the release procedure and rollback.

Natively with systemd

Without Kubernetes, the cephx key goes into the configuration (as CEPH_KEY from an environment file readable only by root), and the pools are named in the ceph section:

ceph:
  user: shardpost
  mon_host: "[v2:192.0.2.11:3300],[v2:192.0.2.12:3300],[v2:192.0.2.13:3300]"
  index_pool: shardpost-index
  data_pool: shardpost-data
  system_pool: shardpost-system   # the same on every node and process

Check it before the first start:

sudo -u shardpost shardpost --config /etc/shardpost/shardpost.yaml config check

Several native servers coordinate through RADOS alone and can share one pool set like pods do. Running without Kubernetes has the host requirements, a complete systemd unit and the push workers and job runners.

Configure

ShardPost reads a YAML file (--config or SHARDPOST_CONFIG), environment variables and secret files. The configuration reference covers every section; the most important ones for a first installation:

• ceph: pools, user, monitors and chunk size.
• server: listeners, TLS, the auth backend, trusted PROXY networks, quota defaults.
• nauthilus and auth: the backchannel to Nauthilus and the auth cache.
• jmap: the JMAP listener, its public base URL, submission, push and limits.
• sieve, imapsieve and extprograms: scripts, learning and the allowlisted programs.

Next steps

• Administer: the admin API and shardpostctl(1) with its roles.
• Move accounts in: shardpostctl-migrate(1) keeps UIDs, UIDVALIDITY and MODSEQ.
• Watch it: metrics, traces and logs, and the monitoring chapter of the storage guide.
• Turn on JMAP: JMAP and JMAP by example; JMapper Mail is a client built for it.