shardpostctl-fts(1)

fts layout3 status

Shows whether the mark is set (when, by whom, for how many pods, whether forced) and every registration with its role, version, capabilities, last heartbeat and freshness. With admin.kv_index other than read it lists the admin namespace. Role: diagnose.

fts layout3 enable

Sets the mark when exactly n fresh registrations of servers and job runners report the capability fts_layout3. Only the processes that write account data under the mark count: servers (role serve, or no role: a registration before 0.47.0) and, since 0.52.0, job runners (role runner); the push workers, scrubs and backfills do not. Servers before 0.45.0 do not register, so the number of pods running now is the check: -pods is required. A pod that crashed less than three minutes ago leaves its fresh record (one that ended in order does not), so the check sees one record too many: wait three minutes, check status, then enable. Enable only after a rollout has completed: a server before 0.47.0 counts the registrations of the push worker, scrubs and backfills as servers. The refusal names the instances and when they were seen. -force skips the check; the audit log and the mark record it. Role: repair.

fts layout3 disable

Removes the mark, waits five seconds (a server that read the mark just before finishes creating its index) and lists the accounts that still have a layout 3 index. While there are any, a downgrade to a version before 0.45.0 is not possible: convert them back first with shardpostctl plan -all -layout 2 fts_layout. Role: repair.

fts dictfilter status

Shows whether the cluster mark of the dictionary filter (fts/dictfilter, since 0.50.0) is set, with the registrations as layout3 status. The mark allows index generations that a version before 0.50.0 must not search: with the identifier filter of the substring dictionary (fts.dict_filter: ids: a body word with letters and digits of at least fts.dict_id_min_runes characters has one start entry instead of a key per trigram, so it is found as a whole word and by its start, not by a part from its inside) and with the text preparation without the signature and trace fields of attached headers (fts.attached_headers: nosig). Without the mark new generations get neither, whatever the configuration says; existing ones change with fts_layout (the filter, see shardpostctl-job(1)) or fts_rebuild (the text preparation). Role: diagnose.

fts dictfilter enable

Sets the mark when exactly n fresh registrations of servers report the capability fts_dict_filter; otherwise as layout3 enable. Role: repair.

fts dictfilter disable

Removes the mark, waits five seconds, then lists the accounts whose entry in the presence index says that a generation of theirs, current or under way, has the filter or the text preparation nosig (set before such a generation is created, cleared after the account switched to one without, kept in step by the scrub; an entry that cannot be read is listed as well). It reads the presence index only, as layout3 disable does. While there are any, a downgrade to a version before 0.50.0 is not possible: convert them with fts_layout -dict-filter none and rebuild those with nosig with fts_rebuild, under settings that ask for neither. Role: repair.

fts online status

Shows whether the cluster mark of online runs (fts/online, since 0.51.0) is set, with the registrations as layout3 status. The mark allows fts_layout and fts_rebuild to run without the account lock: the target generation is built beside the current one while deliveries, logins and sessions go on; writers record what they index meanwhile, the run catches it up and switches in one compare-and-swap. A server before 0.51.0 does not record and waits at the run's freeze (a temporary failure after fts.frozen_wait_seconds), so the mark needs every server to know it. Without the mark every run is offline under the account lock, as before 0.51.0. Role: diagnose.

fts online enable

Sets the mark when exactly n fresh registrations of servers report the capability fts_online_build; otherwise as layout3 enable. Plans with -online are allowed from then on, and with fts.online_build: auto also plans without it (the reshards of the fts_due worker included). Role: repair.

fts online disable

Removes the mark: new plans run offline, a plan with -online is refused, a planned online job that has not started ends without a change (result mark_off; the fts_due worker plans it again, offline). Runs that started finish; before a downgrade to a version before 0.51.0 wait until no account shows next_online in fts layout show (or clear it with fts layout clear). Role: repair.

fts layout3 list

Lists the presence index of the accounts with a layout 3 index (or a conversion to it), a page at a time, with the time each was entered. With -verify each account's layout record is read (its shape shows dict_filter ids/n and attached_headers nosig for a generation that has them), an entry whose account has no layout 3 index is marked MISMATCH, and the largest number of dictionary objects D on the page is printed with its account: before a way back to a version before 0.49.0, fts.shards_max must be at least the largest D of all pages (see shardpostctl-job(1), fts_layout). Export the list before an emergency downgrade. Role: diagnose with the scope of all accounts.

fts layout show

Prints the layout record of an account: the current generation (layout 2 without a record) with its object counts, its identifier filter (dict_filter ids/25) and its text preparation (attached_headers nosig) when it has them, a conversion in progress (next, the token of its run and, for an import reshard, its last heartbeat; next_online yes, mode copy or reindex for an online run, since 0.51.0) and sources not removed yet (old, each with the token of the run that left it and, for an import reshard, its last heartbeat). Role: diagnose.

fts layout clear

Removes the conversion in progress of an account when its run has ended: a job that does not run any more, or an import reshard whose heartbeat is older than 90 seconds. Its freeze is lifted (only the freeze of exactly that run) and its target objects are deleted; a conversion whose run still works is left alone. Sources a run left behind after its switch (it stopped before deleting them) are deleted the same way when that run has ended. For an online run its records in the id map go as well. Deliveries into an account with a stale conversion fail temporarily after fts.frozen_wait_seconds until it is cleared; the job itself, a job or import that takes the account lock and the jobs scrub clear it as well. Role: repair.

fts stats

Prints the key counts of an account's index: its shape (with the identifier filter and the text preparation), the documents, the terms (posting keys) and dictionary keys, in total and per message, and the mean number of dictionary keys per dictionary object. Without options it reads the counters !n and !dn of the posting objects and the document count of a layout 3 index: S + 1 reads, no lock. There are no counters per dictionary or header object, and a layout 2 index has none at all; use -exact.

-exact counts the keys of every object page by page (librados has no key count, and the client library cannot read keys without their values): about one read per thousand keys, 12 000 reads for an account with 12 million index keys, alongside the deliveries, so the counts are only roughly consistent. The values come along, so it transfers about the account's whole index omap, posting lists and header texts included (close to a gigabyte for the largest account on croessner). It runs -parallel objects at a time (one by default, two for a test account) and ends with the 5 minute write timeout of the admin listener; raise -timeout of shardpostctl(1) for a large account.

-classes also counts the terms by field, class (letters, digits, mixed, other) and length, and projects the dictionary keys for every identifier bound n from 13 to 40: what the dictionary would hold if every body term with letters and digits of at least n runes had one key instead of one per trigram. It reads the terms of the posting objects and is allowed only for the test accounts that fts.stats_test_accounts of the server names (for example the bench accounts and a test account filled with shardpost-load); for any other account it is refused (exit status 77). The output never contains a term, on any account. Role: diagnose.