arch idx: Maintain Document Indexes#

arch treats an idx as an address attached to a document structure, not as decorative text. The same scanner produces immutable plans for the command line and terminal interface. A check, diff, JSON record, TUI preview, and applied file therefore describe the same proposed edits.

1 Quick start#

arch check README.md docs/
arch plan --diff docs/
arch apply --dry-run --diff docs/
arch apply docs/

check and plan exit 1 when they find work and 0 when clean. apply exits 0 after a successful run, whether or not bytes changed. Its JSON result and stderr summary carry the change count. Other exits are stable:

Code

Meaning

0

clean check or successful operation

1

findings or a non-empty plan

2

command usage or invalid idx input

3

invalid configuration

4

I/O or decoding failure

5

stale plan; the file changed before application

Use --json before the subcommand for an agent-stable versioned payload:

arch --json check src docs
arch --json idx parse 1.2

Diagnostics in human mode go to stderr. Diffs and JSON go to stdout, so either can be piped without scraping progress prose.

2 Individual idx operations#

arch idx parse 1.2
arch idx build 0 1       # zero-based places -> 1.2 under the default spec
arch --config project.yaml idx parse --spec doctrine 1.a.iii.

The Python API additionally exposes Idx.build_relative() for parent, previous-sibling, absolute-depth, relative-depth, absolute-place, and relative-place construction. Relations compare normalized places rather than rendering styles. Compact indexes must have a unique tokenization. If adjacent variable-width places make text such as 12 ambiguous, parsing fails closed. Choose a separator or set max_depth to state the intended grammar explicitly.

arch.idx.Idx is the definitive arbitrary-depth document-index model. It is separate from arch.Quad, the preferred name for the fixed four-place architectonic address. arch.Idx remains available as the Quad compatibility spelling. The models are intentionally not related by subclassing: a Quad admits the root and abstract a/b halves, while a document idx is a non-empty sequence with prefix-style hierarchy and a separately configurable rendering policy.

3 Configuration#

Layers are merged from low to high precedence:

  1. built-ins;

  2. ~/.config/my/arch.yaml (or $XDG_CONFIG_HOME/my/arch.yaml);

  3. the nearest .arch.yaml, stopping at the repository root;

  4. an explicit --config file;

  5. command flags.

Named index specs and named header profiles merge by name, but their path selectors are independent. Higher-layer rules are tried before lower-layer rules. Order within one file is preserved. Every rule has exactly one path, glob, or regex matcher:

specs:
  formal-markdown:
    styles: [number]
    separator: '.'
    marker: ''
    wrapper: '`'
    numeric_origin: 1
    max_depth: 8
  dotted-exception:
    styles: [number]
    separator: '.'
    marker: '.'
    wrapper: ''

indexing:
  default: off
  insertion: observed  # observed | never; `all` belongs on a path rule
  paths:
    - glob: 'policies/**'
      spec: formal-markdown
      insertion: all
    - path: 'one/contextual-outline.md'
      spec: dotted-exception
      insertion: observed
      include_title: false

header_mode: flex  # default; use strict for profile-driven normalization

header_defaults:
  markdown: markdown
  source: source
  typst: off
  rst: off
  unknown: off

header_paths:
  - path: 'README.md'
    profile: off
  - glob: 'src/legacy/**'
    profile: code-legacy

headers:
  code-legacy:
    levels:
      - {formatters: [first], index_policy: optional}

ignore:
  - 'generated/**'
  - 'fixtures/**'

indexing.default: off is the built-in floor. A path is enrolled only by indexing.paths (or an explicit --spec command override). Merely observing a numeral never enrolls it. Markdown rules default include_title to false, so H1 remains the document title and address derivation begins at H2. Source rules default it to true. An unscoped insertion: all is invalid: put all on each path rule whose entire hierarchy is intentionally addressable.

Headerization defaults to flex: existing indexed shapes establish file-local hierarchy precedents, a uniquely established presentation is reused at its semantic depth when a required header receives an index, and headers needing no index edit remain byte-stable. Competing depth or presentation precedents fail closed with ARCH103. Use header_mode: strict or arch format --mode strict to enforce configured case and geometry exactly. The built-in markdown header profile is non-case-changing, forbids an index on H1, and treats H2+ indexes as optional dotted backtick tags. The built-in source profile retains the established code-header geometry and notation. Typst, reStructuredText, and unknown families default off until a real adapter exists. A header_paths rule selects only a header profile. It can never enroll indexing or select an index spec.

Styles are number, alpha-lower, alpha-upper, roman-lower, roman-upper, and symbol. The final configured style repeats at deeper places. Separators may be empty, ., -, or /. Markers may be empty, ., ), or :. A backtick is the supported wrapper.

Legacy top-level paths and insertion still load through a deterministic index-only expansion and emit warnings. They never select a header profile. Run arch config validate after editing, and use arch config show --path FILE to inspect the family, enrollment and rule source, selected spec/profile and their source layers, header mode, insertion policy, title participation, and matching ignore.

4 What is scanned#

Markdown ATX headings are structural. The built-in adapters also recognize standalone comments:

Family

Forms

Markdown

## `1` Section

Python, shell, Ruby, YAML, TOML

# 1 Note

JavaScript/TypeScript, Rust, C/C++, Java, Go

// 1 Note, /* 1 Note */

SQL, Haskell, Lua

-- 1 Note

INI/config

; 1 Note

extensionless source

# 1 Note

Typst, reStructuredText, and unknown suffixes are classified separately and yield no occurrences. They are not treated as Markdown aliases.

A plain comment with an idx is linted as a free occurrence. It does not silently become part of the structural hierarchy. Decorated comment headings such as ## 1 Main or // ## 1.1 Child are structural and can participate in insertion and reindexing.

For stdin, pass -. It is accepted by check and plan, never by apply. --stdin-filename controls syntax and path-rule selection and defaults to stdin.md. Stdin is capped at 16 MiB so an unbounded pipe cannot exhaust process memory.

5 Mutation boundary#

The planner enforces these constraints before either interface can write:

  • Repository discovery builds index plans only for paths positively enrolled by configuration. Existing numerals and malformed candidates never broaden participation.

  • Passing a path directly to the Python planner is itself explicit selection. Callers can set enrolled=False for diagnostic-only analysis.

  • For enrolled Markdown, H1 is unindexed by default and the addressable hierarchy begins at H2.

  • observed inserts a missing idx only at a hierarchy depth already used in that enrolled file.

  • all indexes every structural node and is legal only on an explicit path rule (or an explicit command override).

  • never validates and updates existing idxs without inserting any.

  • A malformed or grammar-incompatible numeric-looking token produces ARCH005 and no edit to that occurrence.

  • A hierarchy deeper than its spec’s max_depth produces ARCH006. That site and its descendants remain unchanged rather than receiving truncated addresses.

  • Configured regex path rules have a 50 ms per-match budget. A timeout is a structured configuration error and stops discovery at the first hostile expression.

  • Filesystem reads use identity-checked, no-follow descriptors. Symlink swaps are refused.

  • Files over 16 MiB, binary input, and non-UTF-8 text are skipped.

  • UTF-8 BOMs, LF/CRLF style, final-newline state, and permission bits are preserved.

  • Application compares the planned SHA-256 digest with current bytes and refuses stale content.

  • Replacement uses a same-directory temporary file, flushes it, preserves its mode, and atomically replaces the target. The parent directory sync is best-effort because replacement has already succeeded by that point.

--operation fix is the default. reindex recomputes every existing structural address. remove removes recognized idxs. Each is previewable with plan or apply --dry-run.

6 Terminal workspace#

Bare arch on terminal stdin/stdout, or arch tui [PATH ...], opens the PyRatatui workspace. Discovery and planning run in the background, leaving redraw and quit responsive.

Key

Action

j / k, arrows

move

space

select a file

m

cycle fix, reindex, remove

d

toggle diagnostic/diff preview

e

open in $VISUAL or $EDITOR

r

refresh in the background

a, then y

arm and confirm selected plans

?

show key help

q, Esc, Ctrl-C

restore the terminal and quit

Bare invocation off a terminal refuses with exit 2. It never guesses that a pipe should become an interactive session.

Header blocks use a parallel, purpose-built formatter. See Header formatting for HeaderSpec profiles, arch format, legacy subl/headers mappings, and the arch spec editor.

7 Migration from subl/headers#

subl/headers concept

my-arch document-idx equivalent

Idx

arch.idx.Idx

IdxSpec

immutable arch.idx.IdxSpec

attempt_to_read

Idx.attempt_to_read()

build_relative

Idx.build_relative()

header refresh

structural plan_file()

Sublime regions and view writes

decoded occurrences, immutable edits, atomic application

plugin settings

layered YAML path rules and named specs

Header presentation is a separate port: legacy HeaderSpec profiles map to immutable arch.idx.headers.HeaderSpec and HeaderProfile objects, while arch format uses the same content-hash-guarded file-plan boundary as document idx maintenance. The port intentionally does not replace arch.Idx: existing architectonic callers continue to receive the fixed four-place type (now also named arch.Quad), and document tooling imports from arch.idx.