Design decisions :: Tools :: OpenShellOrg Docs

Rows marked PROVISIONAL can change before v1. Rows marked Decided are settled.

Question Answer Status Why

Question	Answer	Status	Why
Free-flag sigil	` (for example `+yes`, `+verbose)	Decided	Free flags are unordered and mean the same thing in every command, and ` says exactly that: you add things together, and the order doesn't matter (`+yes +verbose` = `+verbose +yes`). `-` and `--` stay with each tool's own flags. Repeating the sigil (`, `, `+) steps through `levels`.
Manifest format	JSON5 (`*.manifest.json5`), `manifest: 0`, validated by JSON Schema after parsing	Decided	Manifests are written by people, so comments, trailing commas and unquoted keys help. JSON5 parses to the JSON data model, so the existing JSON Schemas (draft 2020-12) still validate it, and plain JSON is still valid input. The D library uses a small in-repo JSON5-to-JSON translator (`json5.d`, unit-tested) feeding `std.json`. On code.dlang.org, `ddn-data-json5` is retired (merged into the larger `ddn` package) and `jsoniopipe` would pull in `iopipe`, so an in-repo translator keeps the library dependency-free. Infinity and NaN are rejected because JSON has no equivalent.
Namespace tree storage	SQLite index compiled from the JSON5 manifests	PROVISIONAL	See Namespace tree storage (PROVISIONAL).
Command name for the resolver	`install` (literally; no prefixed fallback)	Decided	Ryan’s rationale: "I wanted the command to literally be 'install' not 'osh-install' otherwise it breaks the illusion of a seamless unified interface." In the namespace, `install` is the root `install` node and coreutils/BSD `install(1)` is registered at `legacy/install`, so `install` resolves to the resolver. See Coexisting with install(1).
install(1) passthrough	Hand install(1)-style arguments to `legacy/install`	PROVISIONAL	Keeps `make install` style build scripts working. See Coexisting with install(1).
Implementation language	D (LDC), packaged with dub	PROVISIONAL	The Open Shell runtime and shell-context stubs are in D. It builds native single binaries for Windows, Linux and macOS.
Legacy subtree name	`legacy/`	PROVISIONAL	The name says what the subtree is, and it sorts away from real verbs.
Inspection tool name	`name` (`name which git`, `name conflicts`)	Decided	`name` reads as a verb or a noun depending on the rest of the command. No existing `name` command was found in WSL, pwsh or cmd, or in Debian, Ubuntu, Arch, Homebrew or PyPI. Formerly `ns`.

Free-flag sigil

` (for example `+yes`, `+verbose)

Decided

Free flags are unordered and mean the same thing in every command, and ` says exactly that: you add things together, and the order doesn't matter (`+yes +verbose` = `+verbose +yes`). `-` and `--` stay with each tool's own flags. Repeating the sigil (`, `, `+) steps through levels.

Manifest format

JSON5 (*.manifest.json5), manifest: 0, validated by JSON Schema after parsing

Decided

Manifests are written by people, so comments, trailing commas and unquoted keys help. JSON5 parses to the JSON data model, so the existing JSON Schemas (draft 2020-12) still validate it, and plain JSON is still valid input. The D library uses a small in-repo JSON5-to-JSON translator (json5.d, unit-tested) feeding std.json. On code.dlang.org, ddn-data-json5 is retired (merged into the larger ddn package) and jsoniopipe would pull in iopipe, so an in-repo translator keeps the library dependency-free. Infinity and NaN are rejected because JSON has no equivalent.

Namespace tree storage

SQLite index compiled from the JSON5 manifests

PROVISIONAL

See Namespace tree storage (PROVISIONAL).

Command name for the resolver

install (literally; no prefixed fallback)

Decided

Ryan’s rationale: "I wanted the command to literally be 'install' not 'osh-install' otherwise it breaks the illusion of a seamless unified interface." In the namespace, install is the root install node and coreutils/BSD install(1) is registered at legacy/install, so install resolves to the resolver. See Coexisting with install(1).

install(1) passthrough

Hand install(1)-style arguments to legacy/install

PROVISIONAL

Keeps make install style build scripts working. See Coexisting with install(1).

Implementation language

D (LDC), packaged with dub

PROVISIONAL

The Open Shell runtime and shell-context stubs are in D. It builds native single binaries for Windows, Linux and macOS.

Legacy subtree name

legacy/

PROVISIONAL

The name says what the subtree is, and it sorts away from real verbs.

Inspection tool name

name (name which git, name conflicts)

Decided

name reads as a verb or a noun depending on the rest of the command. No existing name command was found in WSL, pwsh or cmd, or in Debian, Ubuntu, Arch, Homebrew or PyPI. Formerly ns.

Namespace tree storage (PROVISIONAL)

The tree has two layers:

Source of truth: the JSON5 registration manifests, one per package. People and installers write these.
Compiled index: a store built from the manifests that answers resolve, children and conflict queries quickly. It can always be rebuilt, so it is a cache, not data.

Options compared

Need YAML file(s) SQLite index JSON5 file tree (one file per node or package)

Need	YAML file(s)	SQLite index	JSON5 file tree (one file per node or package)
Concurrent installer writes	Poor. One shared file means read-modify-write races, and it needs an external lock file.	Good. Transactions with WAL; several installers can register at once, and a conflicting claim fails atomically.	Fair. One file per package avoids most races, but a check-then-write across files still needs a lock.
Conflict-query speed	Poor. Parse everything, then scan.	Good. An index on `path`; `GROUP BY path HAVING count(*) > 1` is one query.	Fair. Walk and parse the tree, or keep a separate index anyway.
Human inspectability	Good.	Fair. Needs `sqlite3` or `name` to look inside. Mitigated because the manifests stay readable.	Good.
Diffability	Good, but one big file gives noisy diffs.	Poor as a binary file. Mitigated: diff the manifests, or dump the index as text.	Good, with small focused diffs.
Fits the JSON5 source decision	Adds a second text format.	Yes. It is a compiled view of the manifests.	Yes, but then it is also the index, so it still has the query problem.

Concurrent installer writes

Poor. One shared file means read-modify-write races, and it needs an external lock file.

Good. Transactions with WAL; several installers can register at once, and a conflicting claim fails atomically.

Fair. One file per package avoids most races, but a check-then-write across files still needs a lock.

Conflict-query speed

Poor. Parse everything, then scan.

Good. An index on path; GROUP BY path HAVING count(*) > 1 is one query.

Fair. Walk and parse the tree, or keep a separate index anyway.

Human inspectability

Good.

Fair. Needs sqlite3 or name to look inside. Mitigated because the manifests stay readable.

Good.

Diffability

Good, but one big file gives noisy diffs.

Poor as a binary file. Mitigated: diff the manifests, or dump the index as text.

Good, with small focused diffs.

Fits the JSON5 source decision

Adds a second text format.

Yes. It is a compiled view of the manifests.

Yes, but then it is also the index, so it still has the query problem.

Recommendation

SQLite as the compiled index, with JSON5 manifests as the source of truth (PROVISIONAL). This keeps the readable, diffable parts (the manifests) readable and diffable, and gives the tool safe concurrent writes and fast conflict checks where they matter. If the index is lost or corrupted, a rebuild from the manifests restores it.

Rules (proposed):

Installers write their manifest, then register it in one transaction. A refused conflict rolls back the whole package.
name and the shell read from the index. name rebuild (future) recompiles it from the manifests.
The index stores a hash of each manifest. If a manifest changes on disk, the index is out of date and gets rebuilt.

Proposed tables

CREATE TABLE package (
  id            TEXT NOT NULL,
  version       TEXT NOT NULL,
  manifest_path TEXT NOT NULL,
  manifest_hash TEXT NOT NULL,   -- sha256 of the JSON5 source
  policy        TEXT NOT NULL CHECK (policy IN ('refuse','shadow-with-warning')),
  registered_at TEXT NOT NULL,
  PRIMARY KEY (id, version)
);

CREATE TABLE claim (
  rowid       INTEGER PRIMARY KEY,         -- registration order (shadowing)
  path        TEXT NOT NULL,               -- node path, NS-1
  parent      TEXT NOT NULL,               -- '' for top level; children queries
  kind        TEXT NOT NULL CHECK (kind IN ('verb','provider','command','alias','group')),
  summary     TEXT,
  argv_json   TEXT,                        -- entry argv as a JSON array
  style       TEXT NOT NULL DEFAULT 'strict',
  target      TEXT,                        -- alias target
  source      TEXT NOT NULL,               -- 'manifest:<id>@<ver>' or 'legacy:PATH:<dir>'
  pkg_id      TEXT,
  pkg_version TEXT,
  FOREIGN KEY (pkg_id, pkg_version) REFERENCES package(id, version) ON DELETE CASCADE
);
CREATE INDEX claim_path   ON claim(path);
CREATE INDEX claim_parent ON claim(parent);

CREATE TABLE signature (
  node      TEXT NOT NULL,
  pkg_id    TEXT NOT NULL,
  body_json TEXT NOT NULL                  -- input signature, validated JSON
);

-- conflicts (NS-3)
-- SELECT path, count(*) FROM claim GROUP BY path HAVING count(*) > 1;

Not implemented in v0. The in-memory Registry is the reference behaviour that a store must match.

Coexisting with install(1)

On Unix PATH-based shells, coreutils/BSD install(1) already exists and copies files and sets modes.

Namespace model: install is the root install node. install(1) is registered at legacy/install. install resolves to the resolver; legacy/install reaches the file copier.
Plain PATH shells: packaging MUST NOT overwrite /usr/bin/install. Put the resolver ahead on the user’s PATH only (for example ~/.local/bin), or use a shell alias or function. Never change the system PATH.
Build scripts: scripts that call install(1) with its file-copy flags (-m, -D, -d, -o, -g, …) MUST keep working.
Passthrough (PROVISIONAL, implemented in openshellorg/install): when the arguments look like install(1) (any install(1) option, or an existing source file followed by a destination), the resolver hands them unchanged to the real install(1) at a fixed system path (/usr/bin/install, /bin/install, or Homebrew ginstall), never found by searching PATH, and returns its exit status. Resolver commands and flags (search, managers, help, --help, --version) always win, and bare install prints help.
Windows: there is no install command, so there is no clash.