# 3pm modules Optional features ship as **modules**: directories with a `module.toml` and executables, bundled with the 3pm install (tarball, `install.sh`, deb/rpm) and enabled per machine with `3pm install `. A bundled module is a copy of what the release already brought; a module 3pm does not bundle (`automonit`) is downloaded from the download site, the same one `autoupdate` reads. Nothing about a module lives in the 3pm binary: its dependencies stay in its own `Cargo.lock`. The mental model is pm2's `pm2 install `, minus npm. ## Using modules ```console $ 3pm module list # what is installed / bundled with this 3pm $ 3pm serve ./dist 8080 --spa # a bundled command runs without install $ 3pm install serve # copy it into ~/.3pm/modules (pins a version) $ 3pm install ./my-module # any directory holding a module.toml $ 3pm install automonit # not bundled: from the download site $ 3pm install automonit@0.42.0 # a pinned version (always the site) $ 3pm uninstall serve ``` - `3pm …` runs the module's `command` — the installed copy if there is one, else the bundled one. When the bundle is newer than the installed copy, a one-line hint says to reinstall. - `3pm install ` copies the module into `~/.3pm/modules//` (a staged copy swapped in atomically), starts its `[[apps]]` and **saves the process list**, so they come back after a reboot without a separate `3pm save`. Bare `3pm install` lists. On a module already installed the files are swapped and the module's rows are restarted **in place** — same row, same name, a fresh `spec.module.version` — never deleted and recreated: the manifest apps on their new spec (one the release added is started, one it removed is deleted), the apps the module started itself (`3pm serve` sites) and the apps started by hand from its files as they are. - `3pm update` does the same after the daemon, for every installed module whose bundled version changed **and** for the apps running straight from the bundle without an install (the agent after `3pm link`): a row whose stamp differs from the `module.toml` next to its script is restarted on the new files and re-stamped; one already at that version is left alone, so a second `update` touches nothing. The check is per row (an installed copy whose files are current but whose rows are behind them, after a refresh that failed midway, is refreshed too), never a bare comparison of versions. `--skip-modules` skips the step; a module app whose directory vanished is left as is, with a notice. - `3pm uninstall ` deletes the module's apps — and any app whose script runs from the module's files, such as sites started with `3pm serve` — then removes the directory and saves. On a module the release marks `auto` it is also the **refusal**: the name goes into `/modules.off` and nothing puts it back until `3pm install ` asks for it again. - **A module the release marks `auto = true` installs itself.** It is part of 3pm the way a built-in command is (`autoupdate`, so that any node can be updated from the dashboard), so 3pm does what `3pm install` would have done — copy, start, save — without being asked, at the two moments it has a reason to look: a daemon this CLI just spawned (the first command on a fresh machine, and every boot without a startup service) and after a `3pm update` (how a host already running learns that a release added one). Only the **bundle**'s manifest is read for it, only a module this host has neither installed nor refused is touched, and a module that will not install is a warning, never a failed command. A 3pm that predates the key ignores it. - **The configuration can switch an `auto` module off**, the same on every host it is provisioned to: `[modules] off = ["lan"]` in `/config.toml`. A module named there is never auto-installed, and one already installed is uninstalled at the next look (a daemon this CLI spawns, `3pm update`): its apps stopped and deleted, its copy removed, the list saved — as `3pm uninstall` does, but nothing is written to `modules.off`: take the name out of the file and the next look installs it again. `3pm install ` still works, until that next look. The daemon checks the section with the rest of the file (an unknown key is named in its log). - Module apps appear in a second **Modules** table in `3pm ls` and are ordinary targets by name (`3pm stop serve`, `3pm logs ` by namespace), but `stop|restart|delete all`, `delete --flush all`, `apply --prune-all` and `restart --update-env` leave them alone: modules are managed by install/uninstall. A dumped module app whose directory is gone is dropped on `resurrect`. - Unknown subcommands get a did-you-mean over built-ins and modules. ### Modules from the download site `3pm install ` resolves, first hit wins: a path holding a `module.toml`; `@` or `@latest`, which always means the site; a bundled name; an installed name (its apps restart); then the site. The site is `$THREEPM_CDN_URL`, `https://3pm.edgecr4ft.net` by default, laid out by `distribution/scripts/publish-automonit.sh`: ```text /modules//latest/version.json the release to take /modules//v/version.json a pinned one /modules//v/--.tar.gz ``` 3pm reads the index, takes the asset of its platform (`os` `linux` or `macos`, `arch` `x86_64` or `arm64`), downloads it with `curl` (else `wget`), checks its sha256, unpacks it into `/modules/.site.tmp/` and installs that directory exactly as `3pm install ./dir` would: copy, start, save. The installed copy carries `.3pm-site.json` (`site`, `version`, `pinned`); that marker is how `3pm update` knows the module came from the site. On each update an unpinned site module is checked against `latest/version.json` and, when a newer release is out, fetched and reinstalled in place like a module whose bundle moved on; a site out of reach is a warning, never a failed update. `@` pins (the update leaves it there), `@latest` unpins, and installing from a directory or the bundle drops the marker. What is refused, before anything is installed: - a site other than `https://` (a `file://` mirror and plain `http` on the loopback are accepted, for tests and local mirrors); - a name outside the module-name grammar, a version outside `[0-9][A-Za-z0-9.+-]*` (no `/`, no `..`, no leading `-`); - an index for another module or another version than asked; - an archive URL other than `/modules//v/`: only the file name of the index's `url` is used; - an archive larger than its announced `size` (plus 64 KiB), slower than 5 minutes, or whose sha256 differs — the hash is checked before the archive is opened; - an entry with an absolute path or a `..`, outside `/`, a link leading out of it, a device or a fifo, more than 10 000 entries or 2 GiB unpacked; a `module.toml` missing or naming another module. An unknown name says what was looked for and where, with the local did-you-mean: `no module named `automonitt` (did you mean `automonit`?) — available: …; not on https://3pm.edgecr4ft.net either (cannot read …/modules/automonitt/latest/version.json: … 404 …)`. No build for this platform names the published ones. The shell installer (`/modules//install.sh`) stays for a 3pm up to 0.42.0. Where the bundle is looked up, first hit wins: `$THREEPM_MODULES_DIR`, `/modules` (tarball / `install.sh` → `~/.3pm/bin/modules`), `/../lib/3pm/modules` (deb/rpm → `/usr/lib/3pm/modules`). ## Writing a module ``` my-module/ ├── module.toml └── bin/ ├── my-module # `command`: what `3pm my-module …` runs └── worker # scripts of [[apps]] ``` ```toml [module] name = "my-module" # letters, digits, '-', '_'; not a built-in command version = "1.0.0" description = "one line for `3pm module list`" command = "bin/my-module" # optional, relative, inside the module, executable auto = false # optional: true installs it without being asked, # until `3pm uninstall` refuses it (bundle only) [defaults] # optional, ecosystem syntax max_restarts = 5 [[apps]] # optional: started by `3pm install` name = "worker" script = "bin/worker" # relative to the module directory ``` Rules the loader enforces: `[[apps]]` use the ecosystem vocabulary (`docs/ECOSYSTEM.md`) with paths relative to the module directory; scripts without an interpreter must be executable; apps are named `:` (a single app named like the module keeps the bare name — pm2's rule), get `namespace = ` unless set, and carry `spec.module = {name, version}`; `spec.source` points at the installed `module.toml`. Unknown fields warn with a did-you-mean. A `command` receives the caller's arguments after the module name and, on top of the environment: | Variable | Value | |---|---| | `THREEPM_HOME` | the 3pm home (the daemon is already running when the command starts) | | `THREEPM_EXE` | path of the `3pm` binary (e.g. to print the table with `3pm ls`) | | `THREEPM_MODULE_DIR` | the module's directory (installed or bundled) | | `THREEPM_JSON` | `1` when `3pm --json …` was called (a `--json` *after* the name is the module's own) | | `THREEPM_PARENT_FD` | the read end of a pipe whose write end only the dispatching `3pm` holds: EOF there means `3pm` died (even by SIGKILL) | Its exit code is 3pm's, and `SIGINT`, `SIGTERM`, `SIGHUP` and `SIGQUIT` are forwarded to it (3pm waits for it to finish; a signal 3pm was started ignoring stays ignored). A `3pm` killed outright forwards nothing: a module that must not outlive it watches `THREEPM_PARENT_FD` and stops on its EOF (a Rust module calls `threepm_protocol::module::parent::watch()` first thing in `main`, which raises `SIGTERM` at it — its normal way out; `agent` and `serve` do); on Linux the kernel also sends it `SIGTERM` (`PR_SET_PDEATHSIG`). Any language works; a Rust module depends on `threepm-protocol` (`client::connect`, `Request::Start` with a `SpecPatch`-built spec) and, for apps that want `wait_ready`, speaks the control channel (`hello` then `ready` on `$THREEPM_CTL_SOCK`, see OV `3pm-docs-archived/architecture` §4.3b). `threepm_protocol::module` is that contract as code; `serve` and `agent` are built on it, while `autoupdate` and `lan` speak the control channel through the Rust SDK (`threepm-io`, `io().serve()`) instead. The toolkit: `is_run_role` (the `__run` split), `Out` (`--json`/`THREEPM_JSON`, the `[3PM]` chrome — colored like the CLI's, routed to stderr in JSON mode so stdout stays one document), `exit_code` (`[3PM][ERROR] …`), `ctl::{Identity, announce_ready}` (the handshake, plus a callback on `shutdown`), `affected`/`unexpected` (a daemon reply as the rows, or as the one sentence every module fails with) and `shutdown_signal` (SIGINT or SIGTERM, handlers installed at the call). A module prints only through the pipe-safe `oprintln!`/`oeprintln!` (`clippy::print_stdout`/`print_stderr` denied, as in the CLI: `3pm serve … | head` must not panic). Apps a command starts should carry `spec.module = {name, version}` too (as `3pm serve` does): they then list under **Modules** in `3pm ls`, below the apps started by hand, and get the same treatment as installed apps (`all` targets skip them, uninstall removes them). In-tree modules are standalone Cargo workspaces under `modules//` (a `[workspace]` table, path-deps on `../../crates/threepm-core` and `threepm-protocol`, own lockfile), built by `just modules` / `just bundle` and by the release matrix, and packaged as `modules//{module.toml,bin/…}` next to the binary. Their e2e tests use `threepm-testsupport` with `THREEPM_BIN` pointing at a 3pm binary and the module staged as a bundle (`modules/serve/tests/e2e.rs`). ## Catalogue | Module | Command | Apps | Notes | |---|---|---|---| | `serve` | `3pm serve [dir\|page.html] [port]` ([cheatsheet](SERVE.md)) | none | pm2 `serve`: `--spa`, `--ftp`, basic auth, `--host`, `--homepage`, custom `404.html`; free port picked, local/network links + QR code; a single `.html` file is served as homepage, opened in the browser (no QR code) and held until Ctrl-C (`--detach`/`--attach`); `PM2_SERVE_*` env honored | | `agent` | `3pm agent link [server]` (default `wss://fleet.edgecr4ft.net`, `THREEPM_LINK_SERVER` wins) / `link` alone (check, exit 0 if established) / `link stop` / `link start` / `link delete` / `unlink` / `status` (`3pm link …` / `3pm unlink` are built-in sugar over them) | one (`agent`, supervised, `wait_ready`) | Fleet link to a `3pm-backend`: enrolls this host (Ed25519 node key in `/agent`, owner-only), then streams the process table and host metrics over one WebSocket. `THREEPM_LINK_*` env overrides the pasted line; clear text off loopback needs `--insecure`; `--cap`, `--no-env-keys`, `--force`; contract, files and caveats in [backend-agent/](backend-agent/README.md) | | `autoupdate` | none — the actions are the surface | one (`autoupdate`, supervised, **`auto`: installed without being asked**) | Updates 3pm itself: `3pm trigger autoupdate update [{"version":"0.33.0"}]` downloads `install.sh` from `$THREEPM_CDN_URL` (default `https://3pm.edgecr4ft.net`) and runs it — the daemon then hands off — and `check` compares `latest/VERSION` with the version these files shipped with. The call answers as soon as the updater is started, because the update restarts this very app: the updater `setsid`s out of its process group, holds `/autoupdate.lock` and logs to `/autoupdate.log`. **Every hour** (`THREEPM_AUTOUPDATE_EVERY`: `30m`, `90s`, `off`) the app checks the site by itself and updates when it is behind — once per published version (`/autoupdate.last`), never in a loop | | `lan` | none — `3pm trigger lan lan.status` | one (`lan`, supervised, **`auto`: installed without being asked**; works on a root daemon on Linux, idle anywhere else) | A virtual Ethernet between the nodes of a bucket, made only once a peer shows up (removed 10 min after the last one left; never over a network already in use on the host); off with `[modules] off = ["lan"]` in `config.toml`: a TAP device (`3pmlan0`, `10.77.0.0/16`, address and MAC derived from the node id) whose frames ride fleet stream sessions (`io.open`) to every other node of the bucket running `lan`, found by itself (`fleet:list`; sessions need `fleet:invoke`). Optional `/etc/3pm/lan.toml` (else `$THREEPM_HOME/lan.toml`): `iface`, `network`, `address`, `allow` (an allow-list; `peers` still read), `broadcast_route`, `rate_kbit` (per peer, default 4000, cut by a latency guard while the peer's round trip is over 1 s), `expect_direct`, `netns` (a namespace `/run/netns/` holding only the lan: macvlan `3pmlan1`, own address and MAC, no default route — for a program that binds to the default route's interface), `netns_address`. The bucket is the trust boundary. Metrics `lan.frames`/`lan.bytes`/`lan.dropped`/`lan.peers`/`lan.rtt`, events `lan.peer_up`/`lan.peer_down`; see FEATURES.md, *A virtual Ethernet between nodes* | ## Not yet - A config store (`3pm set : ` / `3pm get`, pm2's `module_conf.json`) — the seam is `SpecPatch` as a partial update. - A catalogue of the site's modules (`3pm module list` shows what is installed and bundled only; the did-you-mean does not know the site). - `script = "serve"` shorthand in ecosystem files: use the module binary as the script with `args = ["__run"]` and `PM2_SERVE_*` env.