Decisions
What the formats left open, and what Hatchery chose. Each line is a rule you can rely on.
Specs and ranges
group/id@1.4.0with a complete version means exactly that version. Any other text after@is a range in Cargo's syntax.- In manifests, ranges are in Cargo's syntax too.
"1.2"means^1.2. - An id alone resolves when exactly one group in the configured registries has it. Otherwise the error is
Ambiguous.
Variants
- The fallback variant has neither
osnorarch. Its key isdefaultand its folder isvariants/default. - The fallback is used only when no keyed variant matches, wherever it is listed.
- A variant key is its values joined by a dash,
osfirst.
Targets and manifests
- A target may say
manifest="plugin.json". That is the filehatchery pack --checkvalidates. Without it, the file is<target>.jsonin the target's folder. - Target names are ASCII letters, digits,
-and_, and nevervariants. The manifest parser refuses the rest. - Unknown nodes in a manifest, a root manifest and an index are kept when the file is written again.
Eggs
- Archives are gzipped POSIX ustar, written by Hatchery's own writer and read by its own strict reader. They are not made with the
tarcrate. - Entries are sorted by path and are files only, with no directory entries. Modes are 0644 or 0755, so the executable bit is kept. User and group ids are 0, names are empty, and the modification time is 0. The gzip header is fixed. The same input gives the same SHA-256.
- A base egg is
<id>-<version>.egg:package.kdl, the complete manifest, with the repository's defaults applied and anoriginsection. Every targetpath=and variantpath=is removed, because inside the egg each target's files are at the default folder.- The files
aboutnames, at their relative paths. <target>/<path>for each target's shared files.
- A variant egg is
<id>-<version>.<target>.<key>.egg:variant.kdl, the header, withformat 1and avariantnode holdinggroup,id,version,targetand theosandarchlines the variant has. The fallback variant has neither.<target>/variants/<key>/<path>for the variant's files.
- A fat egg is the base egg's entries plus
variants/<variant egg file name>, each the complete bytes of a variant egg. Removing those entries and writing again gives the base egg byte for byte. - An egg may not hold a
.gitcomponent in any case,git~1, a component ending in a dot or a space, Windows-invalid characters or device names, or two paths that differ only by case. A.gitfolder in an install folder runs code when someone runs git there, and the other names clash or change meaning on Windows and macOS. The reader andpackboth refuse them. origin.builtisSOURCE_DATE_EPOCHif set, else the commit time ofHEADwhen the folder is in a git checkout, else it is left out. Packing the same folder twice gives the same egg.origin.repositoryisabout.repository, else theoriginremote of the checkout with any user and password removed.
Installed layout
<dir>/eggs.lockis TOML, withformat = 1at the top.seen.tomlandunpacked.tomlin<dir>/.hatchery/have it too. Unknown keys are kept when Hatchery writes them again.<dir>/<group>/<id>/is the unpacked base egg:package.kdl, the about files and every target's shared files.<dir>/<group>/<id>/<target>/variants/<key>/is each variant that was fetched, unpacked from a variant egg or from inside a fat egg.- An install folder refuses a package whose group, or whose id within one group, differs from another only by case. Windows and macOS would put both in one folder. A group or id may not end in a dot or be a Windows device name such as
con. <dir>/.hatchery/seen.tomlrecords when this installation first saw each version in each registry.<dir>/.hatchery/unpacked.tomlrecords the base egg checksum of each package folder.installskips a package only when the version in its folder and that checksum match the lock.Installed::versionis the version in the folder.
Static registry layout
<base>/index.jsonis{"packages": ["aiviarium/claude", …]}.<base>/<group>/<id>/index.jsonis the package index.<base>/<group>/<id>/also holds a copy of the icon, the base eggs and the variant eggs.- A version entry without
sha256has only variant eggs published so far. It is not offered. - Signatures are lowercase hex Ed25519 over the bytes
sign::messagebuilds: a fixed tag, then the package, version, file name, checksum and publish time, one on each line. Keys areed25519:<64 hex>. - Signatures cover
published, so the publish time cannot be backdated without the key. - An egg published again with the same bytes is a no-op. With different bytes it is an error, because versions are immutable. Variant eggs may be added to a released version later.
Sources
- A single-package source (a folder, an egg file, one git folder) answers
names()with its one name andversions(name)with its one offer. - An offer with a commit from a git source must still resolve to that commit when fetched. If the tag points elsewhere now, the fetch fails with
TagMoved. Fetch::getreturnsOk(None)for a 404, so that a search over several registries can go on to the next.- The client is not built when the variable of a
token-envis not set or is empty. It is not built either when a token would go to anhttp://address other thanlocalhostor a loopback address. - A package from a git source is the committed files of its folder, read from the objects. Attributes, line-ending settings and filters of the machine are not applied. A tag gives the same egg everywhere. Symlinks and submodules are refused. Executable files keep their bit.
- Hatchery needs git 2.25 or later. That has
--filterand protocol version 2 for fetching,rev-list --missing=printto ask what is in the cache without fetching it, andls-remote --symref. Hatchery uses nothing that needs a later version:ls-tree --formatandGIT_NO_LAZY_FETCHare not used.
The command line
--variant TARGET os=… arch=…names a target and then one or two keys. Values after--variantthat contain=belong to the same use. A value without=starts the next one.pack --checkreads the base egg, so it needs--baseor a pack that writes it.fetchhas no target, so it doesn't check requirements.add,updateandinstallneed at least one--target. Every--factis given to every target.publishtakes the signing key only from an environment variable that you name.publishwithout a destination readsrelease.eggsand theoriginremote.keygenprints the signing key on the first line and the public key on the second.verify --registrycompares the egg with the package index it finds in a static registry: its checksum, and with--keyits signature.registry addwrites./hatchery.kdlwhen no configuration file is found. IfHATCHERY_CONFIGpoints at a file that does not exist yet,registry addcreates it and other commands read an empty configuration.
The Rust API
- The traits return
BoxFutureinstead of usingasync fn, so thatSourceandFetchcan be trait objects without a macro crate. A source or fetcher writesBox::pin(async move { … }). Fetch::getreturnsResult<Option<Bytes>>.Noneis a 404.Sourcehas more thanversionsandfetch:label,is_local,namesfor listings and for an id given alone,detailsfor an offer that comes without its targets (a git tag), andfetch_variant.Client::offeredtakes the install folder, because it reports installed versions and reads the first-seen records there.publishreturnsPublished, which gives the file name and SHA-256 of the egg, the publish time, and whether it was signed.- A client may install for several targets. A package needs at least one of them. Requirements, dependencies and variants come from those targets only.
- A base egg is unpacked whole, with the shared files of every target. Only the targets of the client are checked and locked. Their folders stay in the package folder even though the lock records only the client's targets.
- The public API is the client, the traits, the sources, the errors and the data types that a caller reads or builds. Resolution, the git cache, the tar writer and the file helpers are private, so they can change in a minor release.
- Structs with public fields and the enums other than
Errorarenon_exhaustive. A caller makes one with its constructor or builder, so a new field is not a breaking change. - The futures block their thread while git or
HttpFetchruns. An executor is not needed, and an async caller runs a client call on a thread for blocking work.
Resolution and installs
- The version already installed is never held back by the minimum release age, and it is not dropped because it was withdrawn. It stays, and the report warns about a withdrawn one. An update still moves to a newer version that is allowed.
--allow-younglifts the hold for the package named in the request, not for its dependencies.- When
addinstalls an older version than the newest in the range, its report starts with a warning that gives the newest version and why it was skipped. Client::withdrawnandhatchery listread the registries of the installed packages to mark withdrawn versions. A registry that cannot be read is skipped. A listing then works offline.- A version is skipped only for a reason it cannot be installed here. A failure to read it, such as a network or git error, stops the resolution, so that an older version is not chosen by accident.
- A version whose newest candidates are skipped falls back to an older one. When nothing is left, the error is the reason of the newest version in range:
Withdrawn,Unmet,NoVariant,NotBuiltorTooYoung. - Versions from git tags have no signed publish time, so they count from when this installation first saw them. So a new installation with a hold doesn't install a version that only has a tag until the hold has passed. Give your own repository
min-age="0d", or use--allow-young. - The backdating guard counts per registry and package. Versions in the first listing of a package count from their signed time. A version that appears later counts from the later of its signed time and when it was first seen.
- A registry without a
keyhas no signed time. Itspublishedtimes are ignored, and every version counts from when it was first seen, in the first listing too. - A registry given as a local path or a
file://URL is local, and is not held by default. - An installed package keeps the version it has when
addnames that version again. It is not fetched a second time. - Updating a package replaces its folder, so variants that were fetched on demand are fetched again when needed.
installfrom the lock checks each egg against the locked SHA-256. A package installed from a folder is packed again and its new SHA-256 is recorded, because you chose that folder.- After fetching an egg, the client compares the dependencies in the egg with those in the index it resolved from. An index that differs from its egg is an error, because the signature covers only the egg's bytes.
- The client holds a lock on
<dir>/.hatchery/lockwhile it changes the folder.
Git repositories
- A tag that points at another commit than the one recorded is refused, for a git registry and for one git folder alike. A branch may move:
updateinstalls its new commit, andinstallfrom the lock fetches the locked commit by its id. - A built egg on the eggs branch whose version has a tag is checked against that tag's commit too.
- A withdrawal in
packages.kdlon the default branch counts even when the eggs branch index does not list it. - The eggs branch may hold versions without a tag. They are offered as built eggs with no commit.
- With a
key, a version that has only a tag is not offered and not installed:SignatureInvalid.unsigned-tags=#trueon the line allows it. This holds for installs fromeggs.locktoo. - A package is found by its name among the packages of the tag's own commit. The folder on the default branch is only tried first. An install from the lock tries the locked folder first.
- A package that left the default branch keeps its built eggs, and its locked tags still install from the lock. It is no longer offered for new installs.
- A tag pattern without
{id}is shared. A shared tag counts for a package only when the package's manifest at that commit has the tag's version. Listing such a package fetches each of its candidate tags once. - A package folder whose
package.kdlon the default branch does not parse is skipped, and the source reports it throughSource::warnings. Asking for a package whose folder has that name gives the parse error. - Registry addresses in
eggs.lock, in.hatchery/seen.toml, in an egg'sorigin, in listings and in error messages have their user and password removed. Only git itself is given the address as configured. - The git cache is
$HATCHERY_CACHE/git, or without that variable$XDG_CACHE_HOME/hatchery/git(or~/.cache/hatchery/git), one folder per repository, used by one process at a time.
Static registries
requiresis a JSON object, so the order of a target's requirement lines is not kept in the index.- A package is added to the root
index.jsonwhen its base egg is first published, not when only a variant egg is. - Publishing a base egg again replaces
aboutin the package index with the egg's. Unknown keys are kept.
Edit on GitHub
Last updated on