iris container image

IRIS (an SGI Indy / Indigo2 emulator) as a container image for running IRIX as a build node, for example the irix/openssl workflow.

The image is built from upstream iris plus the patches in patches/, applied with git am on the commit pinned as IRIS_BASE in .forgejo/workflows/image.yml (currently upstream 4c1c87f). To move to a newer upstream, check out the new base, re-apply the patches (git am patches/*.patch), fix conflicts, regenerate them with git format-patch, and bump IRIS_BASE. Upstream has no equivalent of any of them, so all fourteen still apply (the virtio patch and the two after the VNC one come from the fork's feature/virtio-serial-combined, the 9p one from its feature/virtio-9p, exported with git format-patch. They were then rebased over the patches before them, so re-export from a tree with those applied):

Patch	Why
storage: size block devices	a Linux block device reports length 0; use it as a disk (Kubernetes volumeDevices)
serial: ECS JSON lines	serial_log = "-" and serial_log_timestamps = "ecs" put the guest console on stdout as Elastic Common Schema JSON, logger irix.guest
nfs: honor the execute bit	binaries linked on the NFS share can run from it
idle-pause: park without an armed Compare; 8254: no hptimer for a callback-less channel	the IP28 kernel never arms CP0 Compare (it uses the IOC 8254 timers), so idle parking never engaged; IRIX programs the 8254's channel 2 at 5 kHz, which kept a host core spinning
virtio: virtio-serial guest device	a virtio 1.x console/serial device (virtio-mmio in a GIO slot on IP22/24/28, virtio-pci on IP30), ports backed by TCP or Unix sockets; off by default, enabled with [virtio_serial] in the machine config (docs/virtio-serial.md in the source)
vnc: password	viewers are asked for a password (RFB VNC Authentication) read from [vnc] password_file / --vnc-password-file / IRIS_VNC_PASSWORD_FILE, else IRIS_VNC_PASSWORD; unset keeps the server open as before. Only the first 8 bytes count and nothing is encrypted, so tunnel the port on untrusted networks. Standalone iris only
virtio-serial: deliver a host's input before reporting it gone	a client that sent and hung up at once lost the end of what it sent, which for the guest agent meant whole requests
virtio: virtio-9p device	a host directory the guest mounts, 9P2000.L over a virtio device (ID 9) on the same transports as virtio-serial; off by default, [virtio_9p] in the machine config (docs/virtio-9p.md in the source). The guest side is the p9fs package
cow: lock an overlay against a second emulator	two emulators on one overlay ruin the filesystem in it; the second is now refused at start-up
seeq: keep a transmit frame the channel ran dry in	a poll that caught a descriptor chain the driver was still extending dropped what it had read and sent the rest as a frame of its own, starting mid-packet; with passt as the peer that was a garbage station address it then sent to, and a connection stalled
stream: cut a frame at the end of its IP packet	IRIX hands the NIC frames longer than the IPv4 packet in them (the rest is the next bytes of the stream); the NAT backend reads by the length field, passt behind mode = "stream" did not, and ssh through it stalled or failed at key exchange
virtio-9p: created entries belong to the configured uid/gid	what the guest creates in a share is chowned to uid/gid when iris may (root, or CAP_CHOWN), so an unprivileged host user can delete it; before, it belonged to the user running iris
virtio-9p: owners = "passthrough", uid_map, gid_map	the guest's user and group reach the host: getattr reports the host owner through the maps, a Tsetattr's uid/gid is made with fchownat (an unmapped id is EPERM), a create's gid is mapped to a host group; with p9fs's owners mount option the driver follows each create with the creator's uid. The default stays squash

Build-node profile

Cargo features lightning,idle-pause:

• lightning strips breakpoint and traceback support (fastest interpreter; no debugging),
• idle-pause parks the emulated CPU in recognized guest idle loops instead of spinning a host core,
• no graphics JITs (gpujit is left out).

idle-pause is opt-in and lightning does not imply it; without it an idle guest uses two host cores, with it about 0.3.

At run time use headless = true where no display is needed. Override the features with the workflow input, or docker build --build-arg FEATURES=....

Image contents

/usr/local/bin/iris, iris-ci (a wrapper that runs the glibc binary with this image's own loader, so a musl sidecar can use it from a mounted copy), irix-sh (a Forgejo step shell that runs a script in the guest), the IP28 PROM at /opt/iris/prom/070-1477-002.bin, the guest packages in /opt/iris/packages and the provisioning scripts below. The guest memory lives in /dev/shm: give the container a memory-backed volume there, at least as large as the RAM banks.

Guest packages and ssh

packages.list names the IRIX packages (.tardist, installed with inst) from the irix organisation's registry: zlib_modern, openssl_modern, openssh_modern, curl_modern, the virtio-serial driver vioser and qemu_guest_agent and the 9P filesystem p9fs. tools/fetch-packages.py downloads them at image build time and checks the sha256 and size the registry lists, so a bump is a one-line change (PACKAGE_HOST and PACKAGE_OWNER build args pick the registry).

The entrypoint iris-entrypoint starts iris (arguments pass through) and, in the background, irix-provision, unless IRIS_PROVISION=0. Each step is a command of its own:

1. irix-ssh-keygen --fresh makes a new ed25519 key in /run/iris/ssh: random, never in the image or on a volume, replaced at every container start.
2. irix-provision waits for the guest console (iris-ci login, up to IRIS_PROVISION_TIMEOUT = 1800 s), then runs the steps below. Every step runs; the exit status is 1 if one failed.
3. irix-install-packages installs each package with inst, in list order. A marker /var/adm/iris-packages/<package>-<sha256> in the guest makes a restart skip what is installed and install what changed.
4. irix-start-agent loads vioser (/etc/init.d/vioser start) and restarts qemu-guest-agent: no reboot is needed. IRIS_PROVISION_AGENT=0 skips this and 5.
5. irix-check-agent talks to the agent through its virtio-serial port (a Unix socket on this side, IRIS_AGENT_SOCKET, default /run/iris/agent.sock): a delimited guest-sync, guest-ping and guest-info, and logs the agent's version. Exit 0 answered, 1 silent for IRIS_AGENT_TIMEOUT (120 s), 2 no such socket (not checked).
6. irix-start-9p loads the p9fs filesystem (/etc/init.d/p9fs start; no reboot) and, if IRIS_9P_MOUNT="<tag> <directory>" is set, mounts that share there (IRIS_9P_OPTIONS, for example aname=some/dir,ro, are the mount options). A NFS mount on that directory is unmounted first, and /etc/fstab swaps its NFS line for <tag> <directory> 9p rw 0 0, so every boot mounts the share (IRIS_9P_FSTAB=0 leaves /etc/fstab alone). With IRIS_9P_MOUNT="share /share" the 9p device replaces the NFS export as the guest's /share. The guest finds the share if the machine config has the device (below); without one the module loads and idles. IRIS_PROVISION_9P=0 skips this.
7. irix-start-ipv6 (only with IRIS_IPV6_ADDRESS) gives the guest its IPv6 address and keeps it across the guest's reboots (see "IPv6" below). IRIS_PROVISION_IPV6=0 skips this.
8. irix-ssh-setup has the guest create the sshd privilege-separation user and an ed25519 host key (once, on the guest's disk), write /usr/local/etc/ssh/sshd_config.iris (root by key only), put the container's public key in root's authorized_keys (replacing the last start's) and (re)start sshd. IRIS_PROVISION_SSH=0 skips this.

irix-ssh [command] then logs in as root with that key (port IRIS_SSH_PORT, 2222); irix-ssh -6 does it over IPv6 ([::1], port 2223 unless IRIS_SSH_PORT is set).

To make the pod's port 22 the guest's port 22, on every address of the pod (so ssh root@<pod ip> works, with a Service or from outside): IRIS_PASST_ARGS="... -t 22", IRIS_SSH_PORT=22, and NET_BIND_SERVICE among the iris container's capabilities (passt binds before it drops privileges).

How files reach the guest. IRIS_INSTALL_VIA is share (a directory both sides see: the NFS export or the 9p share at IRIS_SHARE, mounted in the guest at IRIS_GUEST_SHARE), put (iris-ci put: through a scratch SCSI volume, no guest network and no share) or auto (the default: share if the guest sees it, which a probe file tells, else put). put needs a scratch volume in the machine config, at least as big as the largest package (openssh is 33 MB), and stages files in IRIS_GUEST_DIR (/var/tmp/iris). Each put restarts iris's network backend (see "passt and pesto"), so auto is for a guest that can always fall back on put: a fresh disk has no p9fs, and a test run of irix/virtio's package removes it again (a marker in the guest is only believed while versions still lists the product). The provisioning order is therefore packages, p9fs and its share, and then everything else over the share:

[scsi.2]
path    = "scratch.raw"
cdrom   = false
overlay = false
scratch = true
size_mb = 64

The guest agent needs iris's virtio-serial device with a port named org.qemu.guest_agent.0 (see iris's docs/virtio-serial.md); without it the check logs "not checked" and the rest still runs:

[virtio_serial]
enabled = true
slot    = "exp0"

[[virtio_serial.port]]
name   = "org.qemu.guest_agent.0"
socket = "/run/iris/agent.sock"

The 9P share needs iris's virtio-9p device ([virtio_9p], patch 0010; docs/virtio-9p.md in the iris source). On an Indigo2 give it slot = "exp0" and the virtio-serial device exp1 (it polls there):

[virtio_9p]
enabled = true
slot    = "exp0"
path    = "/share"
tag     = "share"
uid     = 1000    # what the guest creates in the share belongs to this uid/gid on the host
gid     = 0

That is the default, owners = "squash": the guest's own ids stay in the guest. For the guest's user and group to reach the host instead, add owners = "passthrough" (and uid_map, gid_map: "guest:host[:count],...", for example uid_map = "0:1000" to make the guest's root the host's 1000) and mount with the owners option: IRIS_9P_OPTIONS=owners. irix-start-9p then remounts a share that lacks it and puts it in /etc/fstab. The device trusts the guest's ids, so map only those the guest may act as.

Provisioning drives the guest's console, as irix-sh steps do: with a runner that also logs in to it, set IRIS_PROVISION=0 and run irix-provision where nothing else uses the console. It is safe to run again at any time.

passt and pesto

The image carries passt and pesto (its control client), built from the tag PASST_REF (a build arg; pesto needs a build from May 2026 on), for iris's [network] mode = "stream": user-space networking with the host's address and routes, no host privileges, IPv6. With IRIS_PASST=1 the entrypoint runs passt (restarted if it exits; iris reconnects) on /run/iris/passt.sock and forwards 127.0.0.1:$IRIS_SSH_PORT to the guest's port 22. Point iris at it:

[network]
mode   = "stream"
stream = "unix:/run/iris/passt.sock"

In this mode iris answers nothing on the segment (no DHCP, DNS, NFS, TFTP, [[port_forward]]): the guest gets what passt gives it (IRIX 6.5 can use its DHCP, IRIX 5.3 needs the address passt prints at start). Packages then come over a 9p share (IRIS_INSTALL_VIA=share), or iris-ci put where nothing needs the network just then. iris-ci run, get, login and the rest do not touch the guest's network and work as before. Change the forwards of the running passt with pesto -t 127.0.0.1/8022:22 /run/iris/passt-control.sock (pesto --help), or set options with IRIS_PASST_ARGS (replaces the default forward) and the socket paths with IRIS_PASST_SOCKET / IRIS_PASST_CONTROL.

patches-passt/ has the fixes this image applies to passt (git apply in the Dockerfile): the latest tap_passt_input() moved a half-received frame from a stale offset after a receive that found nothing (edge-triggered epoll makes that ordinary), so the frames after it were read from the wrong byte, which showed as garbage station addresses and reset connections.

passt starts as root, changes to nobody and sandboxes itself in new user, mount and pid namespaces. In a Kubernetes pod that takes SETUID and SETGID besides the capabilities the runner's pod already adds, and an AppArmor profile that allows mount(): under the runtime default passt stops at "Failed to remount /" (appArmorProfile: {type: Unconfined} on the iris container; allowPrivilegeEscalation: false is fine). Run in the cluster this way, it also works with a guest that keeps a static address: IRIS_PASST_ARGS="-m 1500 -a 192.168.0.2 -n 24 -g 192.168.0.1 -t 22" is what the home cluster's runner uses, with IRIS_GATEWAY=192.168.0.1: irix-start-route gives the guest that default route (/etc/config/static-route.options, so it stays across reboots); nothing else does. -m 1500 is the guest's MTU: passt's own (65520) is for guests that take it from DHCP. Restarting passt under a running guest can leave iris with a half-written frame: restart the pod, not passt. Not yet tried: passt with a snapshot restore (the connection is not part of the snapshot).

iris-ci put restarts iris's network backend. Each put (the scratch volume) tears the backend down and builds it again: with mode = "stream" that closes the connection to passt ("network stream closed by peer", "Client connection closed") and the guest's TCP sessions go with it; with NAT the port forward loses its listener to the old instance ("failed to bind: Address already in use"). On the home cluster's runner that looked like passt dropping and garbling frames: ssh to the guest failed at random for as long as provisioning (several puts) had just run, and -d made it pass by changing the timing. So with passt (or a port forward) use IRIS_INSTALL_VIA=share and give the guest a 9p or NFS share instead of a scratch volume; put only where nothing else needs the network. With that, the runner's guest on passt keeps one connection, answers ssh every time and gets https replies (irix-ssh, curl to git.stei.gr).

Three more bugs turned up on the way and are fixed by patches here, although they were not this problem (none was shown to matter in the end): iris's SEEQ dropped a transmit frame the channel ran dry in and sent the rest as a frame of its own (patch 0011), IRIX hands the NIC frames longer than their IPv4 packet and passt wants them cut (patch 0012), and passt moved a half-received frame from a stale offset after a receive that found nothing (patches-passt/). If a capture is needed: IRIS_PASST_ARGS="-d --pcap /tmp/passt.pcap ..." (the directory must be writable by nobody), then tcpdump -nn -e -r.

IPv6

IRIX 6.5.22 has an IPv6 stack, but a default install does not: the eoe.sw.ipv6 subsystem is on the Overlay CD 1 (inst, select it by hand; then reboot twice, the first reboot builds the kernel) and systune -b ip6_enable 1 turns it on. The guest's ifconfig takes only 2000::/3, fec0::/10 (the old site-local) and link-local addresses on an interface; a ULA (fc00::/7), which is what a Kubernetes pod network offers, is refused (SIOCAIFADDR6: Invalid argument: the stack is from 2003, RFC 4193 is from 2005). So passt is given a site-local address for the guest and NATs it, as it does the IPv4 one:

IRIS_PASST_ARGS="-m 1500 -a 192.168.0.2 -n 24 -g 192.168.0.1 -a fec0::2 -t 127.0.0.1/2222:22 -t ::1/2223:22"
IRIS_IPV6_ADDRESS=fec0::2

passt's router advertisements then put the guest on fec0::/64 (it configures an address from them by itself), and outbound IPv6 works at once. For inbound, passt forwards to fec0::2, which is what irix-start-ipv6 puts on ec0, now and (/etc/init.d/iris-ipv6, linked in rc2.d) at every boot. irix-ssh -6 uses the ::1/2223:22 forward. Two limits: the resolver returns no AAAA records, so IPv6 hosts are reached by address (or /etc/hosts); and openssh_modern listens on IPv6 only from the build that has iris_getnameinfo (IRIX's getnameinfo() cannot name ::).

The workflow pushes gitops.home.alaunstras.se/gitops-admin/iris:<iris-sha>-<timestamp> and :build-node (the job token cannot write packages to the irix org, so it uses the REGISTRY_TOKEN repository secret).