Hypervisor configuration

The agent reads /etc/gigahost/hypervisor.toml. For a first setup, see Creating the configuration.

Settings

Where no default is given, you set the value as the setup page shows.

Setting	Default	Meaning	More
backend_url	https://api.gigahost.cloud	The API the agent talks to	The backend URL
host_label		The host’s label	Enrolling once
location	Required	A location slug from ghost catalog locations	Location
mode		home_gateway or cloud_routed	Home mode, cloud mode
cpu_core_limit_per_numa_node		The vCPUs sold on each allowed NUMA node; 0 takes them all	Capacity
memory_bytes_per_cpu_core		Saleable RAM per logical CPU	CPUs, RAM and disk
root_storage_bytes_per_cpu_core		Saleable root storage per logical CPU	CPUs, RAM and disk
allowed_numa_nodes		The NUMA nodes the host sells	Raise a host’s capacity
libvirt_uri		Only qemu:///system is accepted	What enrollment checks
cloud_uplink	"auto": the interface of the default IPv4 route	A cloud host’s uplink	Configuring cloud mode
dns_servers		The IPv4 resolvers of a cloud host’s router VM and guests	Configuring cloud mode
state_directory	None: the state is on the system drive	The host’s state on its root pool’s drive	Paths
state_file	/var/lib/gigahost/state.json	The host identity and state	Paths
journal_file	/var/lib/gigahost/journal.sqlite3	The job journal	Paths
telemetry_spool_file	/var/lib/gigahost/telemetry.sqlite3	The telemetry spool	Paths
instance_directory	/var/lib/gigahost/instances	VM state	Paths
image_cache_directory	/var/lib/gigahost/images	The image cache	The image cache
image_cache_max_bytes	"20G", at least "1G"	The image cache’s limit	The image cache
image_predownload	true on a home host, false on a cloud host	Download the catalog’s images ahead of demand	Downloads ahead of demand
image_predownload_rate_bps	10 Mbit/s	Their pace while the host has no network test	Downloads ahead of demand
backup_upload_rate_bps	500 Mbit/s on a cloud host, 50 Mbit/s on a home host	The aggregate hosted-backup upload limit, in bit/s	Backup transfers
[[file_storage_pools]]		A file-backed pool: external_id, kind (root or backup), directory	Preparing storage
[[lvm_storage_pools]]		A thin-LVM pool: external_id (volume_group/thin_pool), kind	Preparing storage

Capacity

The automatic value of a CPU limit of 0 is worked out at enrollment. Once the host is enrolled, a 0 stands for the limit Gigahost accepted, and the agent never lowers or raises the limit by itself, except in update-capacity, where a CPU limit of 0 takes every CPU of the smallest allowed node (see Raise a host’s capacity).

Paths

The agent keeps its state, journal, and telemetry spool in /var/lib/gigahost (state.json, journal.sqlite3, telemetry.sqlite3), and VM state and the image cache in /var/lib/gigahost/instances and /var/lib/gigahost/images, the directories the dependency installer creates.

With state_directory, which prepare-state makes on the root pool’s drive, the state, journal, telemetry spool, VM state and container kernels are in that directory instead (state.json, journal.sqlite3, telemetry.sqlite3, instances/, kernels/), and the image and container caches in /var/cache/ghostrack (images/, containers/). The directory also holds pool.json, which names its root pool, and a copy of the configuration the agent last started with. The agent refuses to start when the directory is not on the root pool’s filesystem, or, for a thin-LVM root pool, when what is mounted there is not that pool’s ghostrack-state volume, and when image_cache_directory or container_cache_directory names a directory inside it.

You only need to set state_file, journal_file, telemetry_spool_file, instance_directory, or image_cache_directory to move one elsewhere. A relative path is resolved against the configuration file’s directory.

The agent’s filesystem

The agent also watches the filesystem of its state. With less than 10% of it free you get an incident, and with less than 5% the host is degraded and takes no new rentals until more is free. The host’s Overview and ghost provider host metrics show its size and free space.

The backend URL

backend_url defaults to https://api.gigahost.cloud when omitted.

• A plain HTTP URL is accepted only when its host is a literal IPv4 or IPv6 address for testing. http://localhost and every other hostname are not accepted.
• Enrollment and control requests never follow redirects.
• The agent does not resolve the telemetry hostname. Gigahost alone resolves telemetry.gigahost.cloud and publishes its current IPs in the signed discovery manifest used for HTTP bulk traffic.

The image cache

The agent caches the operating-system images it downloads, by default on the filesystem that also holds its state and, on most hosts, the operating system. image_cache_max_bytes bounds the cache.

• Before a download and once an hour the agent evicts the images used the longest ago until the cache fits.
• An image unused for 30 days goes in any case, except on a host that downloads the catalog ahead of demand (below).
• An image that a job is using is never evicted, and a VM’s disk is a copy, so an evicted image only costs a download when it is next used.
• A download needs the image and 5 GiB beside it free on the cache’s filesystem. Cached images that no job uses make that room too, the least recently used first. Only when even they would not do does the creation fail, with a message that says so, before anything is written.

Downloads ahead of demand

image_predownload = true
image_predownload_rate_bps = 10000000

A home host downloads the catalog’s images ahead of demand, so that a new or reinstalled VM does not wait for its image over your uplink. A cloud host does not, unless you set it.

• The agent asks Gigahost every 15 minutes which images to keep: the enabled catalog images your largest VM size can take, those most VMs were created from in the last 30 days first.
• It downloads them one at a time, each only while it fits within image_cache_max_bytes beside the images already cached and leaves the larger of 5 GiB and a fifth of the filesystem free. It evicts nothing for them. An image that does not fit is left out, and a smaller one may still fit. The whole catalog takes about 10.4 GB, so the default limit holds it.
• The downloads take at most half the download bandwidth your host’s network tests measured, or image_predownload_rate_bps while there is no measurement. They pause while a VM’s own image or a restore’s backup downloads or a network test runs, then continue where they stopped.
• Each image is verified by its checksum before it enters the cache, and one that is withdrawn from the catalog or replaced by a newer build is removed within about 15 minutes, once no job uses it.
• The cached images stay however long they go unused, and a VM’s own download evicts the images outside the plan first.

ghost provider host get shows the cache (image_cache): what the images take, the filesystem’s size and free space, the pace, and each image with its state, cached, downloading, pending, left_out or failed, and why.

Backup transfers

Setting backup_upload_rate_bps keeps backups from crowding out guest traffic. On a home host, a value of what its upload can spare works well. At 50 Mbit/s a backup of 30 GB takes about 80 minutes to upload.

• Local and hosted backups both stream through two reusable 64 MiB transfer buffers, and no complete raw staging disk is created.
• All-zero regions of a disk are stored without data.
• Backups written by current agents use a container format that older agents cannot restore, although current agents restore older backups. Updating the agent when a new release is published avoids this.
• A hosted backup that may grow beyond about 625 GiB uploads in larger parts, the smallest whole number of MiB that fits it into S3’s 10,000 parts, and its two buffers grow to that size, so Gigahost accepts larger disks too.
• Running guests use libvirt pull mode and therefore need temporary copy-before-write scratch capacity in the root pool (the backup scratch).
• Restore recreates the root and writes into it directly, without a restore staging volume.

Guest disks

Gigahost does not encrypt live VM root volumes.

• Enrollment and inventory refresh need thin-LVM pool zeroing, which prevents newly provisioned blocks from exposing previous guest data without writing the entire virtual disk.
• Qcow2 roots are always fresh, use preallocation=falloc, and have no backing file.
• Image conversion may punch sparse ranges, so the agent fallocates the complete populated file again and verifies its physical allocation before boot. This reserves capacity without a full-volume write.
• Backup artifacts remain compressed and encrypted independently.

Console logs

libvirt’s virtlogd, which libvirt uses by default, writes what each guest prints on its serial console to console.log in the guest’s directory under /var/lib/gigahost/instances, with mode 0600.

• The agent keeps at most 4 MiB of it per guest, and every ten seconds sends the end of each log that changed, up to 64 KiB for each of eight guests, for the renter to read.
• When it starts a guest again, it first keeps the end of what the guest wrote since its previous start, up to 64 KiB, in console.boot beside the log, also mode 0600, and sends that once too, so a report takes at most about 1 MiB.
• sudo virsh console still attaches to a running guest’s console.
• A renter’s live console session runs virsh console --force through the agent, so it takes the console over from a virsh console of yours, and yours takes it back; the session it took the console from is told and closes. The agent never logs what a console shows. A break the renter sends goes through the guest’s monitor (virsh qemu-monitor-command … chardev-send-break).

Metrics

Beside the guests’ metrics, the agent reports the host’s own every minute:

• CPU busy and steal time overall and per NUMA node, and load;
• memory and hugepages;
• each storage pool’s usage;
• disk I/O per pool device, counted above any RAID, so a mirrored write counts once;
• SMART health per physical disk;
• traffic on the uplink;
• temperatures where sensors exist;
• the memory errors the kernel has counted.

The readings appear under Status on the host’s Overview tab on the website (/provider/hosts/HOST_ID), traffic in the bit/s unit it fills (kbit/s, Mbit/s, Gbit/s) and disk rates in binary units such as MiB/s, with the history on its Usage tab, and through ghost provider host get, metrics, and instance-usage.

Host conditions

The agent judges its readings and reports what is worth your attention as a condition of the host, which you get as an incident for as long as it lasts (what a condition does).

Condition	Warning	Critical
CPU temperature	5 °C below the CPU’s own limit; 98 °C on an AMD CPU, which names none	the CPU’s own limit; 105 °C on an AMD CPU
Disk temperature (NVMe)	the drive’s warning temperature	the drive’s critical temperature
Memory module temperature	75 °C	the module’s critical temperature, 85 °C
Disk health		its SMART self-assessment failed
Disk wear	80% of its rated endurance used	95%
Disk media errors	the count rose within the last 24 hours
Memory errors	corrected errors were counted within the last 24 hours	an error could not be corrected since the host started
Hardware RAID array	it is being rebuilt, or its controller names a state the agent does not know	it is degraded, failed or offline
Hardware RAID controller	the state of its arrays cannot be read
Root pool		less than 5% free
Backup pool	missing: its directory is gone, or on the root filesystem or another pool’s filesystem, or its volume group or thin pool cannot be read
Agent filesystem	less than 10% free	less than 5% free
Memory		less than 2% available
Accepted capacity		a NUMA node has fewer CPUs or less memory, or the root pool less room, than the capacity Gigahost accepted for the host; checked at every start of the agent too
Guest shadow stacks		guests would get CET shadow stacks, and the host does not run with clearcpuid=519
Shared IP port in use (cloud hosts)	a service listens publicly on a port from 1000 to 60999 of the host’s address, which are Gigahost’s for VMs that share it (Shared IP)

Temperatures

A temperature counts once three samples in a row, three minutes, were at or over its limit.

• Only the sensors of the CPU (k10temp, coretemp), of NVMe drives, and of memory modules (spd5118, jc42) are judged.
• The sensor chips of mainboards are shown and not judged: an input that nothing is connected to reads over 100 °C for good.
• A Ryzen 7000 or 9000 holds 95 °C under full load by design, which is no condition.

Memory errors

Memory errors are counted only on a host whose memory corrects errors (ECC) and whose kernel has the EDAC driver of its memory controller loaded. On every other host the host’s Overview says that they are not available, which is not the same as none. gigahost-install-hypervisor-deps --check tells which of these sensors the host reads and which module to load for one that it does not, without failing for it.

CET shadow stacks

sudo gigahost-hypervisor shadow-stacks says what guests get, and exits with:

Status	Meaning
0	Guests get none, or the agent turns them off
2	The parameter is needed and GRUB’s configuration lacks it
3	The configuration has it but the host has not started since

--add-boot-parameter writes /etc/default/grub.d/90-gigahost-no-shadow-stacks.cfg and runs update-grub. The parameter takes effect when the host starts again, which you can schedule like any reboot. Linux then marks its kernel tainted, a warning that changes nothing else. Until the host runs with the parameter, it has the critical condition guest_shadow_stacks and takes no new rentals.

Inventory

The running agent recollects inventory on startup and every 15 minutes. It publishes a new revision only when CPU, NUMA, memory, or usable storage changes, or when the parts of 100.64.0.0/10 that other software on a home host uses change (see Tunnel addresses). Revision state is written durably, so retries after a crash are idempotent.

Work slots

The agent advertises independent free slots for provisioning, backup, restore, upload, and maintenance work, along with the exact step kinds supported by its binary.

• Gigahost leases only compatible work and atomically claims every resource a step touches, such as its instance or backup, so two steps never work on the same one at once.
• Storage pools are not claimed. The free slots bound how much work runs on them.
• On every host the steps that move a whole disk, a backup’s upload or local write and a restore’s writing, share two slots, and Gigahost starts no more than two backups and restores of a host at a time. The other backup steps, such as starting a backup or publishing it, run beside them.

The control protocol

Enrollment also agrees with Gigahost on a version of the control protocol. When Gigahost no longer speaks an agent’s version, it does not accept the agent’s requests until the agent is updated, and it names the release to install, as it does for a release it no longer serves (support window).

• The agent logs this once as an error, keeps its VMs running, and asks again only with its heartbeat, at the heartbeat interval.
• The install command updates it and restarts it.
• sudo gigahost-hypervisor check shows the version the agent speaks, whether Gigahost turned it away, and whether Gigahost has the agent’s message types.