Why a workload lands where it lands
The homelab runs workloads on three active runtimes — the on-prem Talos production cluster, the Hetzner edge cluster, and a plain Docker host on the Synology NAS — plus one retired runtime, the K3s mini-server cluster, that still shapes the docs. Every placement decision follows from a small set of questions, and the answer is encoded in the directory layout of k8s/ rather than in anyone's head.
This page explains both halves: why a workload goes where it goes, and how the base/talos/edge overlay pattern writes that decision down.
The runtimes involved:
| Runtime | Where it lives in docs |
|---|---|
| Talos production cluster (4× NUC, Proxmox VMs) | foundation/cluster-talos · hardware/talos |
| Edge cluster (1× Hetzner cx33) | foundation/cluster-edge · foundation/hetzner |
Docker host maresa (Synology DS723+) | foundation/docker · hardware/hot_storage |
| K3s cluster (retired) | foundation/k3s · hardware/k3s |
| Flux, reconciling both clusters | foundation/flux |
| Ansible, configuring the non-cluster hosts | foundation/ansible |
The topology, top-down
The Ansible inventory confirms the cast: proxmox-1/2/3/4 at 192.168.100.51–54 carry the Talos VMs, lxc-proxmox{1,2,3,4}-netbird are the mesh connectors beside them, maresa at 192.168.0.5 is the Synology, and home-assist at 192.168.2.20 is a standalone appliance. The edge node is not in the inventory at all — it's Talos, there is no SSH.
The placement questions
Four questions decide where a workload runs, asked in order:
| Question | If yes → |
|---|---|
| Must it keep working when both clusters are down? (DNS, dynamic DNS, mesh connectivity, backup replication) | Docker on maresa |
| Is it the public entry point, or does it need internet ingress with its own blast radius? | Edge cluster |
| Does it have persistent state, a database, a GPU need, or bulk NFS media? | Talos cluster |
| Is it a throwaway experiment? | k3d on a workstation, never a real cluster |
The defaults fall out of the answers: talos is where workloads go unless there's a reason not to — it has Longhorn for replicated block storage, CNPG with anti-affinity across nodes, the Intel iGPU plugin for transcoding, and NFS into the TrueNAS box. The edge cluster is deliberately starved of those capabilities so nothing stateful can accidentally settle there. And maresa runs exactly the services whose failure mode "the cluster is down" must not affect: the NAS is the Syncthing backup target, AdGuard is the LAN's DNS, the NetBird agent is the site's mesh presence.
K3s answers none of these questions anymore — the cluster was retired and its role absorbed by talos. The foundation/k3s page documents why (immutability, API-driven config, no base OS, one-step upgrades).
What actually runs where
Talos (k8s/apps/*/talos/, fifty overlays): everything with state or hardware needs. Media — Jellyfin, Immich, Navidrome, Tube Archivist, the arr bundle (Sonarr, Radarr, Lidarr, Prowlarr, Whisparr, Seerr, Flaresolverr in one namespace). Productivity — Nextcloud, Outline, Paperless, Vaultwarden, Vikunja. DevOps — Gitea + gitea-runner, Keycloak, n8n, monitoring. Plus 18 platform controllers (k8s/platform/*/talos/), including the talos-only ones: Longhorn, k8up, External Secrets, Kyverno, Crossplane, Spegel, Tetragon, NetBird operator, the GPU plugin.
Edge (k8s/apps/*/edge/, three overlays): netbird-client (a plain mesh peer that keeps the edge represented on NetBird; the edge → production traffic itself runs through the Envoy sidecars), openbeehive, and seitenraum — small public-facing apps whose entire state fits in a single-node CNPG on local-path storage. The platform layer (k8s/platform/*/edge/) is the minimum needed to terminate the internet: Cilium, Envoy Gateway, cert-manager + the all-inkl DNS-01 webhook, CNPG, local-path-provisioner, metrics-server, kubelet-serving-cert-approver. Eight controllers versus talos's eighteen — no Longhorn, no k8up, no Kyverno. The asymmetry is the point: losing the cx33 loses ingress, nothing else, and recovery is minutes.
Maresa (docker/maresa/): seven Compose services — Traefik (internal *.maresa.int.kueber.eu TLS), AdGuard Home (LAN DNS on port 53), ddclient (dynamic DNS), the NetBird agent, Syncthing (the hot-backup mover — see Backups end-to-end), Portainer (seeded via the root docker-compose.yaml, then managing the rest), and a family dashboard. Digest-pinned images, host volumes under /volume1/docker/<app>, no orchestrator.
How the directory layout encodes placement
The k8s/ tree has four load-bearing directories, each split the same way:
k8s/
├── apps/ base/ talos/ edge/ ← workload manifests
├── platform/ base/ talos/ edge/ ← controllers + CRDs
├── components/ base/ talos/ edge/ ← reusable Kustomize Components
├── resources/ base/ talos/ ← shared routes, NFS storage
└── clusters/ talos/ edge/ ← the Flux entry points
The rule is uniform: inside each app folder, base/ holds the cluster-agnostic definition and talos/ or edge/ holds the placement decision. A workload runs on a cluster if — and only if — its folder contains that cluster's overlay dir and k8s/clusters/<cluster>/ has a Flux Kustomization wiring it in. Placement is still greppable, from both directions: ls k8s/apps/<app>/ answers "where does this run?", and ls -d k8s/apps/*/edge is the authoritative list of edge workloads.
Apps: base defines, overlay places
k8s/apps/<app>/base/ contains the Deployment and nothing environment-specific. The cluster overlay adds everything that depends on where it runs. Vaultwarden's talos overlay (k8s/apps/vaultwarden/talos/kustomization.yaml) is the canonical shape:
namespace: vaultwarden
components:
- network-policies # → components/app-network-policy/talos
- workspace # → components/workspace/talos
- cnpg # → components/cnpg/talos
- backups # → components/backups/talos
resources:
- ../../base/vaultwarden
- http-route.yaml # hostname on this cluster's gateway
- pvc.yaml # this cluster's storage class
- namespace.yaml
- secret-longhorn.yaml
- secret-vaultwarden.yaml
patches:
- path: vaultwarden-patch.yaml
The same base can be placed on both clusters with different dressing — openbeehive and seitenraum exist in apps/base/ and are instantiated only from apps/edge/; Vaultwarden only from apps/talos/. Some talos overlays have no base at all when they're pure aggregations, like apps/talos/arr/, which composes seven apps/base/ services into one namespace.
Components: same name, per-cluster meaning
The components chain is where placement gets teeth. An app overlay references cnpg, which chains to components/cnpg/<cluster>, which layers a cluster patch over components/cnpg/base. The two patches diverge exactly where the clusters do:
Concern (cnpg component) | components/cnpg/talos/ | components/cnpg/edge/ |
|---|---|---|
| Storage class | longhorn-db-single-encrypted | local-path |
| Instances | 2, required anti-affinity across nodes, switchover updates | 1 (base default) |
Same component name in the app's kustomization.yaml, different guarantees underneath — the app doesn't know which cluster it's on. The component catalog also differs: edge offers only app-network-policy, backups, cnpg, and valkey; talos additionally has minio, mongodb, nats, workspace, email, and backups-cnpg. An app that needs MongoDB or NATS cannot be placed on edge — the overlay wouldn't build. Capability gaps are compile-time errors, not runtime surprises.
Platform: the same pattern, one level down
Each controller folder's base/ holds the definition (seventeen of them); its talos/ and edge/ dirs pick it up and add per-cluster Helm values (cert-manager-values.yaml etc.). Edge picks eight, talos picks eighteen. The overlay that isn't there is the decision: there is no platform/longhorn/edge, so no PVC on edge can ever be Longhorn-backed.
Clusters: where Flux reads the answer
k8s/clusters/talos/ and k8s/clusters/edge/ are the two Flux roots — one file per workload. Platform controllers come straight from Git (sourceRef: GitRepository, path: ./k8s/platform/<name>/<cluster>). Apps come from per-app OCI artifacts (oci://gitea.web.kueber.eu/johnny/oci-talos-app-<name> or oci-edge-app-<name>), cosign-verified and SOPS-decrypted in-cluster. Every app Kustomization carries dependsOn: [infra-configs], and infra-configs in turn dependsOn every controller with wait: true — so on either cluster, apps only reconcile once that cluster's full platform layer is healthy. The mechanics of that loop are the subject of GitOps flow.
Maresa sits outside all of this: docker/maresa/ is plain Compose, deployed by hand/Portainer, with the Docker engine coming from Synology's Container Manager rather than from Ansible. That's deliberate — the host that must survive a GitOps outage should not depend on GitOps.
What can break, and where to look
| Symptom | Most likely cause | Where to look first |
|---|---|---|
| App builds locally but Flux won't place it on edge | Overlay references a component edge doesn't have (minio, nats, …) | ls -d k8s/components/*/edge vs the app's components: list |
| Public URL down, app healthy in-cluster | Edge → production forwarding, not placement | Real client IPs across the chain · fabric/netbird |
| App deployed to edge starts losing data on node replacement | It's on local-path storage — edge is disposable by design | Move it to talos; see cluster-edge |
| DNS/ad-blocking dead during a cluster incident | Shouldn't happen — AdGuard is on maresa, not in a cluster | apps/adguard-home · the Synology itself |
| New app reconciles before its database exists | Missing dependsOn: infra-configs in k8s/clusters/<cluster>/<app>.yaml | The cluster-root Kustomization file |
| Whole cluster gone | Different recovery per cluster: edge is minutes, talos is a drill | Disaster recovery |
Why this is a topic, not a foundation page
Each runtime already has its own single-source-of-truth page — cluster-talos, cluster-edge, docker, k3s. What none of them owns is the seam: the decision procedure between them and the overlay convention that makes the decision durable. When you add workload number fifty-one, this page tells you which directory it belongs in; the foundation pages tell you what it will find when it gets there.