Fedora Coreos Installing K3s With Ignition

So i wanted to experiment a bit with K3s, IPv6 and Cilium for that i want to try it with a clean environment each time, that was my excuse to try fiddling with Ignition again.

Also i don’t want to just curl the install script into bash as a oneshot service, instead i downloaded the script ran it with bash -x and came up with this (shortened) Butane config:

 1variant: fcos
 2version: 1.5.0
 3systemd:
 4  units:
 5    - name: rpm-ostree-install-k3s-selinux.service
 6      enabled: true
 7      contents: |
 8        [Unit]
 9        Description=Layer k3s-selinux with rpm-ostree
10        Wants=network-online.target
11        After=network-online.target
12        Before=zincati.service
13        ConditionPathExists=!/var/lib/misc/%N.stamp
14
15        [Service]
16        Type=oneshot
17        RemainAfterExit=yes
18        ExecStart=/usr/bin/rpm-ostree --idempotent install --apply-live --allow-inactive --assumeyes k3s-selinux
19        ExecStart=/bin/touch /var/lib/misc/%N.stamp
20        ExecStartPost=/usr/sbin/restorecon -Rv /usr/local/bin
21
22        [Install]
23        WantedBy=multi-user.target
24    - name: k3s.service
25      enabled: true
26      contents: |
27        [Unit]
28        Description=Lightweight Kubernetes
29        Documentation=https://k3s.io
30        ConditionPathExists=/usr/share/selinux/packages/k3s.pp
31        Wants=network-online.target
32        After=network-online.target
33
34        [Install]
35        WantedBy=multi-user.target
36
37        [Service]
38        Type=notify
39        EnvironmentFile=-/etc/default/%N
40        EnvironmentFile=-/etc/sysconfig/%N
41        EnvironmentFile=-/etc/systemd/system/k3s.service.env
42        KillMode=process
43        Delegate=yes
44        # Having non-zero Limit*s causes performance problems due to accounting overhead
45        # in the kernel. We recommend using cgroups to do container-local accounting.
46        LimitNOFILE=1048576
47        LimitNPROC=infinity
48        LimitCORE=infinity
49        TasksMax=infinity
50        TimeoutStartSec=0
51        Restart=always
52        RestartSec=5s
53        ExecStartPre=/bin/sh -xc '! /usr/bin/systemctl is-enabled --quiet nm-cloud-setup.service'
54        ExecStartPre=-/sbin/modprobe br_netfilter
55        ExecStartPre=-/sbin/modprobe overlay
56        ExecStart=/usr/local/bin/k3s \
57            server \
58          '--selinux'
59storage:
60  files:
61    - path: /usr/local/bin/k3s
62      mode: 0755
63      contents:
64        source: https://github.com/k3s-io/k3s/releases/download/v1.27.5+k3s1/k3s
65        verification:
66          hash: sha256-aa3387f1e01e21478d8042d6ab0cb25c299fe0920e2e30fc760ab36fb7a3d459
67    - path: /usr/local/bin/k3s-killall.sh
68      mode: 0755
69      contents:
70        local: k3s-killall.sh
71    - path: /usr/local/bin/k3s-uninstall.sh
72      mode: 0755
73      contents:
74        local: k3s-uninstall.sh
75    - path: /etc/systemd/system/k3s.service.env
76      mode: 0600
77      contents:
78        inline: ""
79    - path: /etc/yum.repos.d/rancher-k3s-common.repo
80      contents:
81        inline: |
82          [rancher-k3s-common-stable]
83          name=Rancher K3s Common (stable)
84          baseurl=https://rpm.rancher.io/k3s/stable/common/coreos/noarch
85          enabled=1
86          gpgcheck=1
87          repo_gpgcheck=0
88          gpgkey=https://rpm.rancher.io/public.key
89  links:
90    - path: /usr/local/bin/kubectl
91      target: /usr/local/bin/k3s
92    - path: /usr/local/bin/crictl
93      target: /usr/local/bin/k3s

The referenced k3s-uninstall.sh and k3s-killall.sh scripts are straight out of the normal install generated from the script.

Posts

Getting Back Into Kubernetes

Things that used to be

I used to run my own OpenShift Kubernetes cluster that hosted most of my Homelab. I dabbled with the technology for quite some time, trying different installation methods like manual setups, kubespray, Ansible playbooks, and the Ansible-based OpenShift 3.x installer. I experimented with various Kubernetes distributions, including Vanilla K8s, OpenShift, and RKE/Rancher, on different host OSs like Rancher OS, Fedora Atomic, CoreOS, and Fedora 24ish. However, most of these options are no longer maintained.