Hike News
Hike News

Syncthing K8s Setup on Ubuntu 20.04

Install the k8s cluster if not already installed

1
2
microk8s enable dns storage helm3
microk8s status

Pull the syncthing helm chart

1
2
3
microk8s helm repo add truecharts https://charts.truecharts.org/
microk8s helm repo update
microk8s helm pull truecharts/syncthing --version 15.0.1

Modify the values.yaml file to set the configuration

We’re going to set the syncthing configuration within the container.

1
2
3
tar -xvf ./syncthing-15.0.1.tgz
cd syncting
cat values.yamls

Make the data directory

1
2
3
4
5
mkdir /path/to/
chown -R 777 /path/to/

mkdir /path/to/data
chown -R 777 /path/to//data

Create Persistent Volumes and Persistent Volume Claims

If the Syncthing configuration files are on a different disk than your files and the harddisk containing your files fails to mount, Syncthing can think you deleted all your files. Syncthing will then tell the other nodes to delete all files in that directory. To prevent this issue, we will store the configuration along with the files. If the harddisk does not mount the entire node will disappear and will not corrupt any other Syncthing nodes.

Persistent Volumes

Config

syncthing-server-config-pv.yaml

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
apiVersion: v1
kind: PersistentVolume
metadata:
  name: syncthing-server-config-pv
spec:
  capacity:
    storage: 100Gi
  volumeMode: Filesystem
  accessModes:
  - ReadWriteOnce
  persistentVolumeReclaimPolicy: Retain
  storageClassName: local-storage
  local:
    path: /path/to/ # This must exist on the host. This is the path above the data directory 
  nodeAffinity:
    required:
      nodeSelectorTerms:
      - matchExpressions:
        - key: kubernetes.io/hostname
          operator: In
          values:
          - Hostname

microk8s kubectl apply -f syncthing-server-config-pv.yaml

Data

syncthing-server-data-pv.yaml

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
apiVersion: v1
kind: PersistentVolume
metadata:
  name: syncthing-server-data-pv
spec:
  capacity:
    storage: 3Ti
  volumeMode: Filesystem
  accessModes:
  - ReadWriteOnce
  persistentVolumeReclaimPolicy: Retain
  storageClassName: local-storage
  local:
    path: /path/to/data # This must exist on the host
  nodeAffinity:
    required:
      nodeSelectorTerms:
      - matchExpressions:
        - key: kubernetes.io/hostname
          operator: In
          values:
          - Hostname

microk8s kubectl apply -f syncthing-server-data-pv.yaml

microk8s kubectl get pv 

1
2
3
NAME                CAPACITY   ACCESS MODES   RECLAIM POLICY   STATUS   CLAIM                               STORAGECLASS   REASON   AGE                             
syncthing-server-data-pv  3Ti   RWO           Retain           Bound    default/syncthing-server-data-pvc   local-storage   4h41m
syncthing-server-config-pv100Gi RWO           Retain           Bound    default/syncthing-server-config-pvc local-storage   4h38m

Persistent Volume Claims

Config

syncthing-server-config-pvc.yaml
Bonds to syncthing-server-config-pv

1
2
3
4
5
6
7
8
9
10
11
12
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: syncthing-server-config-pvc
spec:
  storageClassName: local-storage # Empty string must be explicitly set otherwise default StorageClass will be set
  accessModes:
    - ReadWriteOnce
  volumeName: syncthing-server-config-pv
  resources:
    requests:
      storage: 100Gi

microk8s kubectl apply -f syncthing-server-config-pvc.yaml

Data

syncthing-server-data-pvc.yaml
Bonds to syncthing-server-data-pv

1
2
3
4
5
6
7
8
9
10
11
12
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: syncthing-server-data-pvc
spec:
  storageClassName: local-storage # Empty string must be explicitly set otherwise default StorageClass will be set
  accessModes:
    - ReadWriteOnce
  volumeName: syncthing-server-data-pv
  resources:
    requests:
      storage: 3Ti

microk8s kubectl apply -f syncthing-server-data-pvc.yaml

microk8s kubectl get pvc

1
2
3
NAME                          STATUS   VOLUME                                     CAPACITY   ACCESS MODES   STORAGECLASS        AGE
syncthing-server-data-pvc     Bound    syncthing-server-data-pv                   3Ti        RWO            local-storage       4h39m
syncthing-server-config-pvc   Bound    syncthing-server-config-pv                 100Gi      RWO            local-storage       4h34m

Update the values file

1
nano values.yamls

Update the persistence section as follows

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
image:
  repository: tccr.io/truecharts/syncthing
  pullPolicy: IfNotPresent
  tag: v1.23.7@sha256:f2dd1847b81302b4f07e0b50019d958b1b77e7fd5de689972eaa0fd9f0d27d41

workload:
  main:
    podSpec:
      containers:
        main:
          probes:
            liveness:
              path: /rest/noauth/health
            readiness:
              path: /rest/noauth/health
            startup:
              path: /rest/noauth/health

service:
  main:
    ports:
      main:
        port: PORTA
        targetPort: PORTA
        type: NodePort
  listeners:
    enabled: true
    ports:
      tcp:
        enabled: true
        port: PORTB
        targetPort: PORTB
        type: NodePort
  listeners-udp:
    enabled: true
    ports:
      udp:
        enabled: true
        port: PORTB
        targetPort: PORTB
        protocol: udp
        type: NodePort
  discovery:
    enabled: true
    ports:
      discovery:
        enabled: true
        port: PORTC
        targetPort: PORTC
        protocol: udp
        type: NodePort

persistence:
  config:
    enabled: true
    mountPath: "/var/syncthing"
    type: pvc
    existingClaim: syncthing-server-config-pvc
  data:
    enabled: true
    mountPath: "/var/syncthing/data"
    type: pvc
    existingClaim: syncthing-server-data-pvc

portal:
  open:
    enabled: true

Install the helm chart with our values.yaml file

microk8s helm install syncthing-server truecharts/syncthing –values ./values.yaml

Check the status of the installed application.

1
2
3
4
5
microk8s status
microk8s kubectl describe pods
microk8s kubectl
microk8s kubectl logs
microk8s kubectl describe pods

Common Errors

1
Error from server (BadRequest): container in pod is waiting to start: ContainerCreating

You probably need to change the permissions on the PV directory. This path is what is written in the PersistentVolume variable. A quick chmod -R 777 to this path will most likely fix the issue. The conatiner should update the permissions once it runs for the first time.

Service yaml file

We need to expose the service to the outside world.
Thankfully microk8s has a built in loadbalancer called metallb

PORTA TCP – Webserver/HTTP
PORTB TCP/UDP – TCP - TCP based sync protocol traffic, UDP - QUIC based sync protocol traffic
PORTC UDP – for discovery broadcasts on IPv4 and multicasts on IPv6

syncthing-server-service.yaml 

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
apiVersion: v1
kind: Service
metadata:
  name: syncthing-server-service
spec:
  type: LoadBalancer
  selector:
    app.kubernetes.io/name: syncthing
  ports:
    - name: http
      protocol: TCP
      port: PORTA
      targetPort: PORTA
    - name: sync-tcp
      protocol: TCP
      port: PORTB
      targetPort: PORTB
    - name: sync-udp
      protocol: UDP
      port: PORTB
      targetPort: PORTB
    - name: discovery
      protocol: UDP
      port: PORTC
      targetPort: PORTC
  externalIPs:
    - A.B.C.D

More information here
https://medium.com/swlh/kubernetes-external-ip-service-type-5e5e9ad62fcd

Apply the service

1
microk8s kubectl apply -f ./syncthing-server-service.yaml

Confirm the service is active

1
microk8s kubectl describe services syncthing-server-service

Firewall Rules

This assumes you are using ufw.
ufw is bascally a wrapper for IPTABLES. If you have ever used IPTABLES before you understand why ufw exists.
https://docs.syncthing.net/users/firewall.html

1
2
3
4
5
6
7
8
9

sudo ufw default allow routed 

sudo ufw allow from A.B.C.D/24 to any port PORTA proto tcp
sudo ufw allow from 0.0.0.0/24 to any port PORTB proto tcp
sudo ufw allow from 0.0.0.0/24 to any port PORTB proto udp
sudo ufw allow from 0.0.0.0/24 to any port PORTC proto udp

sudo ufw status

External GUI Access

1
2
3
4
5
6
sudo nano PATH-FROM-syncthing-server-config-pv-FILE/config.xml
    <gui enabled="true" tls="true" debugging="false">
        <address>A.B.C.0:PORTA</address>
        <apikey>RANDOMSTRING</apikey>
        <theme>default</theme>
    </gui>

Restart Syncthing to pickup gui changes

microk8s kubectl get pods

microk8s kubectl exec syncthing-server-RANDOMHASH – ps -A

1
2
3
4
5
PID   USER     TIME  COMMAND
    1 568       0:00 /bin/syncthing
   16 568       0:08 /bin/syncthing
   33 568       0:00 ps -A

We want to kill the syncthing process. This will cause the container to restart and make syncthing load our config changes. 

1
microk8s kubectl exec syncthing-server-RANDOMHASH kill-9  16

References

https://kubernetes.io/docs/concepts/services-networking/service/