NFS on Raspberry Pi 目前正学习如何在树莓派4集群上结合gromacs+mpich的分子动力学模拟，所以需要搭建一款分布式存储系统，又由于树莓派性能的限制，搭建OpenEBS会比较浪费计算资源，最后就采用轻量级的NFS来完成文件系统的共享和存储。
NFS on K3s aarch64 基础依赖 NFS Server:
sudo apt install nfs-kernel-server 添加共享文件目录到/etc/exports:
/mnt/data/nfs 192.168.1.0/24(rw,sync,no_root_squash,no_subtree_check) # 外部使用 /mnt/data/kubedata 192.168.1.0/24(rw,sync,no_root_squash,no_subtree_check) # k3s 使用 启动 NFS Server:
sudo systemctl restart nfs-kernel-server NFS Client:
sudo apt install nfs-common sudo mount 192.168.1.145:/mnt/data/nfs ./nfs 添加到/etc/fstab:
192.168.1.145:/mnt/data/nfs /home/chenfeng/nfs nfs auto,nofail,noatime,nolock 0 0 K3s NFS Volume 需要使用Kubernets提供的NFS Client Provisioner
k3s是跑在Ubuntu 20.04 aarch64 系统上的，而官方提供的NFS Client Provisioner是基于ARM v7的，所以需要重新编译Provisioner。
以下的操作全部在Raspberry pi 4上完成的。
以下为PATH:
diff -ur ./Makefile /tmp/nfs-client/Makefile --- ./Makefile 2020-06-28 07:47:43.883181030 +0000 +++ /tmp/nfs-client/Makefile 2020-06-28 10:06:00.588586966 +0000 @@ -28,17 +28,16 @@ container: build image build_arm image_arm build: - CGO_ENABLED=0 GOOS=linux go build -a -ldflags '-extldflags "-static"' -o docker/x86_64/nfs-client-provisioner ./cmd/nfs-client-provisioner + go build -a -ldflags '-extldflags "-static"' -o docker/x86_64/nfs-client-provisioner ./cmd/nfs-client-provisioner build_arm: - CGO_ENABLED=0 GOOS=linux GOARCH=arm GOARM=7 go build -a -ldflags '-extldflags "-static"' -o docker/arm/nfs-client-provisioner ./cmd/nfs-client-provisioner + go build -a -ldflags '-extldflags "-static"' -o docker/arm/nfs-client-provisioner ./cmd/nfs-client-provisioner image: docker build -t $(MUTABLE_IMAGE) docker/x86_64 docker tag $(MUTABLE_IMAGE) $(IMAGE) image_arm: - docker run --rm --privileged multiarch/qemu-user-static:register --reset docker build -t $(MUTABLE_IMAGE_ARM) docker/arm docker tag $(MUTABLE_IMAGE_ARM) $(IMAGE_ARM) Only in /tmp/nfs-client/deploy: .deployment-arm.yaml.swp diff -ur ./deploy/deployment-arm.yaml /tmp/nfs-client/deploy/deployment-arm.yaml --- ./deploy/deployment-arm.yaml 2020-06-28 09:24:48.499572298 +0000 +++ /tmp/nfs-client/deploy/deployment-arm.yaml 2020-06-28 10:06:11.876117004 +0000 @@ -4,7 +4,8 @@ name: nfs-client-provisioner labels: app: nfs-client-provisioner - namespace: nfs + # replace with namespace where provisioner is deployed + namespace: default spec: replicas: 1 strategy: @@ -20,7 +21,7 @@ serviceAccountName: nfs-client-provisioner containers: - name: nfs-client-provisioner - image: 192.168.1.114:5000/nfs-client-provisioner-arm + image: quay.io/external_storage/nfs-client-provisioner-arm:latest volumeMounts: - name: nfs-client-root mountPath: /persistentvolumes @@ -28,11 +29,11 @@ - name: PROVISIONER_NAME value: fuseim.pri/ifs - name: NFS_SERVER - value: 192.168.1.145 + value: 10.10.10.60 - name: NFS_PATH - value: /mnt/data/kubedata + value: /ifs/kubernetes volumes: - name: nfs-client-root nfs: - server: 192.168.1.145 - path: /mnt/data/kubedata + server: 10.10.10.60 + path: /ifs/kubernetes diff -ur ./deploy/test-pod.yaml /tmp/nfs-client/deploy/test-pod.yaml --- ./deploy/test-pod.yaml 2020-06-28 09:36:06.872994438 +0000 +++ /tmp/nfs-client/deploy/test-pod.yaml 2020-06-28 10:06:11.920115180 +0000 @@ -5,7 +5,7 @@ spec: containers: - name: test-pod - image: 192.168.1.114:5000/ubuntu:20.04 + image: gcr.io/google_containers/busybox:1.24 command: - "/bin/sh" args: diff -ur ./docker/arm/Dockerfile /tmp/nfs-client/docker/arm/Dockerfile --- ./docker/arm/Dockerfile 2020-06-28 07:47:43.759177680 +0000 +++ /tmp/nfs-client/docker/arm/Dockerfile 2020-06-28 10:06:00.628585293 +0000 @@ -12,7 +12,6 @@ # See the License for the specific language governing permissions and # limitations under the License. -FROM hypriot/rpi-alpine:3.6 -RUN apk update --no-cache && apk add ca-certificates +FROM ubuntu:20.04 COPY nfs-client-provisioner /nfs-client-provisioner ENTRYPOINT ["/nfs-client-provisioner"] nfs-client/cmd/nfs-client-provisioner为Provisioner源码目录。
...
