RustFS 分布式文件系统面试题库

graph TD
    Client["Client SDK / FUSE / CSI"]
    Client --> Meta
    Client --> Data
    subgraph Cluster["RustFS Cluster"]
        Meta["Metadata Server (Raft Cluster)<br/>- 元数据管理<br/>- 文件系统目录树<br/>- 分布式锁"]
        Data["Data Server x N (Object Storage Backend)<br/>- Chunk 存储 (64MB/Chunk)<br/>- 多副本 + EC 冗余<br/>- Checksum 完整性校验"]
    end

Write 请求流程：
Client ──write──> Leader (Raft) ──log replication──> Followers
                    │
                    ├── commit to LSM-Tree
                    ├── update In-Memory Cache
                    └── invalidate Client Lease (if exists)

Mounted as FUSE mount point ──> /mnt/rustfs
Kernel VFS ──> FUSE Module ──> RustFS Client Process ──> Metadata Server / Data Server

graph TD
    subgraph Master["Kubernetes Master"]
        SC["StorageClass (rustfs-csi-sc)<br/>provisioner: csi.rustfs.com"]
    end
    Master --> Worker
    subgraph Worker["Worker Node"]
        CSINode["CSI Node Plugin (DaemonSet)<br/>- NodeStageVolume / NodePublish<br/>- Mount FUSE to Pod"]
        RustFSClient["RustFS Client (FUSE Mount)<br/>/var/lib/kubelet/plugins/<br/>csi.rustfs.com/.../globalmount"]
        CSINode --> RustFSClient
    end

apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
  name: rustfs-csi-sc
provisioner: csi.rustfs.com
parameters:
  metaServerAddr: "rustfs-meta:9900"
  subPath: "${pvc.namespace}/${pvc.name}"
  mountOptions: "allow_other,default_permissions"
reclaimPolicy: Delete
volumeBindingMode: WaitForFirstConsumer

Client ──write(chunk)──> Primary Data Server
                            │
                            ├── local write (sync)
                            ├── replicate to Secondary-1 (parallel)
                            ├── replicate to Secondary-2 (parallel)
                            │
                            └── ack to Client (when >= Quorum=2 confirms)

# 添加 Data Server
rustfs-cli data-server add --endpoint 10.0.1.10:9800 --weight 100

# 下线 Data Server（安全排水）
rustfs-cli data-server decommission --endpoint 10.0.1.5:9800 --drain-timeout 1h

# 添加 Metadata Server 节点到 Raft 集群
rustfs-cli meta-server add-peer --endpoint 10.0.1.20:9900 --role voter

groups:
  - name: rustfs
    rules:
      - alert: RustFSMetaRaftLeaderMissing
        expr: absent(rustfs_meta_raft_is_leader == 1)
        for: 1m
        annotations:
          summary: "RustFS Metadata Raft 集群无 Leader"
      - alert: RustFSDataDiskUsageHigh
        expr: rustfs_data_disk_usage_bytes / rustfs_data_disk_total_bytes > 0.85
        for: 5m
        annotations:
          summary: "RustFS Data Server 磁盘使用率超过 85%"

graph TD
    subgraph Node1["Node-1"]
        Meta1["Meta-1<br/>(9900)"]
        Data1["Data-1<br/>(9800)"]
    end
    subgraph Node2["Node-2"]
        Meta2["Meta-2<br/>(9900)"]
        Data2["Data-2<br/>(9800)"]
    end
    subgraph Node3["Node-3"]
        Meta3["Meta-3<br/>(9900)"]
        Data3["Data-3<br/>(9800)"]
    end

# Metadata Server StatefulSet
apiVersion: apps/v1
kind: StatefulSet
metadata:
  name: rustfs-meta
spec:
  serviceName: rustfs-meta
  replicas: 3
  podManagementPolicy: Parallel
  template:
    spec:
      affinity:
        podAntiAffinity:
          requiredDuringSchedulingIgnoredDuringExecution:
            - labelSelector:
                matchLabels:
                  app: rustfs-meta
              topologyKey: kubernetes.io/hostname
      containers:
        - name: meta
          image: rustfs/meta:v0.3.0
          ports:
            - containerPort: 9900
              name: meta-rpc
          volumeMounts:
            - name: data
              mountPath: /var/lib/rustfs/meta
          resources:
            requests:
              cpu: "2"
              memory: 4Gi
            limits:
              cpu: "4"
              memory: 8Gi
  volumeClaimTemplates:
    - metadata:
        name: data
      spec:
        storageClassName: fast-ssd
        accessModes: ["ReadWriteOnce"]
        resources:
          requests:
            storage: 100Gi

graph TD
    App["Application"]
    App --> FUSE
    subgraph Client["RustFS Client"]
        FUSE["FUSE Layer (fuser crate)"]
        MetaCache["Meta Cache<br/>(LRU)"]
        ChunkCache["Chunk Cache<br/>(Read-Ahead + Write-Back)"]
        RPC["RPC Layer (gRPC)"]
        FUSE --> MetaCache
        FUSE --> ChunkCache
        MetaCache --> RPC
        ChunkCache --> RPC
    end
    RPC --> MetaServer["Metadata Server"]
    RPC --> DataServer["Data Server(s)"]

graph LR
    subgraph Active["活跃文件系统（Read-Write）"]
        RootV2["Root Inode<br/>(version 2)"]
        ChunkA_Active["Chunk A (shared)"]
        ChunkB_Active["Chunk B' (modified)"]
        RootV2 --> ChunkA_Active
        RootV2 --> ChunkB_Active
    end
    subgraph Snap["快照（Read-Only）"]
        RootV1["Root Inode<br/>(version 1)"]
        ChunkA_Snap["Chunk A (shared)"]
        ChunkB_Snap["Chunk B (original)"]
        RootV1 --> ChunkA_Snap
        RootV1 --> ChunkB_Snap
    end
    RootV2 -.->|"copy"| RootV1

rustfs-cli snapshot create /volumes/prod --name daily-20260126 --retention-days 7

rustfs-cli clone /volumes/prod --from-snapshot daily-20260126 --name prod-clone

dataServer:
  scrub:
    enabled: true
    interval: 168h          # 每周全量扫描
    bandwidthLimit: 50MB/s  # 扫描带宽限制
    autoRepair: true        # 自动修复不一致副本

graph TD
    subgraph Cluster["RustFS Cluster"]
        subgraph VolumeDev["Volume: dev"]
            SubVolA["SubVol-A<br/>Quota: 100Gi"]
            SubVolB["SubVol-B"]
        end
        subgraph VolumeProd["Volume: prod"]
            SubVolApp1["SubVol-app1<br/>Quota: 500Gi"]
            SubVolApp2["SubVol-app2<br/>Quota: 1Ti"]
        end
    end

# 设置目录容量配额 100 GiB，inode 配额 1000000
rustfs-cli quota set /volumes/prod/app1 \
  --capacity 100Gi \
  --capacity-hard 120Gi \
  --inodes 1000000 \
  --grace-period 7d

# 查看配额使用情况
rustfs-cli quota get /volumes/prod/app1

Step 1: Chunk 引用扫描
  Metadata Server 全量扫描 inode 树，构建 Chunk 引用 Bitmap

Step 2: Chunk 存活查询
  将 BitMap 发送给所有 Data Server，Data Server 标记本地 Chunk 的存活状态

Step 3: 孤儿识别
  Data Server 识别不在 Bitmap 中的 Chunk → 标记为 Candidate（候选删除）

Step 4: 安全窗口等待
  候选 Chunk 等待 grace_period（默认 24h），防止并发写入误删

Step 5: 回收
  Data Server 删除过期候选 Chunk，释放磁盘空间

gc:
  scanInterval: 24h        # 扫描间隔
  gracePeriod: 24h         # 候选删除等待时间
  maxRemovalBatch: 10000   # 单次批量删除上限
  cpuLimit: 0.5            # GC 进程 CPU 限制（按核数）

Metadata Server:
  Accept Loop ──> spawn task per connection
                    ├── Deserialize Request
                    ├── Apply to Raft State Machine (serialized via mpsc channel)
                    └── Serialize Response (send back)

Data Server:
  Accept Loop ──> spawn task per Chunk operation
                    ├── io_uring read/write (true async)
                    ├── CRC check / compute
                    └── acknowledge to Client

service MetaService {
  // 文件操作
  rpc Create(CreateRequest) returns (CreateResponse);
  rpc Lookup(LookupRequest) returns (LookupResponse);
  rpc ReadDir(ReadDirRequest) returns (stream DirEntry);
  rpc GetAttr(GetAttrRequest) returns (Attr);

  // Chunk 分配
  rpc AllocChunk(AllocChunkRequest) returns (AllocChunkResponse);
  rpc CommitChunk(CommitChunkRequest) returns (CommitChunkResponse);
}

service DataService {
  // 数据面操作
  rpc WriteChunk(stream ChunkData) returns (WriteResult);
  rpc ReadChunk(ReadChunkRequest) returns (stream ChunkData);
  rpc DeleteChunk(DeleteChunkRequest) returns (DeleteResult);
  rpc SyncChunk(SyncChunkRequest) returns (SyncResult);
}

graph TD
    A["Client Buffer<br/>(Write-Back Cache)"]
    B["Primary Data Server"]
    C["1. write to WAL (fsync)"]
    D["2. write Chunk data (fsync)"]
    E["3. replicate to Secondaries (parallel)<br/>each Secondary: write Chunk + fsync"]
    F["4. quarantine >= 2 ACKs (3-replica)"]
    G["5. commit WAL + respond to Client"]
    A -->|"fsync() / close() 强制刷新"| B
    B --> C
    B --> D
    B --> E
    C --> F
    D --> F
    E --> F
    F --> G

Phase 1: 获取锁
  - 锁定源路径的父目录 inode
  - 锁定目标路径的父目录 inode
  - (如果跨 Volume，锁定两个 Volume 的 Raft State Machine)

Phase 2: MVCC 原子提交
  - 在 Raft Log 中创建一个原子日志条目：
    { Op: Rename, SrcInode, SrcName, DstInode, DstName, Version }
  - Leader 提交该日志条目后，原子应用状态变更
  - 释放锁

FROM scratch
COPY --from=builder /build/rustfs-meta /usr/bin/rustfs-meta
COPY --from=builder /build/rustfs-data /usr/bin/rustfs-data
COPY --from=builder /build/rustfs-csi /usr/bin/rustfs-csi
ENTRYPOINT ["/usr/bin/rustfs-meta"]

# 一行命令编译 ARM64 二进制
rustup target add aarch64-unknown-linux-musl
cargo build --release --target aarch64-unknown-linux-musl

graph TD
    NFSClient["NFS Client v3/v4"]
    NFSClient --> Ganesha
    subgraph Ganesha["NFS-Ganesha Server"]
        FSAL["FSAL_RUSTFS<br/>(File System Abstraction Layer)"]
    end
    Ganesha --> RustFSClient["RustFS Client (lib)"]
    RustFSClient --> Cluster["RustFS Metadata + Data Cluster"]

{
  "timestamp": "2026-01-26T10:30:45.123Z",
  "level": "INFO",
  "target": "rustfs_meta::service",
  "span": {
    "request_id": "a1b2c3d4",
    "operation": "CreateFile",
    "user": "uid:1000"
  },
  "fields": {
    "path": "/volumes/prod/data/file.txt",
    "mode": "0644",
    "latency_ms": 2.3,
    "status": "OK"
  }
}

graph TD
    BigData["Spark / Flink / Hive"]
    BigData --> HCFS
    subgraph HCFS["Hadoop FileSystem API"]
        Shim["rustfs-hadoop.jar<br/>(HCFS Shim)"]
    end
    HCFS --> JNI["RustFS Client<br/>(JNI Bindings)"]
    JNI --> Cluster["RustFS Cluster"]

graph LR
    subgraph ClusterA["Cluster A (Primary)"]
        MetaLogA["Metadata Log<br/>(Raft Log)"]
        DataA["Data Servers"]
    end
    subgraph ClusterB["Cluster B (Standby)"]
        MetaLogB["Metadata Log<br/>(Replay)"]
        DataB["Data Servers"]
    end
    MetaLogA -->|"replay"| MetaLogB
    DataA -->|"Chunk sync"| DataB

Step 1: 升级 Data Server（无状态，先升级）
  逐节点 drain → 升级二进制 → 重新加入集群
  （注意：升级期间 Chunk 重建可能触发，确保有足够容量余量）

Step 2: 升级 Metadata Server Follower
  逐 Follower 升级，Raft Log 版本兼容期内无风险

Step 3: 触发 Metadata Server Leader Transfer
  rustfs-cli meta transfer-leader --to <new-version-node>

Step 4: 升级原 Leader
  原 Leader 变为 Follower 后升级

Step 5: 升级 Client / CSI Driver
  Client 版本可独立于服务端升级

# 检查当前元数据版本
rustfs-cli version check /var/lib/rustfs/meta

# 执行元数据格式迁移（离线操作）
rustfs-admin migrate --from 0.3.0 --to 0.4.0 --data-dir /var/lib/rustfs/meta