45drives / cockpit-zfs-manager Goto Github PK

Bug Report: Issue with Cockpit ZFS Manager Replication

Description:

I am encountering a persistent problem with the Cockpit ZFS Manager on the 45drives platform. Specifically, when attempting to adjust replication settings, the interface falsely reports successful completion. However, upon verification, no changes are evident. Additionally, there is an unwarranted behavior where clicking the replication button initiates three popup windows, necessitating closure before reaching a functional modification window. Although changes made in this final popup are reported as successful, they do not reflect in the actual ZFS configuration. This has been verified by checking the setup using 'znapzendzetup edit,' revealing no discernible alterations from the Cockpit UI.

Steps to Reproduce:

1. Navigate to the replication settings in the Cockpit ZFS Manager.
2. Attempt to modify replication settings.
3. Observe the false successful completion message.
4. Click the replication button again and note the appearance of three popup windows.
5. Close the popups to reach the final modification window.
6. Make changes and observe the reported successful completion.
7. Verify changes using 'znapzendzetup edit.' -- NO CHANGES HAPPENING

Expected Behavior:

Successful replication settings modifications should reflect in the ZFS configuration as reported by the Cockpit UI.

Actual Behavior:

Changes made through the Cockpit UI are not being applied to the ZFS configuration, despite receiving a successful completion message.

Environment:

• Cockpit ZFS Manager version:
  We tried both:
  cockpit-zfs-manager_1.3.0-3focal_all.deb
  cockpit-zfs-manager_1.3.0-5focal_all.deb

• Operating System: Ubuntu Live Server 20.04.6 & 22.04.3

• Browsers:
  Google Chrome Browser
  Firefox Browser

Additional Information:

    Hardware: Dell PowerEdge R740xd

The README file contains the old url to the upstream repo.

More recent versions of cockpit support dark mode:

• cockpit-project/cockpit#10189

It would help to save my eyes if zfs-manager was updated to support it too!

Hi,

I am running the latest version of cockpit on Ubuntu server 22.04.

When I try to create a new storage pool using cockpit-ifs-manger, 2 out of 9 disks are not visible in the create storage pool dialog.

If I create a pool from terminal using zpool create, the pool is created with all disks. Any clues?

Thanks

Bug Info
Summary:
Newly created mirrored pool reports as "Replication: Error" even though no errors are reported by zpool status.

Steps to reproduce the behavior:

1. Created a new mirrored pool with command:
   sudo zpool create sharedDirectory mirror _deviceA_ _deviceB_
2. Retrieved pool data with command:
   sudo zpool status
   With results:

 pool: sharedDirectory
 state: ONLINE
 config:

        NAME                                 STATE     READ WRITE CKSUM
        sharedDirectory                      ONLINE       0     0     0
          mirror-0                           ONLINE       0     0     0
            ata-deviceA  ONLINE       0     0     0
            ata-deviceB  ONLINE       0     0     0

errors: No known data errors

3. At the same time cockpit-zfs-manager reports the following:
   

Additional Information
OS: Ubuntu 22.04.1 LTS
cockpit: 264-1ubuntu0.22.04.1
cockpit-bridge: 264-1ubuntu0.22.04.1
cockpit-networkmanager: 264-1ubuntu0.22.04.1
cockpit-packagekit: 264-1ubuntu0.22.04.1
cockpit-storaged: 264-1ubuntu0.22.04.1
cockpit-system: 264-1ubuntu0.22.04.1
cockpit-ws: 264-1ubuntu0.22.04.1
cockpit-zfs-manager: 1.3.0

(sorry if I'm missing something here and there is another place from where to get replication information from, but I couldn't find any and believe me I researched. If zfs-manager gets replication status from another place please let me know and I'll confirm if the error is there to close this issue)

I turned off trigger 'Read Only' on Configure File System dialog, pressed 'Configure', see message
File System successfully configured
But file system is continue to be 'Read-only'.

I have re-created File system, it showed as non-Read-only but from windows on samba I can not add anything - You need permission...
Even if I turned on 'Anonymous access'

The plugin does not work on Rocky Linux 9.4. No pools are displayed or can be configured.

Even though, system has pools and drives ...

Would like to leverage this project with NixOS. Nix is a package manager that focuses on declarative configurations. NixOS is built on top of that approach to provide a declarative operating system.

Cockpit support was added to NixOS. This comment enabling support makes it sound like its reasonably low effort to extend offer support. NixOS/nixpkgs#209192 (comment)

I am just getting started with NixOS. Interested to see if my idea of using NixOS/Cockpit+Extensions for managing a NAS.

Apologies if I am posting in the wrong thread: I don't have an issue, but rather a suggestion.

Is it possible to include the provision for your zfs plugin to create zpools on luks disks (i.e. dev/mapper devices)? As of now, it only shows actual disks.

I for one prefer not to use zfs' built-in encryption. And I always encrypt my data. Luks is very convenient given that keyfiles can be stored in a decrypted /root, and I can even use clevis to provide auto-boot-decryption options.

The way I create zpools on luks now is to manually create the zpool at the command line, then I can log into Houston and your module instantly sees the new pool (so I can mod/add datasets etc.). I wonder if would be cleaner/better(?) to even perform the creation from within your plugin (?) It's by no means essential since there is an easy workaround, but I thought it's easy to suggest.

THANK YOU for the excellent app. V/R

Andrew

Hi!
The dpkg package has as a dependency the zfs-dkms package. However, in Ubuntu, ZFS is also supported as a kernel module and you only need to install zfs-utils.
Is there any reason to have the dkms package as a dependency instead of this?

I'm running Cockpit from Ubuntu 22.04's repositories. The normal repositories package Cockpit 264-1, and this issue is not present there.

Upon upgrading to the version available in backports (277-1), as per Cockpit's website recommendation, icons disappeared on the ZFS manager:

Table headers also look weirdly styled. In general this looks to be some sort of CSS problem.

Happens on Firefox 109 and Chromium 108.

On Rocky 8 (RHEL8) the message '/dev/(null' is shown as a message for each disk in the status view.

I was discussing this topic on the cockpit github project cockpit-project/cockpit#19482. Apparently the memory statistics show the ARC cache as used memory, so it's always full, which makes it difficult to tell if the system is really running low on RAM.

It would be nice to have a memory readout that distinguishes between used / free memory and ARC.

Since this is specific to ZFS-enabled systems, then maybe this is something that could be implemented here.

• Version: cockpit 264, cockpit-zfs-manager 1.3.0, Ubuntu Server 22.04 LTS
• Description: After unlocking an encrypted share, every entry on the volume list is duplicated. Both entries about encrypted volumes provide proper state (open padlock)
  
• Steps to reproduce: Make empty zfs pool. Create encrypted share. Unmount, checking the option "Lock filesystem". Mount again. Enjoy doubled storage.
• Suspicion: Race condition in FnFileSystemUnlock and FnFileSystemMount, as they both execute FnStoragePoolRefresh asynchronously.

I installed the ZFS cockpit add-on fork and clicked through the config screen leaving the defaults. I don't need Samba, or plan to use it, but I can't get back to the config to uncheck the box. Stuck with "This package requires Samba" screen

Bug Info
Summary: when creating a mirror pool with two drives of different sizes with the force option, the Storage Pool could not created error notification presents, and the pool is not created. However, if the zpool create -f mirror /dev/sda /dev/sdb command is executed from the terminal, it works just fine with the pool being manageable from the ZFS tab.

Steps to reproduce the behavior

1. Go to the ZFS tab
2. Click Create Storage Pool
3. Give Name, select Mirror for Virtual Device, select the two disks of different sizes, check option to Forcefully create storage pool
4. Click Create

Expected behavior
Pool should be created just like with the zpool command from terminal

Additional Information

• System: Raspberry Pi 4B
• OS: Ubuntu 22.04
• cockpit: 264-1ubuntu0.22.04.1
• cockpit-bridge: 264-1ubuntu0.22.04.1
• cockpit-networkmanager: 264-1ubuntu0.22.04.1
• cockpit-packagekit: 264-1ubuntu0.22.04.1
• cockpit-storaged: 264-1ubuntu0.22.04.1
• cockpit-system: 264-1ubuntu0.22.04.1
• cockpit-ws: 264-1ubuntu0.22.04.1
• cockpit-zfs-manager: 1.3.0-3focal

When enabling an SMB share, cockpit-zfs-manager enables two shares.

One by executing zfs set sharesmb=on <dataset> which creates a file in /var/lib/samba/usershares and one by creating a configuration within /etc/cockpit/zfs/shares.conf and /etc/cockpit/zfs/shares/.

The share that gets created via the smb config has the name that is set through the UI, but the share that gets created through zfs set has the name of the dataset with slashes replaced by underscores.

I skipped through the zfs.js and found that FnFileSystemShareSmbEnable executes the zfs set command while FnSambaZfsShareEnable creates the smb config files.

Why though? I don't get why it's necessary to enable both, can you explain this?

Bug Info
On centOS7 you cannot create a pool as it shows no available drives, cannot check the status of pools that were created before the issue or created vis CLI

Steps to reproduce the behavior
Install cockpit-zfs-manager-1.3.0-3.el7.x86_64 on centOS7 and go to the ZFS tab

Expected behavior
Ability to create a pool/check status of the pool

Additional info
If you roll back to cockpit-zfs-manager-1.0-1.el7.x86_64 the ZFS tab works as expected

Is zstd not considered stable enough to be added to the UI ? I thought it was.
Even if it is not the default, is it not worth adding as an option at least ?

Installed on Ubuntu 20.04.3 LTS and Cockpit Version 215 using these commands

git clone https://github.com/optimans/cockpit-zfs-manager.git
sudo cp -r cockpit-zfs-manager/zfs /usr/share/cockpit

When I try to run it from cockpit I see:

This package requires ZFS version 0.8 or later
If ZFS is installed, check the module is loaded into the kernel.

What am I doing wrong?

Because you have a Depends entry in your control file that requires znapzend but there is no package on the Ubuntu mirrors and the znapzend README instructs users to install it from source, this dependency can never be fulfilled.

Maybe you should remove that dependency or provide a fork that has a .deb file so users can install it as a package?
I created a small Docker image to build a .deb file for it:

FROM ubuntu:jammy

ARG ZNAPVER
ARG DEBIAN_FRONTEND=noninteractive

WORKDIR /usr/local/src

RUN apt-get update && \
    apt-get install -y build-essential \
      bash \
      perl \
      unzip \
      autoconf \
      carton \
      curl \
      tar \
      wget && \
    wget https://github.com/oetiker/znapzend/releases/download/v${ZNAPVER}/znapzend-${ZNAPVER}.tar.gz && \
    tar zxvf znapzend-${ZNAPVER}.tar.gz

WORKDIR /usr/local/src/znapzend-${ZNAPVER}

RUN autoreconf --force --install --verbose --make && \
    ./configure --prefix=/usr/local && \
    make && \
    make DESTDIR=/build/znapzend_${ZNAPVER}-1_amd64 install

COPY control /build/znapzend_${ZNAPVER}-1_amd64/DEBIAN/control

WORKDIR /build

RUN sed -i "s/ZNAPVER/${ZNAPVER}/" /build/znapzend_${ZNAPVER}-1_amd64/DEBIAN/control && \
    dpkg-deb --build --root-owner-group "znapzend_${ZNAPVER}-1_amd64"

It needs a control file in the context directory:

Package: znapzend
Version: ZNAPVER
Maintainer: oetiker
Architecture: amd64
Description: ZnapZend is a ZFS centric backup tool to create snapshots and send them to backup locations. It relies on the ZFS tools snapshot, send and receive to do its work. It has the built-in ability to manage both local snapshots as well as remote copies by thinning them out as time progresses.
Depends: perl, zfsutils-linux

Building it with can be done with docker build -t znapzend-builder --build-args=ZNAPVER=0.21.1 . and then the .deb file can be copied by creating a container docker create --name znapzend znapzend-builder and copying the .deb package from there docker cp znapzend:/build/znapzend_0.21.1-1_amd64.deb .. Which can then be installed via apt install znapzend_0.21.1-1_amd64.deb.

This is just an example on how it can be done, but I use this in my Ansible scripts.

Bug Info

When selecting the configure replication task in the ZFS tab on Rocky Linux, it returns with an error of

"Replication task could not be configured

${filesystem}"

After which leaves the ZFS page greyed and unable to select anything further

Steps to reproduce the behavior

Go to the ZFS tab, and select the "Configure Replication Task" Option

Expected behavior

Ability to create ZFS snapshot and replication tasks within the UI

Screenshots

Console Output

Failed to load resource: the server responded with a status of 404 (Not Found)
zfs.js:973 () => { } Cockpit, Package, SOS Report, Delete: Skipped
cockpit.js:1 grep: /etc/dnf/automatic.conf: No such file or directory

p @ cockpit.js:1
zfs.js:4182 () => { } Storage Pools, Refresh: In Progress, Pool: tank
zfs.js:4298 () => { } Storage Pools, Refresh: Success, Pool: tank
zfs.js:4865 () => { } File Systems, Get: In Progress, Pool: tank
zfs.js:8857 () => { } Snapshots, Get: In Progress, Pool: tank
zfs.js:9971 () => { } Status, Get: In Progress, Pool: tank
zfs.js:8870 () => { } Snapshots, Get: Success, Pool: tank
zfs.js:4519 () => { } Storage Pools, Size, Get: In Progress, Pool: tank
zfs.js:4890 () => { } File Systems, Get: Success, Pool: tank
zfs.js:11395 () => { } Disks, Lsblk, Get: In Progress
zfs.js:4524 () => { } Storage Pools, Size, Get: Success, Pool: tank
zfs.js:11400 () => { } Disks, Lsblk, Get: Success
zfs.js:9992 () => { } Status, Get: Success, Pool: tank
updates.js:2 Could not determine kpatch availability: {"detail":"higher version "0.9.2-5.el8" of package kpatch.noarch is already installed","code":9}
zfs.js:21186 Uncaught (in promise) TypeError: Cannot read properties of undefined (reading 'length')
at FnModalReplicationTaskCreateContent (zfs.js:21186)
RedHatDisplay-Bold.woff:1 Failed to load resource: the server responded with a status of 404 (Not Found)
DevTools failed to load source map: Could not load content for https://192.168.69.125:9090/cockpit/$d890ef116b803cb1ccd2af727e1957abcc0342a9fcd66af022b80f651284f3ce/zfs/jquery.min.js.map: HTTP error: status code 404, net::ERR_HTTP_RESPONSE_CODE_FAILURE
2DevTools failed to load source map: Could not load content for https://192.168.69.125:9090/cockpit/$d890ef116b803cb1ccd2af727e1957abcc0342a9fcd66af022b80f651284f3ce/zfs/patternfly.min.css.map: HTTP error: status code 404, net::ERR_HTTP_RESPONSE_CODE_FAILURE

Desktop (please complete the following information)

OS: Windows
Browser: Chrome
Version: Version 95.0.4638.69 (Official Build) (64-bit)

Smartphone (please complete the following information)

Device: N/A
OS: N/A
Browser: N/A
Version: N/A

Server Side

OS: Rocky Linux
Cockpit Version: 250

Additional context

N/A

Most SATA SSDs have either bad or no WWN to add.
Adding by /dev/sdX can result in a massive problem if the linux drive lettering changes on reboot
Additionally, if a device fails enough to be "removed" it is no longer possible to look up the dev SN from the pool/by correlating WWN to SN

Cockpit doesn't offer devices by the "ata label" even though they are (at least under linux) in the same /dev/disk/by-id/ folder as the WWN IDs.

So I was trying to clean out my system after some testing but despite removing zfs-manager the ZFS menu remains in Cockpit.
With an error

This package requires ZFS version 0.8 or later If ZFS is installed, check the module is loaded into the kernel.

How to remove and why was it not removed?

I just installed Cockpit on a fresh Ubuntu Server 22.04 install and added the ZFS manager according to the README. I have made sure I have administrator access in the Cockpit UI, however, I cannot use the ZFS manager at all.

I can click the link to enter the tab, but it's empty. Even if I create a pool using the CLI it does not show up. I can't make a pool in the ZFS manager either because none of my disks show up.

Looking in the Chrome dev console, I see a few warnings complaining that the ZFS commands must be run using root.
It just shows "No available disks found".

Any ides what I might be missing to be able to use this plugin?

When using OpenZfs, the binary for zpool is found in /usr/local/sbin and not in /sbin. This means that the ZFS pools will not be listed in the Cockpit ZFS module. An easy fix follows.

cd /sbin
ln -s /usr/local/sbin/zpool zpool

Once this is done, a fresh in the ZFS module should show the pools.

These other binaries also need to be linked in the same way to ensure that all functionality is available.

• zdb
• zfs
• zfs_ids_to_path
• zhack
• zinject
• zpool
• zstream
• zstreamdump
• ztest

Storage pool is 1.07 PiB. When setting a quota on a dataset the slider is unable to go higher than 1.07 TiB. Seems as if the Quota slider is interpreting 1.07 PiB as 1.07 TiB.