Following task fails on a new system when creating a cluster of Scylla 4.4 used on Ubuntu 20.4
TASK [scylla-node : Measure IO settings on one node] ************************************************************************************************************************************************************
fatal: [redacted_ip_of_host]: FAILED! => {"changed": true, "cmd": "scylla_io_setup\n", "delta": "0:00:00.899964", "end": "2021-05-27 13:23:54.277646", "msg": "non-zero return code", "rc": 1, "start": "2021-05-27 13:23:53.377682", "stderr": "/usr/sbin/scylla_io_setup: line 3: warning: setlocale: LC_ALL: cannot change loca
le (en_US.UTF-8): No such file or directory\n/bin/bash: warning: setlocale: LC_ALL: cannot change locale (en_US.UTF-8)\nERROR:root:This is not a recommended Google Cloud instance setup for auto tuning, running manual iotune.\n/bin/bash: warning: setlocale: LC_ALL: cannot change locale (en_US.UTF-8)\nERROR 2021-05-
27 13:23:54,176 [shard 5] seastar - Could not setup Async I/O: Resource temporarily unavailable. The most common cause is not enough request capacity in /proc/sys/fs/aio-max-nr. Try increasing that number or reducing the amount of logical CPUs available for your application\nERROR:root:Command '['/usr/bin/iotune',
'--format', 'envfile', '--options-file', '/etc/scylla.d/io.conf', '--properties-file', '/etc/scylla.d/io_properties.yaml', '--evaluation-directory', '/var/lib/scylla/data', '--evaluation-directory', '/var/lib/scylla/commitlog', '--evaluation-directory', '/var/lib/scylla/hints', '--evaluation-directory', '/var/lib
/scylla/view_hints']' returned non-zero exit status 1.\nERROR:root:['/var/lib/scylla/data', '/var/lib/scylla/commitlog', '/var/lib/scylla/hints', '/var/lib/scylla/view_hints'] did not pass validation tests, it may not be on XFS and/or has limited disk space.\nThis is a non-supported setup, and performance is expec
ted to be very bad.\nFor better performance, placing your data on XFS-formatted directories is required.\nTo override this error, enable developer mode as follow:\nsudo /opt/scylladb/scripts/scylla_dev_mode_setup --developer-mode 1", "stderr_lines": ["/usr/sbin/scylla_io_setup: line 3: warning: setlocale: LC_ALL:
cannot change locale (en_US.UTF-8): No such file or directory", "/bin/bash: warning: setlocale: LC_ALL: cannot change locale (en_US.UTF-8)", "ERROR:root:This is not a recommended Google Cloud instance setup for auto tuning, running manual iotune.", "/bin/bash: warning: setlocale: LC_ALL: cannot change locale (en_U
S.UTF-8)", "ERROR 2021-05-27 13:23:54,176 [shard 5] seastar - Could not setup Async I/O: Resource temporarily unavailable. The most common cause is not enough request capacity in /proc/sys/fs/aio-max-nr. Try increasing that number or reducing the amount of logical CPUs available for your application", "ERROR:root:
Command '['/usr/bin/iotune', '--format', 'envfile', '--options-file', '/etc/scylla.d/io.conf', '--properties-file', '/etc/scylla.d/io_properties.yaml', '--evaluation-directory', '/var/lib/scylla/data', '--evaluation-directory', '/var/lib/scylla/commitlog', '--evaluation-directory', '/var/lib/scylla/hints', '--eval
uation-directory', '/var/lib/scylla/view_hints']' returned non-zero exit status 1.", "ERROR:root:['/var/lib/scylla/data', '/var/lib/scylla/commitlog', '/var/lib/scylla/hints', '/var/lib/scylla/view_hints'] did not pass validation tests, it may not be on XFS and/or has limited disk space.", "This is a non-supported
setup, and performance is expected to be very bad.", "For better performance, placing your data on XFS-formatted directories is required.", "To override this error, enable developer mode as follow:", "sudo /opt/scylladb/scripts/scylla_dev_mode_setup --developer-mode 1"], "stdout": "tuning /sys/devices/virtual/blo
ck/dm-0\ntuning: /sys/devices/virtual/block/dm-0/queue/nomerges 2\ntuning /sys/devices/pci0000:00/0000:00:03.0/virtio0/host0/target0:0:2/0:0:2:0/block/sdb\ntuning: /sys/devices/pci0000:00/0000:00:03.0/virtio0/host0/target0:0:2/0:0:2:0/block/sdb/queue/nomerges 2\ntuning /sys/devices/virtual/block/dm-0\ntuning /sys/
devices/virtual/block/dm-0\ntuning /sys/devices/virtual/block/dm-0", "stdout_lines": ["tuning /sys/devices/virtual/block/dm-0", "tuning: /sys/devices/virtual/block/dm-0/queue/nomerges 2", "tuning /sys/devices/pci0000:00/0000:00:03.0/virtio0/host0/target0:0:2/0:0:2:0/block/sdb", "tuning: /sys/devices/pci0000:00/000
0:00:03.0/virtio0/host0/target0:0:2/0:0:2:0/block/sdb/queue/nomerges 2", "tuning /sys/devices/virtual/block/dm-0", "tuning /sys/devices/virtual/block/dm-0", "tuning /sys/devices/virtual/block/dm-0"]}
echo 1048576 > /proc/sys/fs/aio-max-nr