Possibility of data-corruption with raw VMs on Sculpt OS 25.10

TL;DR If you only use VMs via an VDI, e.g. machine.vdi and/or use an unaltered vbox6 pkg, you are not affected and can ignore this post.

Description

Larger I/O requests—as emitted by a VM—that are processed by the VFS Block plugin are split into smaller requests whenever the underlying storage device cannot handle the original I/O request in one go. Due to a bug in the VFS plugin these additional split writes will be silently omitted and do not produce any data. Stale data already stored in the data buffer will be written instead.

For this to happen the storage device needs to limit the maximal data transfer size to a value less than 1¹ MiB and the configured I/O buffer size of the VFS Block plugin needs to be larger than the maximal data transfer size.

For NVMe devices one way to determine if you could be affected is checking if the
output of the following command is less than 8:

smartctl -a -C /dev/nvmeXnY | grep 'Maximum Data Transfer Size'

AHCI devices should be less likely to be affected as they should support 1 MiB requests.

If your block node does not include the io_buffer attribute or still uses the no longer uses block_buffer_count attribute the I/O buffer is by default set to 4 MiB and thus the I/O buffer is usually larger than the maximal data transfer size.

Impact

Raw VMs that access storage devices (completely or partially via a partition) by using a disk.vmdk configuration directly can experience data corruption.

Workaround

Adapting the VFS Block plugin node in the VBox6 runtime configuration in the following way mitigates the problem by limiting the I/O buffer size in a way that prevents this particular bug:

<dir name="dev">
  […]
  <block name="block" io_buffer="8192"/>
</dir> 

Solution

Upgrade to the new Sculpt OS 25.10 hotfix relase that contains a fix, which we are going to release soon.


¹) 1 MiB is the largest single request a Linux guest should issue via the AHCI device-model in VBox6.

2 Likes