Mounted images fail to boot

[rootxnet]

On NanoKVM PCIe version (img: 1.4.0/app 2.2.9), I've noticed that my .iso images are failing to boot - the boot sector seems to be fine but the rest of .iso is missing.
My BIOS reported all .iso I've used to be 162 MB in size which after some research with LLMs pointed me to this:

Why your ISO mounted as only 162 MB

1. Wrong backing device was used
   • The original code set imageNone = /dev/mmcblk0p3.
   • When no file was passed (or logic didn’t flip correctly), the gadget exposed your real eMMC partition (~162 MB) as the storage device, not the ISO.
   • BIOS therefore reported the partition’s true size = 162 MB.

I've built a workaround via AI:
kvm-mass-storage.sh

# kvm-mass-storage.sh — Recreate & mount USB gadget mass_storage properly
# Works with BusyBox ash (no bashisms).

set -eu

# Defaults
ISO_OR_IMG=""
MODE="iso"                 # iso | img
GADGET="g0"
LUN="0"
CFG="c.1"
UDC=""                     # auto-detect if empty
RO=""                      # auto by MODE: iso->1, img->0 (overridable)
REMOVABLE=""               # empty = skip; set 0/1 to force
NOFUA=""                   # empty = skip; set 0/1 to force
UNMOUNT=0

usage() {
  cat <<EOF
Usage:
  $0 -i /data/image.iso [-m iso|img] [-g g0] [-l 0] [-c c.1] [-u <udc>] [--ro 0|1] [--removable 0|1] [--nofua 0|1]
  $0 --unmount [-g g0] [-l 0] [-c c.1] [-u <udc>]

Options:
  -i PATH     Path to ISO/IMG file (required except with --unmount)
  -m MODE     iso (CD-ROM, 2048B sectors, RO) or img (disk). Default: iso
  -g NAME     Gadget name. Default: g0
  -l N        LUN index. Default: 0  (path: lun.N)
  -c NAME     Config name. Default: c.1
  -u UDC      UDC controller to bind. Default: auto-detect first in /sys/class/udc
  --ro X      Force ro flag (0/1). Overrides mode default.
  --removable X   Set lun.N/removable (0/1)
  --nofua X       Set lun.N/nofua (0/1)
  --unmount       Detach backing file and unbind/bind to signal media change

Examples:
  # Mount ISO as CD-ROM (RO):
  $0 -i /data/my.iso

  # Mount IMG as writable disk:
  $0 -i /data/disk.img -m img

  # Unmount / detach:
  $0 --unmount
EOF
}

# --- Parse args (POSIX) ---
while [ $# -gt 0 ]; do
  case "$1" in
    -i) ISO_OR_IMG="$2"; shift 2 ;;
    -m) MODE="$2"; shift 2 ;;
    -g) GADGET="$2"; shift 2 ;;
    -l) LUN="$2"; shift 2 ;;
    -c) CFG="$2"; shift 2 ;;
    -u) UDC="$2"; shift 2 ;;
    --ro) RO="$2"; shift 2 ;;
    --removable) REMOVABLE="$2"; shift 2 ;;
    --nofua) NOFUA="$2"; shift 2 ;;
    --unmount) UNMOUNT=1; shift ;;
    -h|--help) usage; exit 0 ;;
    *) echo "Unknown option: $1" >&2; usage; exit 1 ;;
  esac
done

G="/sys/kernel/config/usb_gadget/$GADGET"
L="$G/functions/mass_storage.disk0/lun.$LUN"
C="$G/configs/$CFG"

die() { echo "ERR: $*" >&2; exit 1; }
exists_dir() { [ -d "$1" ] || die "Missing dir: $1"; }
exists_file() { [ -f "$1" ] || die "Missing file: $1"; }

# Auto RO by mode if not forced
if [ -z "${RO:-}" ]; then
  if [ "$MODE" = "iso" ]; then RO=1; else RO=0; fi
fi

# Resolve UDC if empty
if [ -z "${UDC:-}" ]; then
  # shellcheck disable=SC2012
  UDC=$(ls -1 /sys/class/udc 2>/dev/null | head -n1 || true)
fi

# Sanity checks
exists_dir "$G" || true
[ "$UNMOUNT" -eq 1 ] || exists_file "$ISO_OR_IMG"

[ -n "$UDC" ] || [ "$UNMOUNT" -eq 1 ] || die "No UDC found; specify with -u"

# Helpers to write sysfs without newline
w() { echo -n "$2" > "$1"; }

unbind() { w "$G/UDC" ""; }
bind()   { w "$G/UDC" "$UDC"; }

unlink_func() {
  if [ -L "$C/mass_storage.disk0" ]; then rm -f "$C/mass_storage.disk0" || true; fi
}

detach_file() {
  if [ -e "$L/file" ]; then w "$L/file" ""; fi
}

remove_func() {
  if [ -d "$G/functions/mass_storage.disk0" ]; then
    rmdir "$G/functions/mass_storage.disk0" 2>/dev/null || true
  fi
}

create_func() {
  mkdir -p "$G/functions/mass_storage.disk0"
}

ensure_config() {
  mkdir -p "$C"
  mkdir -p "$C/strings/0x409"
  [ -f "$C/strings/0x409/configuration" ] || w "$C/strings/0x409/configuration" "Config 1"
}

link_func() {
  ln -s "$G/functions/mass_storage.disk0" "$C/mass_storage.disk0"
}

set_flags() {
  # MODE determines cdrom; RO already chosen
  if [ "$MODE" = "iso" ]; then
    w "$L/cdrom" "1"
  else
    w "$L/cdrom" "0"
  fi
  w "$L/ro" "$RO"

  # Optional flags if requested and nodes exist
  if [ -n "${REMOVABLE:-}" ] && [ -e "$L/removable" ]; then w "$L/removable" "$REMOVABLE"; fi
  if [ -n "${NOFUA:-}" ] && [ -e "$L/nofua" ]; then w "$L/nofua" "$NOFUA"; fi
}

attach_file() {
  w "$L/file" "$ISO_OR_IMG"
}

verify() {
  echo "UDC:   $(cat "$G/UDC" 2>/dev/null || echo)"
  [ -e "$L/file" ] && echo "file:  $(cat "$L/file")" || true
  [ -e "$L/cdrom" ] && echo "cdrom: $(cat "$L/cdrom")" || true
  [ -e "$L/ro" ] && echo "ro:    $(cat "$L/ro")" || true
}

# --- Actions ---
if [ "$UNMOUNT" -eq 1 ]; then
  # Cleanly detach whatever is mounted and unbind/bind to signal media change
  exists_dir "$G"
  exists_dir "$C" || true
  echo "[*] Unmounting from $G (LUN $LUN) ..."
  unbind || true
  detach_file || true
  unlink_func || true
  remove_func || true
  # Recreate empty function so host sees 'no media'
  create_func
  ensure_config
  link_func
  # Bind back (still no file attached)
  if [ -n "$UDC" ]; then bind || true; fi
  verify
  exit 0
fi

# Mount flow
echo "[*] Mounting '$ISO_OR_IMG' as $MODE on gadget '$GADGET', LUN $LUN, cfg $CFG, UDC=${UDC:-auto-none}"

exists_dir "$G" || die "Gadget $GADGET not found in configfs"
ensure_config

# 1) Unbind to avoid host probing mid-change
unbind || true

# 2) Unlink & remove any existing mass_storage function
unlink_func || true
detach_file || true
# small settle helps some controllers
sleep 0.2
remove_func || true

# 3) Recreate function fresh
create_func

# 4) Set flags BEFORE attaching file
set_flags

# 5) Attach file (no newline)
attach_file

# 6) Link function into the config
link_func

# 7) Bind to UDC
[ -n "$UDC" ] && bind

# 8) Verify
verify

echo "[*] Done."

chmod +x kvm-mass-storage.sh
./kvm-mass-storage.sh -i /data/my.iso - this properly mounts it, BIOS recognizes the proper ISO size and it is booting properly

[alexislefebvre]

What error did you encounter?

I can see the boot menu of Debian Live but when I select the first entry I have this error:

attempt to read or write outside of disk cd0

So I'm not sure if I have the same issue than you.

I've built a workaround via AI:

Well, that's a lot of code, I'll try to extract the relevant lines because it looks like it does a lot of things and I'm not sure that I can trust this.

---

Update: I was able to boot debian-live-13.0.0-amd64-gnome.iso from https://cdimage.debian.org/mirror/cdimage/archive/13.0.0-live/amd64/iso-hybrid/

I think it worked after I disabled the “CD-ROM” mode but I think I already tried this before, so I'm not sure of what is happening.

[dustins]

I'm seeing a similar issue where even though it sees the virtual disk and gets to the boot menu it fails to boot and gives this error…

This is a custom NixOS installer, but I tried with the official one too to no avail.

[github-catch-22]

I, too, am seeing a similar issue. I have used the above script to mount the debian-live-13.0.0-amd=gnome.iso

There is a blank selection that shows up in my BIOS boot menu.

When I select it, it displays the following message. If I boot into my normal Windows and start a partition manager, the partition is displayed as follows.

In Windows the properties are displayed as per below

What can i do to help push this along?

[MagnaCapax]

Root Cause Analysis & Fix

We traced this bug to its root cause across three code locations. The NanoKVM exposes its own eMMC partition (/dev/mmcblk0p3) as a USB mass storage device in three separate code paths:

1. Init script: kvmapp/system/init.d/S03usbdev (line 116)

When /boot/usb.disk0 exists but is empty (the default out-of-box state), the script defaults to:

echo /dev/mmcblk0p3 > functions/mass_storage.disk0/lun.0/file

2. Go server: server/service/storage/image.go (line 114)

When unmounting an ISO, the server re-mounts the eMMC partition:

const imageNone = "/dev/mmcblk0p3"
// ...
if image == "" {
    image = imageNone  // re-exposes eMMC after unmount
}

3. Go server: server/service/vm/virtual-device.go (line 35)

Toggling "Virtual Disk" ON creates an empty gate file, triggering path (1):

mountDiskCommands = []string{
    "touch /boot/usb.disk0",  // empty file → triggers mmcblk0p3 default
    "/etc/init.d/S03usbdev stop",
    "/etc/init.d/S03usbdev start",
}

Impact: Legacy BIOS Boot Hang

The eMMC partition contains a Linux filesystem with no MBR. When a Legacy BIOS reads sector 0 and finds no 0x55AA boot signature, some implementations (we confirmed on ASRock DeskMini A300) enter a HLT loop instead of trying the next boot device. The system hangs completely — no keyboard input accepted, no remote recovery possible without a physical power cycle.

This is a zero-config out-of-box bug: any NanoKVM plugged into a Legacy BIOS system will cause this problem because the default image ships with /boot/usb.disk0 as a 0-byte empty file.

The Fix

S03usbdev: When usb.disk0 is empty, don't set any backing file. The mass_storage device (already removable=1) reports "no media inserted" — which BIOS handles safely by skipping to the next boot device.

image.go: When unmounting, leave the device unmounted instead of falling back to mmcblk0p3.

PR submitted: https://github.com/sipeed/NanoKVM/pull/new/fix/mass-storage-default-backing (from MagnaCapax/NanoKVM)

Workaround (No Code Changes Required)

Delete the gate file on the NanoKVM:

rm /boot/usb.disk0
reboot

This prevents S03usbdev from creating the mass_storage gadget entirely. The NanoKVM's USB interface presents only HID devices (keyboard, mouse, touchpad) and RNDIS network. To re-enable virtual USB disk later, recreate the file with a specific image path: echo /data/some.iso > /boot/usb.disk0

[MagnaCapax]

PR with the fix: #741