Missing InstanceConfig in LXD CPI configuration causes 'no matching stemcell configuration found' error

[rkoster]

Problem

When deploying CF (or any deployment using the LXD/Incus CPI), all VMs fail to create with:

Error: CPI error 'Bosh::Clouds::CloudError' with message 'no matching stemcell configuration found' in 'create_vm' CPI method

This happens because the InstanceConfig section is missing from the CPI configuration file at /var/vcap/jobs/lxd_cpi/config/cpi.json.

Root Cause

The lxd_cpi job spec in bosh-lxd-cpi-release defines a default instance_config:

lxd_cpi.server.instance_config:
  default:
    jammy:
      security.csm: "true"
      security.secureboot: "false"
    default:
      security.secureboot: "false"

However, the deployed CPI config in instant-bosh does not include this InstanceConfig section. The CPI code in create_vm.go requires this configuration to determine VM settings based on the stemcell type:

// create_vm.go:102-114
for k, v := range c.config.Server.InstanceConfig {
    if "default" == k {
        defaultConfig = v
    } else if strings.Contains(imageDescription, k) {
        instanceConfig = v
    }
}
if len(instanceConfig) == 0 {
    instanceConfig = defaultConfig
}
if len(instanceConfig) == 0 {
    return apiv1.VMCID{}, apiv1.Networks{}, bosherr.Error("no matching stemcell configuration found")
}

Workaround

Manually add the InstanceConfig to the deployed CPI configuration:

# Extract current config, add InstanceConfig, and push back
incus exec instant-bosh --project ibosh -- cat /var/vcap/jobs/lxd_cpi/config/cpi.json | \
  jq '.Server.InstanceConfig = {
    "jammy": {"security.csm": "true", "security.secureboot": "false"},
    "default": {"security.secureboot": "false"}
  }' > /tmp/cpi.json

incus file push /tmp/cpi.json instant-bosh/var/vcap/jobs/lxd_cpi/config/cpi.json --project ibosh

Potential Fixes

1. Update instant-bosh manifest: Ensure the lxd_cpi.server.instance_config property is explicitly set in the BOSH manifest used by instant-bosh, so it gets rendered into the CPI config file.

2. Update bosh-lxd-cpi-release version: If instant-bosh is using an older version of the CPI release that predates the instance_config feature, update to a newer version.

3. Investigate why defaults aren't applied: The job spec has defaults defined, but they're not appearing in the rendered config. This could be:

   • A manifest override setting it to empty/null
   • An older release version without the defaults
   • A rendering issue in the BOSH release

Environment

• instant-bosh with Incus CPI
• Stemcell: bosh-openstack-kvm-ubuntu-jammy-go_agent v1.1065