Debian Bug report logs - #1010026
qemu-system-x86: fails to start: kvm_buf_set_msrs: Assertion `ret == cpu->kvm_msr_buf->nmsrs' failed.

Reply or subscribe to this bug.

Toggle useless messages

Message #5 received at submit@bugs.debian.org (full text, mbox, reply):

From: Adrian Davey <adrian.davey@gmail.com>

To: submit@bugs.debian.org

Subject: qemu-system-x86: fails to start VM with "host" cpu missing features

Date: Fri, 22 Apr 2022 15:10:55 +0100

Package: qemu-system-x86
Version: 1:7.0+dfsg-1
Severity: normal

Dear Maintainer,

VMs controlled by libvirt failed to start when using "host" cpu type with kvm
acceleration

libvirt log gives:

2022-04-21T17:07:40.354354Z qemu-system-x86_64: warning: This feature depends
on other features that were not requested: CPUID.8000000AH:EDX.npt [bit 0]
2022-04-21T17:07:40.354467Z qemu-system-x86_64: warning: This feature depends
on other features that were not requested: CPUID.8000000AH:EDX.nrip-save [bit
3]
2022-04-21T17:07:40.354476Z qemu-system-x86_64: warning: This feature depends
on other features that were not requested: CPUID.8000000AH:EDX.vmcb-clean [bit
5]
2022-04-21T17:07:40.354482Z qemu-system-x86_64: warning: This feature depends
on other features that were not requested: CPUID.8000000AH:EDX.svme-addr-chk
[bit 28]
2022-04-21T17:07:40.355818Z qemu-system-x86_64: warning: This feature depends
on other features that were not requested: CPUID.8000000AH:EDX.npt [bit 0]
2022-04-21T17:07:40.355850Z qemu-system-x86_64: warning: This feature depends
on other features that were not requested: CPUID.8000000AH:EDX.nrip-save [bit
3]
2022-04-21T17:07:40.355857Z qemu-system-x86_64: warning: This feature depends
on other features that were not requested: CPUID.8000000AH:EDX.vmcb-clean [bit
5]
2022-04-21T17:07:40.355864Z qemu-system-x86_64: warning: This feature depends
on other features that were not requested: CPUID.8000000AH:EDX.svme-addr-chk
[bit 28]
2022-04-21T17:07:40.419616Z qemu-system-x86_64: error: failed to set MSR
0xc0000104 to 0x100000000
qemu-system-x86_64: ../../target/i386/kvm/kvm.c:2996: kvm_buf_set_msrs:
Assertion `ret == cpu->kvm_msr_buf->nmsrs' failed.

Reverting to qemu-system-x86 6.2+dfsg-3 and the VMs start-up perfectly fine
using the same libvirt xml.

host cpu flags from /proc/cpuinfo :

        flags           : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge
mca cmov pat pse36 clflush mmx fxsr sse sse2 ht syscall nx mmxext fxsr_opt
pdpe1gb rdtscp lm 3dnowext 3dnow constant_tsc rep_good nopl nonstop_tsc cpuid
extd_apicid pni monitor cx16 popcnt lahf_lm cmp_legacy svm extapic cr8_legacy
abm sse4a misalignsse 3dnowprefetch osvw ibs skinit wdt nodeid_msr hw_pstate
vmmcall npt lbrv svm_lock nrip_save

Cheers,

Adrian


-- System Information:
Debian Release: bookworm/sid
  APT prefers unstable
  APT policy: (500, 'unstable')
Architecture: amd64 (x86_64)
Foreign Architectures: i386

Kernel: Linux 5.16.0-5-amd64 (SMP w/2 CPU threads; PREEMPT)
Locale: LANG=en_GB.UTF-8, LC_CTYPE=en_GB.UTF-8 (charmap=UTF-8),
LANGUAGE=en_GB:en
Shell: /bin/sh linked to /bin/dash
Init: systemd (via /run/systemd/system)
LSM: AppArmor: enabled

Versions of packages qemu-system-x86 depends on:
ii  ipxe-qemu             1.0.0+git-20190125.36a4c85-5.1
ii  libaio1               0.3.113-2
ii  libbpf0               1:0.7.0-2
ii  libc6                 2.33-7
ii  libcapstone4          4.0.2-5
ii  libfdt1               1.6.1-1
ii  libfuse3-3            3.10.5-1
ii  libgcc-s1             12-20220319-1
ii  libglib2.0-0          2.72.1-1
ii  libgnutls30           3.7.4-2
ii  libibverbs1           39.0-1+b1
ii  libjpeg62-turbo       1:2.1.2-1
ii  libnettle8            3.7.3-1
ii  libnuma1              2.0.14-3
ii  libpixman-1-0         0.40.0-1
ii  libpmem1              1.11.1-3
ii  libpng16-16           1.6.37-3
ii  librdmacm1            39.0-1+b1
ii  libsasl2-2            2.1.28+dfsg-4
ii  libseccomp2           2.5.3-2
ii  libslirp0             4.6.1-1
ii  libudev1              250.4-1
ii  liburing2             2.1-2
ii  libvdeplug2           4.0.1-3
ii  libxendevicemodel1    4.16.0+51-g0941d6cb-1+b1
ii  libxenevtchn1         4.16.0+51-g0941d6cb-1+b1
ii  libxenforeignmemory1  4.16.0+51-g0941d6cb-1+b1
ii  libxengnttab1         4.16.0+51-g0941d6cb-1+b1
ii  libxenmisc4.16        4.16.0+51-g0941d6cb-1+b1
ii  libxenstore4          4.16.0+51-g0941d6cb-1+b1
ii  libxentoolcore1       4.16.0+51-g0941d6cb-1+b1
ii  libzstd1              1.5.2+dfsg-1
ii  qemu-system-common    1:7.0+dfsg-1
ii  qemu-system-data      1:7.0+dfsg-1
ii  seabios               1.15.0-1
ii  zlib1g                1:1.2.11.dfsg-4

Versions of packages qemu-system-x86 recommends:
ii  ovmf              2022.02-3
pn  qemu-block-extra  <none>
pn  qemu-system-gui   <none>
ii  qemu-utils        1:7.0+dfsg-1

Versions of packages qemu-system-x86 suggests:
ii  samba  2:4.16.0+dfsg-6
pn  vde2   <none>

-- no debconf information

Message #10 received at submit@bugs.debian.org (full text, mbox, reply):

From: Michael Tokarev <mjt@tls.msk.ru>

To: Adrian Davey <adrian.davey@gmail.com>, 1010026@bugs.debian.org, submit@bugs.debian.org

Subject: Re: Bug#1010026: qemu-system-x86: fails to start VM with "host" cpu missing features

Date: Fri, 22 Apr 2022 17:22:32 +0300

Control: tag -1 + moreinfo

22.04.2022 17:10, Adrian Davey wrote:
> Package: qemu-system-x86
> Version: 1:7.0+dfsg-1
> Severity: normal
> 
> Dear Maintainer,
> 
> VMs controlled by libvirt failed to start when using "host" cpu type with kvm
> acceleration
> 
> libvirt log gives:
> 
> 2022-04-21T17:07:40.354354Z qemu-system-x86_64: warning: This feature depends
> on other features that were not requested: CPUID.8000000AH:EDX.npt [bit 0]

..
> 2022-04-21T17:07:40.419616Z qemu-system-x86_64: error: failed to set MSR
> 0xc0000104 to 0x100000000
> qemu-system-x86_64: ../../target/i386/kvm/kvm.c:2996: kvm_buf_set_msrs:
> Assertion `ret == cpu->kvm_msr_buf->nmsrs' failed.

Please show the complete qemu command line (it should be available in the log too).

This assertion does not look like something cpu-feature-specific, it smells
like a bug by its own, not depending on -cpu host.

Thanks,

/mjt

Message #22 received at submit@bugs.debian.org (full text, mbox, reply):

From: Michael Tokarev <mjt@tls.msk.ru>

To: Adrian Davey <adrian.davey@gmail.com>, 1010026@bugs.debian.org, submit@bugs.debian.org

Subject: Re: Bug#1010026: qemu-system-x86: fails to start VM with "host" cpu missing features

Date: Fri, 22 Apr 2022 17:54:24 +0300

22.04.2022 17:10, Adrian Davey wrote:
> Package: qemu-system-x86
> Version: 1:7.0+dfsg-1
> Severity: normal
> 
> 2022-04-21T17:07:40.354354Z qemu-system-x86_64: warning: This feature depends
> on other features that were not requested: CPUID.8000000AH:EDX.npt [bit 0]

As I said, this is unrelated.

> 2022-04-21T17:07:40.419616Z qemu-system-x86_64: error: failed to set MSR
> 0xc0000104 to 0x100000000
> qemu-system-x86_64: ../../target/i386/kvm/kvm.c:2996: kvm_buf_set_msrs:
> Assertion `ret == cpu->kvm_msr_buf->nmsrs' failed.

And this is the actual issue in the.. KERNEL.

The commit in the kernel which introduce it is this one:

commit 880993138396f8f0be620c425d08f84490c35251
Author: Maxim Levitsky <mlevitsk@redhat.com>
Date:   Tue Mar 22 19:24:48 2022 +0200

    KVM: x86: SVM: fix tsc scaling when the host doesn't support it

which is part of 5.16.0 kernel.  And the commit which fixed this is

commit e910a53fb4f20aa012e46371ffb4c32c8da259b4
Author: Maxim Levitsky <mlevitsk@redhat.com>
Date:   Wed Feb 23 13:56:49 2022 +0200

    KVM: x86: nSVM: disallow userspace setting of MSR_AMD64_TSC_RATIO to non default value when tsc scaling disabled

which is a part of 5.16.12 kernel.

I don't know which is 5.16.0-5-amd64, - but it looks like 5.16.18 is
in Debian now. Is it your current kernel? What does `uname -a' say?

It looks like the only thing you need is to upgrade the kernel.

> Reverting to qemu-system-x86 6.2+dfsg-3 and the VMs start-up perfectly fine
> using the same libvirt xml.

It is because new qemu started using the MSRs it didn't use previously,
and hit this very issue. It is all in the kernel, -- it is the kernel
who reports the list of MSRs it supports, and qemu sets only those MSRs
which the kernel reports are supported. And out of the sudden one of
the reported-as-supported MSRs turned out to be unsupported by the kernel -
that's the meaining of this assert().

Please verify your kernel is at least 5.16.18.

Thanks,

/mjt

> Kernel: Linux 5.16.0-5-amd64 (SMP w/2 CPU threads; PREEMPT)

Message #32 received at 1010026@bugs.debian.org (full text, mbox, reply):

From: Adrian Davey <adrian.davey@gmail.com>

To: mjt@tls.msk.ru

Cc: 1010026@bugs.debian.org

Subject: Re: Bug#1010026: qemu-system-x86: fails to start VM with "host" cpu missing features

Date: Fri, 22 Apr 2022 17:01:58 +0100

[Message part 1 (text/plain, inline)]

HI Michael,

Apologies the reportbug package is installed on a laptop, the issue is on a
headless system, unfortunately debian doesn't allow SMTP from my mail
server so I am trying to get this posted via reportbug -> text editor ->
gmail -> bugs.debian , sometimes I miss replacing parts that are taken from
the laptop system information with the real system information.

This headless server has both Kernel: Linux 5.16.0-6-amd64 as well as Linux
5.17.0-1-amd64 #1 SMP PREEMPT Debian 5.17.3-1 (2022-04-18) x86_64 GNU/Linux
same result as above.

libvirt full log (modified for anonymity) :
2022-04-21 17:07:40.149+0000: starting up libvirt version: 8.2.0, package:
1 (Andrea Bolognani <eof@kiyuko.org> Wed, 20 Apr 2022 22:51:26 +0200), qemu
version: 7.0.0Debian 1:7.0+dfsg-1, kernel: 5.17.0-1-amd64, hostname:
REPLACED_VHOSTNAME
LC_ALL=C \
PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin \
HOME=/var/lib/libvirt/qemu/domain-3-REPLACED_VM_NAME \
XDG_DATA_HOME=/var/lib/libvirt/qemu/domain-3-REPLACED_VM_NAME/.local/share \
XDG_CACHE_HOME=/var/lib/libvirt/qemu/domain-3-REPLACED_VM_NAME/.cache \
XDG_CONFIG_HOME=/var/lib/libvirt/qemu/domain-3-REPLACED_VM_NAME/.config \
/usr/bin/qemu-system-x86_64 \
-name guest=REPLACED_VM_NAME,debug-threads=on \
-S \
-object
'{"qom-type":"secret","id":"masterKey0","format":"raw","file":"/var/lib/libvirt/qemu/domain-3-REPLACED_VM_NAME/master-key.aes"}'
\
-blockdev
'{"driver":"file","filename":"/usr/share/OVMF/OVMF_CODE_4M.ms.fd","node-name":"libvirt-pflash0-storage","auto-read-only":true,"discard":"unmap"}'
\
-blockdev
'{"node-name":"libvirt-pflash0-format","read-only":true,"driver":"raw","file":"libvirt-pflash0-storage"}'
\
-blockdev
'{"driver":"file","filename":"/var/lib/libvirt/qemu/nvram/REPLACED_VM_NAME_VARS.fd","node-name":"libvirt-pflash1-storage","auto-read-only":true,"discard":"unmap"}'
\
-blockdev
'{"node-name":"libvirt-pflash1-format","read-only":false,"driver":"raw","file":"libvirt-pflash1-storage"}'
\
-machine
pc-q35-5.2,usb=off,vmport=off,dump-guest-core=off,pflash0=libvirt-pflash0-format,pflash1=libvirt-pflash1-format,memory-backend=pc.ram
\
-accel kvm \
-cpu
Opteron_G3,vme=on,x2apic=on,tsc-deadline=on,hypervisor=on,arat=on,mmxext=on,fxsr-opt=on,pdpe1gb=on,3dnowext=on,3dnow=on,cmp-legacy=on,cr8legacy=on,3dnowprefetch=on,osvw=on,amd-no-ssb=on,npt=on,nrip-save=on,vmcb-clean=on,svme-addr-chk=on,monitor=off
\
-m 512 \
-object '{"qom-type":"memory-backend-ram","id":"pc.ram","size":536870912}' \
-overcommit mem-lock=off \
-smp 2,sockets=2,cores=1,threads=1 \
-uuid f7722398-98ca-020a-13e7-93de4f798123 \
-no-user-config \
-nodefaults \
-chardev socket,id=charmonitor,fd=31,server=on,wait=off \
-mon chardev=charmonitor,id=monitor,mode=control \
-rtc base=utc,driftfix=slew \
-global kvm-pit.lost_tick_policy=delay \
-no-hpet \
-no-shutdown \
-global ICH9-LPC.disable_s3=1 \
-global ICH9-LPC.disable_s4=1 \
-boot menu=off,strict=on \
-device
'{"driver":"pcie-root-port","port":16,"chassis":1,"id":"pci.1","bus":"pcie.0","multifunction":true,"addr":"0x2"}'
\
-device
'{"driver":"pcie-root-port","port":17,"chassis":2,"id":"pci.2","bus":"pcie.0","addr":"0x2.0x1"}'
\
-device
'{"driver":"pcie-root-port","port":18,"chassis":3,"id":"pci.3","bus":"pcie.0","addr":"0x2.0x2"}'
\
-device
'{"driver":"pcie-root-port","port":19,"chassis":4,"id":"pci.4","bus":"pcie.0","addr":"0x2.0x3"}'
\
-device
'{"driver":"pcie-root-port","port":20,"chassis":5,"id":"pci.5","bus":"pcie.0","addr":"0x2.0x4"}'
\
-device
'{"driver":"pcie-root-port","port":21,"chassis":6,"id":"pci.6","bus":"pcie.0","addr":"0x2.0x5"}'
\
-device
'{"driver":"pcie-root-port","port":22,"chassis":7,"id":"pci.7","bus":"pcie.0","addr":"0x2.0x6"}'
\
-device
'{"driver":"qemu-xhci","p2":15,"p3":15,"id":"usb","bus":"pci.2","addr":"0x0"}'
\
-device
'{"driver":"virtio-scsi-pci","id":"scsi0","bus":"pci.1","addr":"0x0"}' \
-device
'{"driver":"virtio-serial-pci","id":"virtio-serial0","bus":"pci.3","addr":"0x0"}'
\
-blockdev
'{"driver":"file","filename":"/opt/vm/images/REPLACED_VHOSTNAME/default/REPLACED_VM_NAME.qcow2","node-name":"libvirt-1-storage","auto-read-only":true,"discard":"unmap"}'
\
-blockdev
'{"node-name":"libvirt-1-format","read-only":false,"driver":"qcow2","file":"libvirt-1-storage","backing":null}'
\
-device
'{"driver":"virtio-blk-pci","bus":"pci.4","addr":"0x0","drive":"libvirt-1-format","id":"virtio-disk0","bootindex":1}'
\
-netdev tap,fd=32,id=hostnet0,vhost=on,vhostfd=34 \
-device
'{"driver":"virtio-net-pci","netdev":"hostnet0","id":"net0","mac":"52:54:00:a7:24:12","bus":"pcie.0","addr":"0x3"}'
\
-netdev tap,fd=35,id=hostnet1,vhost=on,vhostfd=36 \
-device
'{"driver":"virtio-net-pci","netdev":"hostnet1","id":"net1","mac":"52:54:00:97:27:bf","bus":"pcie.0","addr":"0x6"}'
\
-chardev socket,id=charchannel0,fd=30,server=on,wait=off \
-device
'{"driver":"virtserialport","bus":"virtio-serial0.0","nr":1,"chardev":"charchannel0","id":"channel0","name":"org.qemu.guest_agent.0"}'
\
-audiodev '{"id":"audio1","driver":"none"}' \
-vnc 127.0.0.1:0,audiodev=audio1 \
-device
'{"driver":"virtio-vga","id":"video0","max_outputs":1,"bus":"pcie.0","addr":"0x1"}'
\
-device
'{"driver":"virtio-balloon-pci","id":"balloon0","bus":"pci.5","addr":"0x0"}'
\
-object
'{"qom-type":"rng-random","id":"objrng0","filename":"/dev/urandom"}' \
-device
'{"driver":"virtio-rng-pci","rng":"objrng0","id":"rng0","bus":"pci.6","addr":"0x0"}'
\
-sandbox
on,obsolete=deny,elevateprivileges=deny,spawn=deny,resourcecontrol=deny \
-msg timestamp=on


On Fri, 22 Apr 2022 at 15:54, Michael Tokarev <mjt@tls.msk.ru> wrote:

> 22.04.2022 17:10, Adrian Davey wrote:
> > Package: qemu-system-x86
> > Version: 1:7.0+dfsg-1
> > Severity: normal
> >
> > 2022-04-21T17:07:40.354354Z qemu-system-x86_64: warning: This feature
> depends
> > on other features that were not requested: CPUID.8000000AH:EDX.npt [bit
> 0]
>
> As I said, this is unrelated.
>
> > 2022-04-21T17:07:40.419616Z qemu-system-x86_64: error: failed to set MSR
> > 0xc0000104 to 0x100000000
> > qemu-system-x86_64: ../../target/i386/kvm/kvm.c:2996: kvm_buf_set_msrs:
> > Assertion `ret == cpu->kvm_msr_buf->nmsrs' failed.
>
> And this is the actual issue in the.. KERNEL.
>
> The commit in the kernel which introduce it is this one:
>
> commit 880993138396f8f0be620c425d08f84490c35251
> Author: Maxim Levitsky <mlevitsk@redhat.com>
> Date:   Tue Mar 22 19:24:48 2022 +0200
>
>      KVM: x86: SVM: fix tsc scaling when the host doesn't support it
>
> which is part of 5.16.0 kernel.  And the commit which fixed this is
>
> commit e910a53fb4f20aa012e46371ffb4c32c8da259b4
> Author: Maxim Levitsky <mlevitsk@redhat.com>
> Date:   Wed Feb 23 13:56:49 2022 +0200
>
>      KVM: x86: nSVM: disallow userspace setting of MSR_AMD64_TSC_RATIO to
> non default value when tsc scaling disabled
>
> which is a part of 5.16.12 kernel.
>
> I don't know which is 5.16.0-5-amd64, - but it looks like 5.16.18 is
> in Debian now. Is it your current kernel? What does `uname -a' say?
>
> It looks like the only thing you need is to upgrade the kernel.
>
> > Reverting to qemu-system-x86 6.2+dfsg-3 and the VMs start-up perfectly
> fine
> > using the same libvirt xml.
>
> It is because new qemu started using the MSRs it didn't use previously,
> and hit this very issue. It is all in the kernel, -- it is the kernel
> who reports the list of MSRs it supports, and qemu sets only those MSRs
> which the kernel reports are supported. And out of the sudden one of
> the reported-as-supported MSRs turned out to be unsupported by the kernel -
> that's the meaining of this assert().
>
> Please verify your kernel is at least 5.16.18.
>
> Thanks,
>
> /mjt
>
> > Kernel: Linux 5.16.0-5-amd64 (SMP w/2 CPU threads; PREEMPT)
>

[Message part 2 (text/html, inline)]

Message #37 received at 1010026@bugs.debian.org (full text, mbox, reply):

From: Michael Tokarev <mjt@tls.msk.ru>

To: Adrian Davey <adrian.davey@gmail.com>

Cc: 1010026@bugs.debian.org

Subject: Re: Bug#1010026: qemu-system-x86: fails to start VM with "host" cpu missing features

Date: Fri, 22 Apr 2022 19:50:56 +0300

22.04.2022 19:01, Adrian Davey wrote:
> HI Michael,
> 
> Apologies the reportbug package is installed on a laptop, the issue is on a headless system [..]

That's okay, that happens.

> This headless server has both Kernel: Linux 5.16.0-6-amd64 as well as Linux 5.17.0-1-amd64 #1 SMP PREEMPT Debian 5.17.3-1 (2022-04-18) x86_64 GNU/Linux
> same result as above.

The fix went into 5.17.0-rc3 kernel so it is included in your 5.17 kernel.

Now, 5.16.0-6-amd64 - this one is based on 5.16.18 which includes the fix.
While 5.16.0-5-amd64 is based on 5.16.14, which does not have it.

Are you sure the assertion failure problem occur with any of these *fixed*
kernels - either with  5.16.0-6-amd64 or with 5.17.3-1?

Please post the qemu error message(s) from any ofthe "fixed" kernels.

Also you can try the _older_ kernel, such as 5.15, - that one should work too.

> 
> libvirt full log (modified for anonymity) :

um. Where's the errors in there? I see full qemu command line (for which
I asked initially, before discovering the bad and the good commits).  Now
it seems the command line isn't really necessary (but we do have it anyway
which is good).

Thanks!

/mjt

Message #42 received at 1010026@bugs.debian.org (full text, mbox, reply):

From: Adrian Davey <adrian.davey@gmail.com>

To: Michael Tokarev <mjt@tls.msk.ru>

Cc: 1010026@bugs.debian.org

Subject: Re: Bug#1010026: qemu-system-x86: fails to start VM with "host" cpu missing features

Date: Fri, 22 Apr 2022 18:34:46 +0100

[Message part 1 (text/plain, inline)]

Hi Michael,

Please ignore the 5.16.0-5-amd64 that is the laptop kernel, it only
features on the bug report due to my error, not the server's kernel @
5.16.0-6-amd64 and 5.17.0-1-amd64.

log using 5.17.0-1-amd64 kernel :

2022-04-21 17:07:40.149+0000: starting up libvirt version: 8.2.0, package:
1 (Andrea Bolognani <eof@kiyuko.org> Wed, 20 Apr 2022 22:51:26 +0200), qemu
version: 7.0.0Debian 1:7.0+dfsg-1, kernel: 5.17.0-1-amd64, hostname:
REPLACED_VHOSTNAME
LC_ALL=C \
PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin \
HOME=/var/lib/libvirt/qemu/domain-3-REPLACED_VM_NAME \
XDG_DATA_HOME=/var/lib/libvirt/qemu/domain-3-REPLACED_VM_NAME/.local/share \
XDG_CACHE_HOME=/var/lib/libvirt/qemu/domain-3-REPLACED_VM_NAME/.cache \
XDG_CONFIG_HOME=/var/lib/libvirt/qemu/domain-3-REPLACED_VM_NAME/.config \
/usr/bin/qemu-system-x86_64 \
-name guest=REPLACED_VM_NAME,debug-threads=on \
-S \
-object
'{"qom-type":"secret","id":"masterKey0","format":"raw","file":"/var/lib/libvirt/qemu/domain-3-REPLACED_VM_NAME/master-key.aes"}'
\
-blockdev
'{"driver":"file","filename":"/usr/share/OVMF/OVMF_CODE_4M.ms.fd","node-name":"libvirt-pflash0-storage","auto-read-only":true,"discard":"unmap"}'
\
-blockdev
'{"node-name":"libvirt-pflash0-format","read-only":true,"driver":"raw","file":"libvirt-pflash0-storage"}'
\
-blockdev
'{"driver":"file","filename":"/var/lib/libvirt/qemu/nvram/REPLACED_VM_NAME_VARS.fd","node-name":"libvirt-pflash1-storage","auto-read-only":true,"discard":"unmap"}'
\
-blockdev
'{"node-name":"libvirt-pflash1-format","read-only":false,"driver":"raw","file":"libvirt-pflash1-storage"}'
\
-machine
pc-q35-5.2,usb=off,vmport=off,dump-guest-core=off,pflash0=libvirt-pflash0-format,pflash1=libvirt-pflash1-format,memory-backend=pc.ram
\
-accel kvm \
-cpu
Opteron_G3,vme=on,x2apic=on,tsc-deadline=on,hypervisor=on,arat=on,mmxext=on,fxsr-opt=on,pdpe1gb=on,3dnowext=on,3dnow=on,cmp-legacy=on,cr8legacy=on,3dnowprefetch=on,osvw=on,amd-no-ssb=on,npt=on,nrip-save=on,vmcb-clean=on,svme-addr-chk=on,monitor=off
\
-m 512 \
-object '{"qom-type":"memory-backend-ram","id":"pc.ram","size":536870912}' \
-overcommit mem-lock=off \
-smp 2,sockets=2,cores=1,threads=1 \
-uuid f7722398-98ca-020a-13e7-93de4f798282 \
-no-user-config \
-nodefaults \
-chardev socket,id=charmonitor,fd=31,server=on,wait=off \
-mon chardev=charmonitor,id=monitor,mode=control \
-rtc base=utc,driftfix=slew \
-global kvm-pit.lost_tick_policy=delay \
-no-hpet \
-no-shutdown \
-global ICH9-LPC.disable_s3=1 \
-global ICH9-LPC.disable_s4=1 \
-boot menu=off,strict=on \
-device
'{"driver":"pcie-root-port","port":16,"chassis":1,"id":"pci.1","bus":"pcie.0","multifunction":true,"addr":"0x2"}'
\
-device
'{"driver":"pcie-root-port","port":17,"chassis":2,"id":"pci.2","bus":"pcie.0","addr":"0x2.0x1"}'
\
-device
'{"driver":"pcie-root-port","port":18,"chassis":3,"id":"pci.3","bus":"pcie.0","addr":"0x2.0x2"}'
\
-device
'{"driver":"pcie-root-port","port":19,"chassis":4,"id":"pci.4","bus":"pcie.0","addr":"0x2.0x3"}'
\
-device
'{"driver":"pcie-root-port","port":20,"chassis":5,"id":"pci.5","bus":"pcie.0","addr":"0x2.0x4"}'
\
-device
'{"driver":"pcie-root-port","port":21,"chassis":6,"id":"pci.6","bus":"pcie.0","addr":"0x2.0x5"}'
\
-device
'{"driver":"pcie-root-port","port":22,"chassis":7,"id":"pci.7","bus":"pcie.0","addr":"0x2.0x6"}'
\
-device
'{"driver":"qemu-xhci","p2":15,"p3":15,"id":"usb","bus":"pci.2","addr":"0x0"}'
\
-device
'{"driver":"virtio-scsi-pci","id":"scsi0","bus":"pci.1","addr":"0x0"}' \
-device
'{"driver":"virtio-serial-pci","id":"virtio-serial0","bus":"pci.3","addr":"0x0"}'
\
-blockdev
'{"driver":"file","filename":"/opt/vm/images/REPLACED_VHOSTNAME/default/REPLACED_VM_NAME.qcow2","node-name":"libvirt-1-storage","auto-read-only":true,"discard":"unmap"}'
\
-blockdev
'{"node-name":"libvirt-1-format","read-only":false,"driver":"qcow2","file":"libvirt-1-storage","backing":null}'
\
-device
'{"driver":"virtio-blk-pci","bus":"pci.4","addr":"0x0","drive":"libvirt-1-format","id":"virtio-disk0","bootindex":1}'
\
-netdev tap,fd=32,id=hostnet0,vhost=on,vhostfd=34 \
-device
'{"driver":"virtio-net-pci","netdev":"hostnet0","id":"net0","mac":"52:54:00:a7:24:12","bus":"pcie.0","addr":"0x3"}'
\
-netdev tap,fd=35,id=hostnet1,vhost=on,vhostfd=36 \
-device
'{"driver":"virtio-net-pci","netdev":"hostnet1","id":"net1","mac":"52:54:00:97:27:bf","bus":"pcie.0","addr":"0x6"}'
\
-chardev socket,id=charchannel0,fd=30,server=on,wait=off \
-device
'{"driver":"virtserialport","bus":"virtio-serial0.0","nr":1,"chardev":"charchannel0","id":"channel0","name":"org.qemu.guest_agent.0"}'
\
-audiodev '{"id":"audio1","driver":"none"}' \
-vnc 127.0.0.1:0,audiodev=audio1 \
-device
'{"driver":"virtio-vga","id":"video0","max_outputs":1,"bus":"pcie.0","addr":"0x1"}'
\
-device
'{"driver":"virtio-balloon-pci","id":"balloon0","bus":"pci.5","addr":"0x0"}'
\
-object
'{"qom-type":"rng-random","id":"objrng0","filename":"/dev/urandom"}' \
-device
'{"driver":"virtio-rng-pci","rng":"objrng0","id":"rng0","bus":"pci.6","addr":"0x0"}'
\
-sandbox
on,obsolete=deny,elevateprivileges=deny,spawn=deny,resourcecontrol=deny \
-msg timestamp=on
2022-04-21T17:07:40.354354Z qemu-system-x86_64: warning: This feature
depends on other features that were not requested: CPUID.8000000AH:EDX.npt
[bit 0]
2022-04-21T17:07:40.354467Z qemu-system-x86_64: warning: This feature
depends on other features that were not requested:
CPUID.8000000AH:EDX.nrip-save [bit 3]
2022-04-21T17:07:40.354476Z qemu-system-x86_64: warning: This feature
depends on other features that were not requested:
CPUID.8000000AH:EDX.vmcb-clean [bit 5]
2022-04-21T17:07:40.354482Z qemu-system-x86_64: warning: This feature
depends on other features that were not requested:
CPUID.8000000AH:EDX.svme-addr-chk [bit 28]
2022-04-21T17:07:40.355818Z qemu-system-x86_64: warning: This feature
depends on other features that were not requested: CPUID.8000000AH:EDX.npt
[bit 0]
2022-04-21T17:07:40.355850Z qemu-system-x86_64: warning: This feature
depends on other features that were not requested:
CPUID.8000000AH:EDX.nrip-save [bit 3]
2022-04-21T17:07:40.355857Z qemu-system-x86_64: warning: This feature
depends on other features that were not requested:
CPUID.8000000AH:EDX.vmcb-clean [bit 5]
2022-04-21T17:07:40.355864Z qemu-system-x86_64: warning: This feature
depends on other features that were not requested:
CPUID.8000000AH:EDX.svme-addr-chk [bit 28]
2022-04-21T17:07:40.419616Z qemu-system-x86_64: error: failed to set MSR
0xc0000104 to 0x100000000
qemu-system-x86_64: ../../target/i386/kvm/kvm.c:2996: kvm_buf_set_msrs:
Assertion `ret == cpu->kvm_msr_buf->nmsrs' failed.



log using 5.16.0-6-amd64 kernel:

2022-04-21 17:49:46.873+0000: starting up libvirt version: 8.2.0, package:
1 (Andrea Bolognani <eof@kiyuko.org> Wed, 20 Apr 2022 22:51:26 +0200), qemu
version: 7.0.0Debian 1:7.0+dfsg-1, kernel: 5.16.0-6-amd64, hostname:
REPLACED_VHOSTNAME
LC_ALL=C \
PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin \
HOME=/var/lib/libvirt/qemu/domain-1-REPLACED_VM_NAME \
XDG_DATA_HOME=/var/lib/libvirt/qemu/domain-1-REPLACED_VM_NAME/.local/share \
XDG_CACHE_HOME=/var/lib/libvirt/qemu/domain-1-REPLACED_VM_NAME/.cache \
XDG_CONFIG_HOME=/var/lib/libvirt/qemu/domain-1-REPLACED_VM_NAME/.config \
/usr/bin/qemu-system-x86_64 \
-name guest=REPLACED_VM_NAME,debug-threads=on \
-S \
-object
'{"qom-type":"secret","id":"masterKey0","format":"raw","file":"/var/lib/libvirt/qemu/domain-1-REPLACED_VM_NAME/master-key.aes"}'
\
-blockdev
'{"driver":"file","filename":"/usr/share/OVMF/OVMF_CODE_4M.ms.fd","node-name":"libvirt-pflash0-storage","auto-read-only":true,"discard":"unmap"}'
\
-blockdev
'{"node-name":"libvirt-pflash0-format","read-only":true,"driver":"raw","file":"libvirt-pflash0-storage"}'
\
-blockdev
'{"driver":"file","filename":"/var/lib/libvirt/qemu/nvram/REPLACED_VM_NAME_VARS.fd","node-name":"libvirt-pflash1-storage","auto-read-only":true,"discard":"unmap"}'
\
-blockdev
'{"node-name":"libvirt-pflash1-format","read-only":false,"driver":"raw","file":"libvirt-pflash1-storage"}'
\
-machine
pc-q35-5.2,usb=off,vmport=off,dump-guest-core=off,pflash0=libvirt-pflash0-format,pflash1=libvirt-pflash1-format,memory-backend=pc.ram
\
-accel kvm \
-cpu
Opteron_G3,vme=on,x2apic=on,tsc-deadline=on,hypervisor=on,arat=on,mmxext=on,fxsr-opt=on,pdpe1gb=on,3dnowext=on,3dnow=on,cmp-legacy=on,cr8legacy=on,3dnowprefetch=on,osvw=on,amd-no-ssb=on,npt=on,nrip-save=on,svme-addr-chk=on,monitor=off
\
-m 512 \
-object '{"qom-type":"memory-backend-ram","id":"pc.ram","size":536870912}' \
-overcommit mem-lock=off \
-smp 2,sockets=2,cores=1,threads=1 \
-uuid f7722398-98ca-020a-13e7-93de4f798282 \
-no-user-config \
-nodefaults \
-chardev socket,id=charmonitor,fd=33,server=on,wait=off \
-mon chardev=charmonitor,id=monitor,mode=control \
-rtc base=utc,driftfix=slew \
-global kvm-pit.lost_tick_policy=delay \
-no-hpet \
-no-shutdown \
-global ICH9-LPC.disable_s3=1 \
-global ICH9-LPC.disable_s4=1 \
-boot menu=off,strict=on \
-device
'{"driver":"pcie-root-port","port":16,"chassis":1,"id":"pci.1","bus":"pcie.0","multifunction":true,"addr":"0x2"}'
\
-device
'{"driver":"pcie-root-port","port":17,"chassis":2,"id":"pci.2","bus":"pcie.0","addr":"0x2.0x1"}'
\
-device
'{"driver":"pcie-root-port","port":18,"chassis":3,"id":"pci.3","bus":"pcie.0","addr":"0x2.0x2"}'
\
-device
'{"driver":"pcie-root-port","port":19,"chassis":4,"id":"pci.4","bus":"pcie.0","addr":"0x2.0x3"}'
\
-device
'{"driver":"pcie-root-port","port":20,"chassis":5,"id":"pci.5","bus":"pcie.0","addr":"0x2.0x4"}'
\
-device
'{"driver":"pcie-root-port","port":21,"chassis":6,"id":"pci.6","bus":"pcie.0","addr":"0x2.0x5"}'
\
-device
'{"driver":"pcie-root-port","port":22,"chassis":7,"id":"pci.7","bus":"pcie.0","addr":"0x2.0x6"}'
\
-device
'{"driver":"qemu-xhci","p2":15,"p3":15,"id":"usb","bus":"pci.2","addr":"0x0"}'
\
-device
'{"driver":"virtio-scsi-pci","id":"scsi0","bus":"pci.1","addr":"0x0"}' \
-device
'{"driver":"virtio-serial-pci","id":"virtio-serial0","bus":"pci.3","addr":"0x0"}'
\
-blockdev
'{"driver":"file","filename":"/opt/vm/images/REPLACED_VHOSTNAME/default/REPLACED_VM_NAME.qcow2","node-name":"libvirt-1-storage","auto-read-only":true,"discard":"unmap"}'
\
-blockdev
'{"node-name":"libvirt-1-format","read-only":false,"driver":"qcow2","file":"libvirt-1-storage","backing":null}'
\
-device
'{"driver":"virtio-blk-pci","bus":"pci.4","addr":"0x0","drive":"libvirt-1-format","id":"virtio-disk0","bootindex":1}'
\
-netdev tap,fd=34,id=hostnet0,vhost=on,vhostfd=37 \
-device
'{"driver":"virtio-net-pci","netdev":"hostnet0","id":"net0","mac":"52:54:00:a7:24:12","bus":"pcie.0","addr":"0x3"}'
\
-netdev tap,fd=38,id=hostnet1,vhost=on,vhostfd=36 \
-device
'{"driver":"virtio-net-pci","netdev":"hostnet1","id":"net1","mac":"52:54:00:97:27:bf","bus":"pcie.0","addr":"0x6"}'
\
-chardev socket,id=charchannel0,fd=32,server=on,wait=off \
-device
'{"driver":"virtserialport","bus":"virtio-serial0.0","nr":1,"chardev":"charchannel0","id":"channel0","name":"org.qemu.guest_agent.0"}'
\
-audiodev '{"id":"audio1","driver":"none"}' \
-vnc 127.0.0.1:0,audiodev=audio1 \
-device
'{"driver":"virtio-vga","id":"video0","max_outputs":1,"bus":"pcie.0","addr":"0x1"}'
\
-device
'{"driver":"virtio-balloon-pci","id":"balloon0","bus":"pci.5","addr":"0x0"}'
\
-object
'{"qom-type":"rng-random","id":"objrng0","filename":"/dev/urandom"}' \
-device
'{"driver":"virtio-rng-pci","rng":"objrng0","id":"rng0","bus":"pci.6","addr":"0x0"}'
\
-sandbox
on,obsolete=deny,elevateprivileges=deny,spawn=deny,resourcecontrol=deny \
-msg timestamp=on
2022-04-21T17:49:47.230945Z qemu-system-x86_64: warning: This feature
depends on other features that were not requested: CPUID.8000000AH:EDX.npt
[bit 0]
2022-04-21T17:49:47.231093Z qemu-system-x86_64: warning: This feature
depends on other features that were not requested:
CPUID.8000000AH:EDX.nrip-save [bit 3]
2022-04-21T17:49:47.231103Z qemu-system-x86_64: warning: This feature
depends on other features that were not requested:
CPUID.8000000AH:EDX.svme-addr-chk [bit 28]
2022-04-21T17:49:47.232675Z qemu-system-x86_64: warning: This feature
depends on other features that were not requested: CPUID.8000000AH:EDX.npt
[bit 0]
2022-04-21T17:49:47.232713Z qemu-system-x86_64: warning: This feature
depends on other features that were not requested:
CPUID.8000000AH:EDX.nrip-save [bit 3]
2022-04-21T17:49:47.232722Z qemu-system-x86_64: warning: This feature
depends on other features that were not requested:
CPUID.8000000AH:EDX.svme-addr-chk [bit 28]
2022-04-21T17:49:47.488753Z qemu-system-x86_64: error: failed to set MSR
0xc0000104 to 0x100000000
qemu-system-x86_64: ../../target/i386/kvm/kvm.c:2996: kvm_buf_set_msrs:
Assertion `ret == cpu->kvm_msr_buf->nmsrs' failed.


the difference between the two is the extra 2 lines (I assume 1 per vCPU)
in 5.17, but then the -cpu entry in the command line also adds
vmcb-clean=on, even though the xml file has "host" cpu and never changes
between the two invocations with different kernels.
qemu-system-x86_64: warning: This feature depends on other features that
were not requested: CPUID.8000000AH:EDX.vmcb-clean [bit 5]
qemu-system-x86_64: warning: This feature depends on other features that
were not requested: CPUID.8000000AH:EDX.vmcb-clean [bit 5]

I will try to find a 5.15 kernel on snapshot.debian to try shortly.

Cheers,

Adrian



On Fri, 22 Apr 2022 at 17:50, Michael Tokarev <mjt@tls.msk.ru> wrote:

> 22.04.2022 19:01, Adrian Davey wrote:
> > HI Michael,
> >
> > Apologies the reportbug package is installed on a laptop, the issue is
> on a headless system [..]
>
> That's okay, that happens.
>
> > This headless server has both Kernel: Linux 5.16.0-6-amd64 as well as
> Linux 5.17.0-1-amd64 #1 SMP PREEMPT Debian 5.17.3-1 (2022-04-18) x86_64
> GNU/Linux
> > same result as above.
>
> The fix went into 5.17.0-rc3 kernel so it is included in your 5.17 kernel.
>
> Now, 5.16.0-6-amd64 - this one is based on 5.16.18 which includes the fix.
> While 5.16.0-5-amd64 is based on 5.16.14, which does not have it.
>
> Are you sure the assertion failure problem occur with any of these *fixed*
> kernels - either with  5.16.0-6-amd64 or with 5.17.3-1?
>
> Please post the qemu error message(s) from any ofthe "fixed" kernels.
>
> Also you can try the _older_ kernel, such as 5.15, - that one should work
> too.
>
> >
> > libvirt full log (modified for anonymity) :
>
> um. Where's the errors in there? I see full qemu command line (for which
> I asked initially, before discovering the bad and the good commits).  Now
> it seems the command line isn't really necessary (but we do have it anyway
> which is good).
>
> Thanks!
>
> /mjt
>

[Message part 2 (text/html, inline)]

Message #47 received at 1010026@bugs.debian.org (full text, mbox, reply):

From: Michael Tokarev <mjt@tls.msk.ru>

To: Adrian Davey <adrian.davey@gmail.com>, 1010026@bugs.debian.org

Subject: Re: Bug#1010026: qemu-system-x86: fails to start VM with "host" cpu missing features

Date: Fri, 22 Apr 2022 21:23:46 +0300

Control: retitle -1 qemu-system-x86_64: ../../target/i386/kvm/kvm.c:2996: kvm_buf_set_msrs: Assertion `ret == cpu->kvm_msr_buf->nmsrs' failed.

22.04.2022 20:34, Adrian Davey wrote:
> Hi Michael,
> 
> Please ignore the 5.16.0-5-amd64 that is the laptop kernel, it only features on the bug report due to my error, not the server's kernel @ 
> 5.16.0-6-amd64 and 5.17.0-1-amd64.
> 
> log using 5.17.0-1-amd64 kernel :

Ok, that works. So it looks like the already mentioned commit didn't
actually fix the issue.  Let's retitle the bugreport (it fails to start
due to the assertion failure, the requested feature warnings are just
that - warnings, they're harmless).

And let's ask upstream about this..

Thank you!

/mjt

Message #54 received at 1010026@bugs.debian.org (full text, mbox, reply):

From: Bob Weber <bobrweber2@gmail.com>

To: 1010026@bugs.debian.org

Subject: Bug#1010026: qemu-system-x86: fails to start VM with "host" cpu missing features

Date: Fri, 27 May 2022 18:28:58 -0400

I have this same problem when I upgraded a testing system on May 24.  I also 
installed:

linux-image-5.17.0-2-amd64 from unstable

This is the error from a (all) VM:

vm: error: failed to set MSR 0xc0000104 to 0x100000000

kvm: ../../target/i386/kvm/kvm.c:2996: kvm_buf_set_msrs: Assertion `ret == 
cpu->kvm_msr_buf->nmsrs' failed.



libvirt-daemon-driver-qemu/testing,unstable,now 8.3.0-1 amd64
qemu-system-x86/testing,unstable,now 1:7.0+dfsg-7 amd64

When I downgraded the kernel to:

 linux-image-5.15.0-3-amd64/now 5.15.15-2 amd64   (my previous kernel)

All OK!

...bob

Message #59 received at 1010026@bugs.debian.org (full text, mbox, reply):

From: Michael Tokarev <mjt@tls.msk.ru>

To: Bob Weber <bobrweber2@gmail.com>, 1010026@bugs.debian.org

Subject: Re: Bug#1010026: qemu-system-x86: fails to start VM with "host" cpu missing features

Date: Sun, 29 May 2022 15:01:40 +0300

Control: retitle -1 qemu-system-x86: fails to start: kvm_buf_set_msrs:  Assertion `ret == cpu->kvm_msr_buf->nmsrs' failed.

28.05.2022 01:28, Bob Weber wrote:
> I have this same problem when I upgraded a testing system on May 24.  I also installed:
> 
> linux-image-5.17.0-2-amd64 from unstable
> 
> This is the error from a (all) VM:
> 
> vm: error: failed to set MSR 0xc0000104 to 0x100000000
> 
> kvm: ../../target/i386/kvm/kvm.c:2996: kvm_buf_set_msrs: Assertion `ret == cpu->kvm_msr_buf->nmsrs' failed.
> 
> When I downgraded the kernel to:
> 
>   linux-image-5.15.0-3-amd64/now 5.15.15-2 amd64   (my previous kernel)

Does it work with later kernels?

What CPU do you have?

/mjt

Send a report that this bug log contains spam.