Package: xen-utils-4.4; Maintainer for xen-utils-4.4 is (unknown);
Reported by: Gedalya <gedalya@gedalya.net>
Date: Wed, 29 Oct 2014 21:39:01 UTC
Severity: important
Found in version xen/4.4.1-3
Fixed in version xen/4.4.1-5
Done: Bastian Blank <waldi@debian.org>
Bug is archived. No further changes may be made.
View this report as an mbox folder, status mbox, maintainer mbox
Report forwarded
to debian-bugs-dist@lists.debian.org, Debian Xen Team <pkg-xen-devel@lists.alioth.debian.org>:
Bug#767295; Package xen-utils-4.4.
(Wed, 29 Oct 2014 21:39:07 GMT) (full text, mbox, link).
Acknowledgement sent
to Gedalya <gedalya@gedalya.net>:
New Bug report received and forwarded. Copy sent to Debian Xen Team <pkg-xen-devel@lists.alioth.debian.org>.
(Wed, 29 Oct 2014 21:39:07 GMT) (full text, mbox, link).
Message #5 received at submit@bugs.debian.org (full text, mbox, reply):
Package: xen-utils-4.4 Version: 4.4.1-3 When booting domU's running amd64 jessie, I notice some memory problems with xl. root@xen:~# pmap -x 4121 4121: /usr/lib/xen-4.4/bin/xl create --quiet --defconfig /etc/xen/auto/mail_deb80.cfg Address Kbytes RSS Dirty Mode Mapping 0000000000400000 144 128 0 r-x-- xl 0000000000623000 4 4 4 r---- xl 0000000000624000 8 8 8 rw--- xl 0000000000626000 4 4 4 rw--- [ anon ] 0000000000841000 288 240 240 rw--- [ anon ] 00007f9ca48b4000 23852 14464 14464 rw--- [ anon ] 00007f9ca8000000 132 8 8 rw--- [ anon ] 00007f9ca8021000 65404 0 0 ----- [ anon ] 00007f9cae639000 104 0 0 r-x-- libz.so.1.2.8 << snip >> 00007ffff736e000 132 104 104 rw--- [ stack ] 00007ffff73f4000 8 8 0 r-x-- [ anon ] 00007ffff73f6000 8 0 0 r---- [ anon ] ffffffffff600000 4 0 0 r-x-- [ anon ] ---------------- ------- ------- ------- total kB 119812 17124 15052 After init 6 within the VM: root@xen:~# pmap -x 4121 4121: /usr/lib/xen-4.4/bin/xl create --quiet --defconfig /etc/xen/auto/mail_deb80.cfg Address Kbytes RSS Dirty Mode Mapping 0000000000400000 144 144 0 r-x-- xl 0000000000623000 4 4 4 r---- xl 0000000000624000 8 8 8 rw--- xl 0000000000626000 4 4 4 rw--- [ anon ] 0000000000841000 288 288 288 rw--- [ anon ] 0000000000889000 2292 2292 2292 rw--- [ anon ] 00007f9ca3169000 23852 14464 14464 rw--- [ anon ] 00007f9ca48b4000 23852 14464 14464 rw--- [ anon ] 00007f9ca8000000 132 8 8 rw--- [ anon ] 00007f9ca8021000 65404 0 0 ----- [ anon ] 00007f9cae639000 104 88 0 r-x-- libz.so.1.2.8 << snip >> ---------------- ------- ------- ------- total kB 145964 34620 31864 another init 6: root@xen:~# pmap -x 4121 4121: /usr/lib/xen-4.4/bin/xl create --quiet --defconfig /etc/xen/auto/mail_deb80.cfg Address Kbytes RSS Dirty Mode Mapping 0000000000400000 144 144 0 r-x-- xl 0000000000623000 4 4 4 r---- xl 0000000000624000 8 8 8 rw--- xl 0000000000626000 4 4 4 rw--- [ anon ] 0000000000841000 288 288 288 rw--- [ anon ] 0000000000889000 2292 2292 2292 rw--- [ anon ] 00007f9ca1a1e000 23852 14464 14464 rw--- [ anon ] 00007f9ca3169000 23852 14464 14464 rw--- [ anon ] 00007f9ca48b4000 23852 14464 14464 rw--- [ anon ] << snip >> ---------------- ------- ------- ------- total kB 169816 49084 46328 --------------- For my older domUs running i386 wheezy, the initial size of xl is hugely smaller: # pmap -x 10380 10380: /usr/lib/xen-4.4/bin/xl create aptcache_deb70.cfg Address Kbytes RSS Dirty Mode Mapping 0000000000400000 144 128 0 r-x-- xl 0000000000623000 4 4 4 r---- xl 0000000000624000 8 8 8 rw--- xl 0000000000626000 4 4 4 rw--- [ anon ] 00000000015e3000 132 132 132 rw--- [ anon ] 00007fb484000000 132 8 8 rw--- [ anon ] 00007fb484021000 65404 0 0 ----- [ anon ] 00007fb4893d4000 104 0 0 r-x-- libz.so.1.2.8 << snip >> ---------------- ------- ------- ------- total kB 95808 2604 488 After first init 6: # pmap -x 10380 10380: /usr/lib/xen-4.4/bin/xl create aptcache_deb70.cfg Address Kbytes RSS Dirty Mode Mapping 0000000000400000 144 144 0 r-x-- xl 0000000000623000 4 4 4 r---- xl 0000000000624000 8 8 8 rw--- xl 0000000000626000 4 4 4 rw--- [ anon ] 00000000015e3000 132 132 132 rw--- [ anon ] 0000000001604000 5840 5780 5780 rw--- [ anon ] 00007fb484000000 132 8 8 rw--- [ anon ] 00007fb484021000 65404 0 0 ----- [ anon ] 00007fb4893d4000 104 100 0 r-x-- libz.so.1.2.8 << snip >> ---------------- ------- ------- ------- total kB 101656 8792 6276 And subsequent reboots don't make it grow any more.
Information forwarded
to debian-bugs-dist@lists.debian.org, Debian Xen Team <pkg-xen-devel@lists.alioth.debian.org>:
Bug#767295; Package xen-utils-4.4.
(Thu, 06 Nov 2014 12:18:10 GMT) (full text, mbox, link).
Acknowledgement sent
to Ian Campbell <ijc@hellion.org.uk>:
Extra info received and forwarded to list. Copy sent to Debian Xen Team <pkg-xen-devel@lists.alioth.debian.org>.
(Thu, 06 Nov 2014 12:18:10 GMT) (full text, mbox, link).
Message #10 received at 767295@bugs.debian.org (full text, mbox, reply):
On Wed, 2014-10-29 at 17:36 -0400, Gedalya wrote:
> When booting domU's running amd64 jessie, I notice some memory problems
> with xl.
I ran it (actually, upstream staging-4.4) under valgrind and it reported
after a reboot:
==5722== 24 bytes in 1 blocks are definitely lost in loss record 35 of 72
==5722== at 0x4026B2D: malloc (vg_replace_malloc.c:299)
==5722== by 0x41739FF: strdup (strdup.c:43)
==5722== by 0x405F38E: libxl__device_disk_local_initiate_attach (libxl.c:2681)
==5722== by 0x408D2A6: libxl__bootloader_run (libxl_bootloader.c:385)
==5722== by 0x406B50C: initiate_domain_create (libxl_create.c:807)
==5722== by 0x406B50C: do_domain_create (libxl_create.c:1354)
==5722== by 0x406B6AF: libxl_domain_create_new (libxl_create.c:1377)
==5722== by 0x8056182: create_domain (xl_cmdimpl.c:2254)
==5722== by 0x8059F27: main_create (xl_cmdimpl.c:4407)
==5722== by 0x804E119: main (xl.c:360)
==5722==
==5722== 28 bytes in 1 blocks are definitely lost in loss record 39 of 72
==5722== at 0x4026B2D: malloc (vg_replace_malloc.c:299)
==5722== by 0x4279EBB: read_message (xs.c:1143)
==5722== by 0x427A89B: read_thread (xs.c:1219)
==5722== by 0x40E7953: start_thread (pthread_create.c:304)
==5722== by 0x41CF95D: clone (clone.S:130)
And the bootloader one appears to increase with each reboot.
FTR I did:
valgrind --leak-check=full xl cr -F /etc/xen/d32-1
(-F runs xl in the foreground, so this won't return to a prompt).
Then in another window:
xl reboot d32-1
(equivalent to running init 6).
Ian.
Information forwarded
to debian-bugs-dist@lists.debian.org, Debian Xen Team <pkg-xen-devel@lists.alioth.debian.org>:
Bug#767295; Package xen-utils-4.4.
(Thu, 06 Nov 2014 14:15:04 GMT) (full text, mbox, link).
Acknowledgement sent
to Ian Campbell <ijc@hellion.org.uk>:
Extra info received and forwarded to list. Copy sent to Debian Xen Team <pkg-xen-devel@lists.alioth.debian.org>.
(Thu, 06 Nov 2014 14:15:04 GMT) (full text, mbox, link).
Message #15 received at 767295@bugs.debian.org (full text, mbox, reply):
On Thu, 2014-11-06 at 12:14 +0000, Ian Campbell wrote: > On Wed, 2014-10-29 at 17:36 -0400, Gedalya wrote: > > When booting domU's running amd64 jessie, I notice some memory problems > > with xl. > > I ran it (actually, upstream staging-4.4) under valgrind and it reported > after a reboot: > > ==5722== 24 bytes in 1 blocks are definitely lost in loss record 35 of 72 > ==5722== at 0x4026B2D: malloc (vg_replace_malloc.c:299) > ==5722== by 0x41739FF: strdup (strdup.c:43) > ==5722== by 0x405F38E: libxl__device_disk_local_initiate_attach (libxl.c:2681) > ==5722== by 0x408D2A6: libxl__bootloader_run (libxl_bootloader.c:385) > ==5722== by 0x406B50C: initiate_domain_create (libxl_create.c:807) > ==5722== by 0x406B50C: do_domain_create (libxl_create.c:1354) > ==5722== by 0x406B6AF: libxl_domain_create_new (libxl_create.c:1377) > ==5722== by 0x8056182: create_domain (xl_cmdimpl.c:2254) > ==5722== by 0x8059F27: main_create (xl_cmdimpl.c:4407) > ==5722== by 0x804E119: main (xl.c:360) > ==5722== > ==5722== 28 bytes in 1 blocks are definitely lost in loss record 39 of 72 > ==5722== at 0x4026B2D: malloc (vg_replace_malloc.c:299) > ==5722== by 0x4279EBB: read_message (xs.c:1143) > ==5722== by 0x427A89B: read_thread (xs.c:1219) > ==5722== by 0x40E7953: start_thread (pthread_create.c:304) > ==5722== by 0x41CF95D: clone (clone.S:130) I've posted a fix for this upstream: http://lists.xen.org/archives/html/xen-devel/2014-11/msg00542.html I've requested that it go into the next 4.4.x release. I also plan to backport it to the package regardless once it is accepted into the dev branch. Ian.
Information forwarded
to debian-bugs-dist@lists.debian.org, Debian Xen Team <pkg-xen-devel@lists.alioth.debian.org>:
Bug#767295; Package xen-utils-4.4.
(Sat, 08 Nov 2014 02:45:04 GMT) (full text, mbox, link).
Acknowledgement sent
to Gedalya <gedalya@gedalya.net>:
Extra info received and forwarded to list. Copy sent to Debian Xen Team <pkg-xen-devel@lists.alioth.debian.org>.
(Sat, 08 Nov 2014 02:45:04 GMT) (full text, mbox, link).
Message #20 received at 767295@bugs.debian.org (full text, mbox, reply):
[Message part 1 (text/plain, inline)]
On 11/06/2014 09:12 AM, Ian Campbell wrote: > I've posted a fix for this upstream: > http://lists.xen.org/archives/html/xen-devel/2014-11/msg00542.html > > I've requested that it go into the next 4.4.x release. I also plan to > backport it to the package regardless once it is accepted into the dev > branch. > > Ian. So I've tried building xen 4.4.1-3 with this patch (attached), and I replaced the binary packages libxen-4.4 and xen-utils-4.4 with the new ones. The situation is the same: the xl process for jessie amd64 guests starts off at a resident size of 15mb as opposed to ~500kb for wheezy i386 guests. At guest reboot it jumps to 17mb and then, after a few seconds, to ~34mb. I also rebooted the host to help make sure the test is valid. Thanks, Gedalya
[0039-do-not-leak-diskpath-during-localdisk-attach.patch (text/x-patch, attachment)]
Information forwarded
to debian-bugs-dist@lists.debian.org, Debian Xen Team <pkg-xen-devel@lists.alioth.debian.org>:
Bug#767295; Package xen-utils-4.4.
(Sat, 08 Nov 2014 10:42:19 GMT) (full text, mbox, link).
Acknowledgement sent
to Ian Campbell <ijc@hellion.org.uk>:
Extra info received and forwarded to list. Copy sent to Debian Xen Team <pkg-xen-devel@lists.alioth.debian.org>.
(Sat, 08 Nov 2014 10:42:20 GMT) (full text, mbox, link).
Message #25 received at 767295@bugs.debian.org (full text, mbox, reply):
On Fri, 2014-11-07 at 21:32 -0500, Gedalya wrote: > On 11/06/2014 09:12 AM, Ian Campbell wrote: > > I've posted a fix for this upstream: > > http://lists.xen.org/archives/html/xen-devel/2014-11/msg00542.html > > > > I've requested that it go into the next 4.4.x release. I also plan to > > backport it to the package regardless once it is accepted into the dev > > branch. > > > > Ian. > > So I've tried building xen 4.4.1-3 with this patch (attached), and I > replaced the binary packages libxen-4.4 and xen-utils-4.4 with the new ones. > The situation is the same: the xl process for jessie amd64 guests starts > off at a resident size of 15mb as opposed to ~500kb for wheezy i386 > guests. At guest reboot it jumps to 17mb and then, after a few seconds, > to ~34mb. > I also rebooted the host to help make sure the test is valid. Please can you try running xl under valgrind, something similar to what I described earlier should work. Could you also post your guest cfg file please. Thanks, Ian.
Information forwarded
to debian-bugs-dist@lists.debian.org, Debian Xen Team <pkg-xen-devel@lists.alioth.debian.org>:
Bug#767295; Package xen-utils-4.4.
(Sat, 08 Nov 2014 13:36:04 GMT) (full text, mbox, link).
Acknowledgement sent
to Gedalya <gedalya@gedalya.net>:
Extra info received and forwarded to list. Copy sent to Debian Xen Team <pkg-xen-devel@lists.alioth.debian.org>.
(Sat, 08 Nov 2014 13:36:04 GMT) (full text, mbox, link).
Message #30 received at 767295@bugs.debian.org (full text, mbox, reply):
On 11/08/2014 05:41 AM, Ian Campbell wrote:
> Please can you try running xl under valgrind, something similar to what
> I described earlier should work.
I guess it didn't find much..
# valgrind --leak-check=full xl cr -F /etc/xen/auto/asterisk_deb80.cfg
==6736== Memcheck, a memory error detector
==6736== Copyright (C) 2002-2013, and GNU GPL'd, by Julian Seward et al.
==6736== Using Valgrind-3.10.0 and LibVEX; rerun with -h for copyright info
==6736== Command: /usr/sbin/xl cr -F /etc/xen/auto/asterisk_deb80.cfg
==6736==
==6740==
==6740== HEAP SUMMARY:
==6740== in use at exit: 11,663 bytes in 40 blocks
==6740== total heap usage: 74 allocs, 34 frees, 44,570 bytes allocated
==6740==
==6740== LEAK SUMMARY:
==6740== definitely lost: 0 bytes in 0 blocks
==6740== indirectly lost: 0 bytes in 0 blocks
==6740== possibly lost: 0 bytes in 0 blocks
==6740== still reachable: 11,663 bytes in 40 blocks
==6740== suppressed: 0 bytes in 0 blocks
==6740== Reachable blocks (those to which a pointer was found) are not
shown.
==6740== To see them, rerun with: --leak-check=full --show-leak-kinds=all
==6740==
==6740== For counts of detected and suppressed errors, rerun with: -v
==6740== ERROR SUMMARY: 0 errors from 0 contexts (suppressed: 0 from 0)
==6739==
==6739== HEAP SUMMARY:
==6739== in use at exit: 10,711 bytes in 35 blocks
==6739== total heap usage: 63 allocs, 28 frees, 35,257 bytes allocated
==6739==
==6739== LEAK SUMMARY:
==6739== definitely lost: 0 bytes in 0 blocks
==6739== indirectly lost: 0 bytes in 0 blocks
==6739== possibly lost: 0 bytes in 0 blocks
==6739== still reachable: 10,711 bytes in 35 blocks
==6739== suppressed: 0 bytes in 0 blocks
==6739== Reachable blocks (those to which a pointer was found) are not
shown.
==6739== To see them, rerun with: --leak-check=full --show-leak-kinds=all
==6739==
==6739== For counts of detected and suppressed errors, rerun with: -v
==6739== ERROR SUMMARY: 0 errors from 0 contexts (suppressed: 0 from 0)
==6744==
==6744== HEAP SUMMARY:
==6744== in use at exit: 4,827 bytes in 42 blocks
==6744== total heap usage: 91 allocs, 49 frees, 38,343 bytes allocated
==6744==
==6744== LEAK SUMMARY:
==6744== definitely lost: 0 bytes in 0 blocks
==6744== indirectly lost: 0 bytes in 0 blocks
==6744== possibly lost: 0 bytes in 0 blocks
==6744== still reachable: 4,827 bytes in 42 blocks
==6744== suppressed: 0 bytes in 0 blocks
==6744== Reachable blocks (those to which a pointer was found) are not
shown.
==6744== To see them, rerun with: --leak-check=full --show-leak-kinds=all
==6744==
==6744== For counts of detected and suppressed errors, rerun with: -v
==6744== ERROR SUMMARY: 0 errors from 0 contexts (suppressed: 0 from 0)
==6745==
==6745== HEAP SUMMARY:
==6745== in use at exit: 13,034 bytes in 44 blocks
==6745== total heap usage: 97 allocs, 53 frees, 48,672 bytes allocated
==6745==
==6745== LEAK SUMMARY:
==6745== definitely lost: 0 bytes in 0 blocks
==6745== indirectly lost: 0 bytes in 0 blocks
==6745== possibly lost: 0 bytes in 0 blocks
==6745== still reachable: 13,034 bytes in 44 blocks
==6745== suppressed: 0 bytes in 0 blocks
==6745== Reachable blocks (those to which a pointer was found) are not
shown.
==6745== To see them, rerun with: --leak-check=full --show-leak-kinds=all
==6745==
==6745== For counts of detected and suppressed errors, rerun with: -v
==6745== ERROR SUMMARY: 0 errors from 0 contexts (suppressed: 0 from 0)
==6738==
==6738== HEAP SUMMARY:
==6738== in use at exit: 11,017 bytes in 41 blocks
==6738== total heap usage: 91 allocs, 50 frees, 48,555 bytes allocated
==6738==
==6738== LEAK SUMMARY:
==6738== definitely lost: 0 bytes in 0 blocks
==6738== indirectly lost: 0 bytes in 0 blocks
==6738== possibly lost: 0 bytes in 0 blocks
==6738== still reachable: 11,017 bytes in 41 blocks
==6738== suppressed: 0 bytes in 0 blocks
==6738== Reachable blocks (those to which a pointer was found) are not
shown.
==6738== To see them, rerun with: --leak-check=full --show-leak-kinds=all
==6738==
==6738== For counts of detected and suppressed errors, rerun with: -v
==6738== ERROR SUMMARY: 0 errors from 0 contexts (suppressed: 0 from 0)
Parsing config from /etc/xen/auto/asterisk_deb80.cfg
Waiting for domain asterisk_deb08 (domid 8) to die [pid 6736]
Domain 8 has shut down, reason code 1 0x1
Action for shutdown reason code 1 is restart
Domain 8 needs to be cleaned up: destroying the domain
Done. Rebooting now
Waiting for domain asterisk_deb08 (domid 9) to die [pid 6736]
Domain 9 has shut down, reason code 0 0x0
Action for shutdown reason code 0 is destroy
Domain 9 needs to be cleaned up: destroying the domain
Done. Exiting now
> Could you also post your guest cfg file please.
bootloader="pygrub"
memory = 1024
name = "asterisk_deb08"
vcpus = 2
pvh = 1
vif = ['mac=00:16:3e:00:00:12, bridge=breth1', 'mac=00:16:3e:00:00:13,
bridge=breth0']
disk = ['phy:/dev/rvg0/asterisk_deb80_rootfs,xvda,w',
'phy:/dev/rvg0/asterisk_deb80_var,xvdb,w',
'phy:/dev/rvg0/asterisk_deb80_rec,xvdc,w',
'phy:/dev/rvg0/asterisk_deb80_swap,xvdd,w']
(The pvh part doesn't seem to make a difference)
Information forwarded
to debian-bugs-dist@lists.debian.org, Debian Xen Team <pkg-xen-devel@lists.alioth.debian.org>:
Bug#767295; Package xen-utils-4.4.
(Sun, 09 Nov 2014 12:06:05 GMT) (full text, mbox, link).
Acknowledgement sent
to Ian Campbell <ijc@hellion.org.uk>:
Extra info received and forwarded to list. Copy sent to Debian Xen Team <pkg-xen-devel@lists.alioth.debian.org>.
(Sun, 09 Nov 2014 12:06:05 GMT) (full text, mbox, link).
Message #35 received at 767295@bugs.debian.org (full text, mbox, reply):
On Sat, 2014-11-08 at 08:32 -0500, Gedalya wrote: > On 11/08/2014 05:41 AM, Ian Campbell wrote: > > Please can you try running xl under valgrind, something similar to what > > I described earlier should work. > I guess it didn't find much.. Indeed not :-/ I managed to get a test box up and running, so I backported a couple of additional upstream patches to help valgrind analyse Xen toolstacks and even with those I'm not seeing leaks. However with pmap I am seeing an additional +00007f37f516a000 23852 14464 14464 rw--- [ anon ] with each reboot. The size seems to be the same each time. I can't quite figure out what this new mapping is. I'll keep digging. [...] > > Could you also post your guest cfg file please. > Thanks, nothing unusual here but worth confirming. Ian. [0] b1dfc531d8ad and 171c6d7ac17e, FWIW
Information forwarded
to debian-bugs-dist@lists.debian.org, Debian Xen Team <pkg-xen-devel@lists.alioth.debian.org>:
Bug#767295; Package xen-utils-4.4.
(Sun, 09 Nov 2014 12:21:05 GMT) (full text, mbox, link).
Acknowledgement sent
to Gedalya <gedalya@gedalya.net>:
Extra info received and forwarded to list. Copy sent to Debian Xen Team <pkg-xen-devel@lists.alioth.debian.org>.
(Sun, 09 Nov 2014 12:21:05 GMT) (full text, mbox, link).
Message #40 received at 767295@bugs.debian.org (full text, mbox, reply):
On 11/09/2014 07:02 AM, Ian Campbell wrote: > On Sat, 2014-11-08 at 08:32 -0500, Gedalya wrote: >> On 11/08/2014 05:41 AM, Ian Campbell wrote: >>> Please can you try running xl under valgrind, something similar to what >>> I described earlier should work. >> I guess it didn't find much.. > Indeed not :-/ > > I managed to get a test box up and running, so I backported a couple of > additional upstream patches to help valgrind analyse Xen toolstacks and > even with those I'm not seeing leaks. > > However with pmap I am seeing an additional > > +00007f37f516a000 23852 14464 14464 rw--- [ anon ] > > with each reboot. The size seems to be the same each time. I can't > quite figure out what this new mapping is. I'll keep digging. > > [...] Yea. That's precisely what I was seeing. And even the first one shouldn't be there. There is no such block on xl processes for squeeze or wheezy VMs. That memory block is just a bit larger than the size of the initrd in the VM, could there be a connection? > >>> Could you also post your guest cfg file please. > Thanks, nothing unusual here but worth confirming. > > Ian. > > [0] b1dfc531d8ad and 171c6d7ac17e, FWIW >
Information forwarded
to debian-bugs-dist@lists.debian.org, Debian Xen Team <pkg-xen-devel@lists.alioth.debian.org>:
Bug#767295; Package xen-utils-4.4.
(Sun, 09 Nov 2014 12:33:09 GMT) (full text, mbox, link).
Acknowledgement sent
to Gedalya <gedalya@gedalya.net>:
Extra info received and forwarded to list. Copy sent to Debian Xen Team <pkg-xen-devel@lists.alioth.debian.org>.
(Sun, 09 Nov 2014 12:33:09 GMT) (full text, mbox, link).
Message #45 received at 767295@bugs.debian.org (full text, mbox, reply):
On 11/09/2014 07:18 AM, Gedalya wrote: > That memory block is just a bit larger than the size of the initrd in > the VM, could there be a connection? Nope. I changed the initrd to 2.6mb and that memory block is still exactly at 23852 / 14464 / 14464. Actually the process size is around 12 mb when pygrub is counting down to boot, then jumps up to 14+ mb
Information forwarded
to debian-bugs-dist@lists.debian.org, Debian Xen Team <pkg-xen-devel@lists.alioth.debian.org>:
Bug#767295; Package xen-utils-4.4.
(Sun, 09 Nov 2014 15:00:10 GMT) (full text, mbox, link).
Acknowledgement sent
to Ian Campbell <ijc@hellion.org.uk>:
Extra info received and forwarded to list. Copy sent to Debian Xen Team <pkg-xen-devel@lists.alioth.debian.org>.
(Sun, 09 Nov 2014 15:00:10 GMT) (full text, mbox, link).
Message #50 received at 767295@bugs.debian.org (full text, mbox, reply):
On Sun, 2014-11-09 at 07:28 -0500, Gedalya wrote:
> On 11/09/2014 07:18 AM, Gedalya wrote:
> > That memory block is just a bit larger than the size of the initrd in
> > the VM, could there be a connection?
>
> Nope. I changed the initrd to 2.6mb and that memory block is still
> exactly at 23852 / 14464 / 14464.
> Actually the process size is around 12 mb when pygrub is counting down
> to boot, then jumps up to 14+ mb
Running under gdb and dumping the contents of the leak block it appears
to be the uncompressed payout of the vmlinuz
i.e. using bzexplode[0]:
$ file../bzexplode /boot/vmlinuz-3.16-3-amd64 | xzcat | od -t x4 | less
0000000 464c457f 00010102 00000000 00000000
0000020 003e0002 00000001 01000000 00000000
0000040 00000040 00000000 00e1f160 00000000
vs:
(gdb) x/32xw 0x00007fffea8b5000
0x7fffea8b5000: 0x00000000 0x00000000 0x0174b002 0x00000000
0x7fffea8b5010: 0x464c457f 0x00010102 0x00000000 0x00000000
0x7fffea8b5020: 0x003e0002 0x00000001 0x01000000 0x00000000
(the first 4 words are no doubt the allocator's metadata).
I think I can see the leak now, it seems to be present in the latest
upstream git tree too.
Ian.
[0]
http://lists.xen.org/archives/html/xen-devel/2012-02/txtVnQboj3Whe.txt
Information forwarded
to debian-bugs-dist@lists.debian.org, Debian Xen Team <pkg-xen-devel@lists.alioth.debian.org>:
Bug#767295; Package xen-utils-4.4.
(Fri, 21 Nov 2014 03:15:05 GMT) (full text, mbox, link).
Acknowledgement sent
to Gedalya <gedalya@gedalya.net>:
Extra info received and forwarded to list. Copy sent to Debian Xen Team <pkg-xen-devel@lists.alioth.debian.org>.
(Fri, 21 Nov 2014 03:15:05 GMT) (full text, mbox, link).
Message #55 received at 767295@bugs.debian.org (full text, mbox, reply):
On 11/20/2014 03:21 PM, Konrad Rzeszutek Wilk wrote:
> On Thu, Nov 20, 2014 at 03:48:47PM +0000, Ian Campbell wrote:
>> The libxc xc_dom_* infrastructure uses a very simple malloc memory pool which
>> is freed by xc_dom_release. However the various xc_try_*_decode routines (other
>> than the gzip one) just use plain malloc/realloc and therefore the buffer ends
>> up leaked.
>>
>> The memory pool currently supports mmap'd buffers as well as a directly
>> allocated buffers, however the try decode routines make use of realloc and do
>> not fit well into this model. Introduce a concept of an external memory block
>> to the memory pool and provide an interface to register such memory.
>>
>> The mmap_ptr and mmap_len fields of the memblock tracking struct lose their
>> mmap_ prefix since they are now also used for external memory blocks.
>>
>> We are only seeing this now because the gzip decoder doesn't leak and it's only
>> relatively recently that kernels in the wild have switched to better
>> compression.
>>
>> This is https://bugs.debian.org/767295
>>
>> Reported by: Gedalya <gedalya@gedalya.net>
> Gedelya,
>
> Could you also test this patch to make sure it does fix the
> reported issue please?
So here's what happens now.
1. Starts up tiny
2. reboot: leak
3. reboot: freed (process larger, but the delta is all/mostly shared pages)
4. reboot: leak
5. reboot: freed
etc..
root@xen:~/xen-pkgs# xl cr /etc/xen/auto/asterisk_deb80.cfg
Parsing config from /etc/xen/auto/asterisk_deb80.cfg
root@xen:~/xen-pkgs# ps aux | grep asterisk_deb80
root 22981 0.0 0.0 95968 588 ? SLsl 21:55 0:00
/usr/lib/xen-4.4/bin/xl cr /etc/xen/auto/asterisk_deb80.cfg
root@xen:~/xen-pkgs# pmap -x 22981
22981: /usr/lib/xen-4.4/bin/xl cr /etc/xen/auto/asterisk_deb80.cfg
Address Kbytes RSS Dirty Mode Mapping
0000000000400000 144 128 0 r-x-- xl
0000000000623000 4 4 4 r---- xl
0000000000624000 8 8 8 rw--- xl
0000000000626000 4 4 4 rw--- [ anon ]
00000000009a6000 288 240 240 rw--- [ anon ]
00007f14d4000000 132 8 8 rw--- [ anon ]
00007f14d4021000 65404 0 0 ----- [ anon ]
<< snip >>
---------------- ------- ------- -------
total kB 95968 2728 596
--- reboot domu ---
root@xen:~/xen-pkgs# ps aux | grep asterisk_deb80
root 22981 0.6 3.3 131652 20008 ? SLsl 21:55 0:00
/usr/lib/xen-4.4/bin/xl cr /etc/xen/auto/asterisk_deb80.cfg
root@xen:~/xen-pkgs# pmap -x 22981
22981: /usr/lib/xen-4.4/bin/xl cr /etc/xen/auto/asterisk_deb80.cfg
Address Kbytes RSS Dirty Mode Mapping
0000000000400000 144 144 0 r-x-- xl
0000000000623000 4 4 4 r---- xl
0000000000624000 8 8 8 rw--- xl
0000000000626000 4 4 4 rw--- [ anon ]
00000000009a6000 288 288 288 rw--- [ anon ]
00000000009ee000 35676 16772 16772 rw--- [ anon ]
00007f14d4000000 132 8 8 rw--- [ anon ]
00007f14d4021000 65404 0 0 ----- [ anon ]
00007f14d9ae0000 104 100 0 r-x-- libz.so.1.2.8
<< snip >>
---------------- ------- ------- -------
total kB 131652 20072 17424
--- reboot domu ---
root@xen:~/xen-pkgs# ps aux | grep asterisk_deb80
root 22981 0.5 0.5 95876 3136 ? SLsl 21:55 0:01
/usr/lib/xen-4.4/bin/xl cr /etc/xen/auto/asterisk_deb80.cfg
root@xen:~/xen-pkgs# pmap -x 22981
22981: /usr/lib/xen-4.4/bin/xl cr /etc/xen/auto/asterisk_deb80.cfg
Address Kbytes RSS Dirty Mode Mapping
0000000000400000 144 144 0 r-x-- xl
0000000000623000 4 4 4 r---- xl
0000000000624000 8 8 8 rw--- xl
0000000000626000 4 4 4 rw--- [ anon ]
00000000009a6000 188 188 188 rw--- [ anon ]
00007f14d4000000 132 8 8 rw--- [ anon ]
00007f14d4021000 65404 0 0 ----- [ anon ]
00007f14d9ae0000 104 100 0 r-x-- libz.so.1.2.8
<< snip >>
---------------- ------- ------- -------
total kB 95876 3200 552
--- reboot domu ---
root@xen:~/xen-pkgs# ps aux | grep asterisk_deb80
root 22981 0.6 3.4 134660 20548 ? SLsl 21:55 0:02
/usr/lib/xen-4.4/bin/xl cr /etc/xen/auto/asterisk_deb80.cfg
root@xen:~/xen-pkgs# pmap -x 22981
22981: /usr/lib/xen-4.4/bin/xl cr /etc/xen/auto/asterisk_deb80.cfg
Address Kbytes RSS Dirty Mode Mapping
0000000000400000 144 144 0 r-x-- xl
0000000000623000 4 4 4 r---- xl
0000000000624000 8 8 8 rw--- xl
0000000000626000 4 4 4 rw--- [ anon ]
00000000009a6000 188 188 188 rw--- [ anon ]
00000000009d5000 38784 17348 17348 rw--- [ anon ]
00007f14d4000000 132 8 8 rw--- [ anon ]
00007f14d4021000 65404 0 0 ----- [ anon ]
00007f14d9ae0000 104 100 0 r-x-- libz.so.1.2.8
<< snip >>
---------------- ------- ------- -------
total kB 134660 20548 17900
--- reboot domu ---
root@xen:~/xen-pkgs# ps aux | grep asterisk_deb80
root 22981 0.6 0.5 95876 3200 ? SLsl 21:55 0:03
/usr/lib/xen-4.4/bin/xl cr /etc/xen/auto/asterisk_deb80.cfg
root@xen:~/xen-pkgs# pmap -x 22981
22981: /usr/lib/xen-4.4/bin/xl cr /etc/xen/auto/asterisk_deb80.cfg
Address Kbytes RSS Dirty Mode Mapping
0000000000400000 144 144 0 r-x-- xl
0000000000623000 4 4 4 r---- xl
0000000000624000 8 8 8 rw--- xl
0000000000626000 4 4 4 rw--- [ anon ]
00000000009a6000 188 188 188 rw--- [ anon ]
00007f14d4000000 132 8 8 rw--- [ anon ]
00007f14d4021000 65404 0 0 ----- [ anon ]
00007f14d9ae0000 104 100 0 r-x-- libz.so.1.2.8
<< snip >>
---------------- ------- ------- -------
total kB 95876 3200 552
Tried valgrind, it doesn't look like it was able to see what was going on
root@xen:~# valgrind --leak-check=full xl cr -F
/etc/xen/auto/asterisk_deb80.cfg
==24967== Memcheck, a memory error detector
==24967== Copyright (C) 2002-2013, and GNU GPL'd, by Julian Seward et al.
==24967== Using Valgrind-3.10.0 and LibVEX; rerun with -h for copyright info
==24967== Command: /usr/sbin/xl cr -F /etc/xen/auto/asterisk_deb80.cfg
==24967==
==24971==
==24971== HEAP SUMMARY:
==24971== in use at exit: 11,695 bytes in 41 blocks
==24971== total heap usage: 75 allocs, 34 frees, 44,602 bytes allocated
==24971==
==24971== LEAK SUMMARY:
==24971== definitely lost: 0 bytes in 0 blocks
==24971== indirectly lost: 0 bytes in 0 blocks
==24971== possibly lost: 0 bytes in 0 blocks
==24971== still reachable: 11,695 bytes in 41 blocks
==24971== suppressed: 0 bytes in 0 blocks
==24971== Reachable blocks (those to which a pointer was found) are not
shown.
==24971== To see them, rerun with: --leak-check=full --show-leak-kinds=all
==24971==
==24971== For counts of detected and suppressed errors, rerun with: -v
==24971== ERROR SUMMARY: 0 errors from 0 contexts (suppressed: 0 from 0)
==24970==
==24970== HEAP SUMMARY:
==24970== in use at exit: 10,743 bytes in 36 blocks
==24970== total heap usage: 64 allocs, 28 frees, 35,289 bytes allocated
==24970==
==24970== LEAK SUMMARY:
==24970== definitely lost: 0 bytes in 0 blocks
==24970== indirectly lost: 0 bytes in 0 blocks
==24970== possibly lost: 0 bytes in 0 blocks
==24970== still reachable: 10,743 bytes in 36 blocks
==24970== suppressed: 0 bytes in 0 blocks
==24970== Reachable blocks (those to which a pointer was found) are not
shown.
==24970== To see them, rerun with: --leak-check=full --show-leak-kinds=all
==24970==
==24970== For counts of detected and suppressed errors, rerun with: -v
==24970== ERROR SUMMARY: 0 errors from 0 contexts (suppressed: 0 from 0)
==24975==
==24975== HEAP SUMMARY:
==24975== in use at exit: 4,859 bytes in 43 blocks
==24975== total heap usage: 92 allocs, 49 frees, 38,375 bytes allocated
==24975==
==24975== LEAK SUMMARY:
==24975== definitely lost: 0 bytes in 0 blocks
==24975== indirectly lost: 0 bytes in 0 blocks
==24975== possibly lost: 0 bytes in 0 blocks
==24975== still reachable: 4,859 bytes in 43 blocks
==24975== suppressed: 0 bytes in 0 blocks
==24975== Reachable blocks (those to which a pointer was found) are not
shown.
==24975== To see them, rerun with: --leak-check=full --show-leak-kinds=all
==24975==
==24975== For counts of detected and suppressed errors, rerun with: -v
==24975== ERROR SUMMARY: 0 errors from 0 contexts (suppressed: 0 from 0)
==24976==
==24976== HEAP SUMMARY:
==24976== in use at exit: 13,066 bytes in 45 blocks
==24976== total heap usage: 98 allocs, 53 frees, 48,704 bytes allocated
==24976==
==24976== LEAK SUMMARY:
==24976== definitely lost: 0 bytes in 0 blocks
==24976== indirectly lost: 0 bytes in 0 blocks
==24976== possibly lost: 0 bytes in 0 blocks
==24976== still reachable: 13,066 bytes in 45 blocks
==24976== suppressed: 0 bytes in 0 blocks
==24976== Reachable blocks (those to which a pointer was found) are not
shown.
==24976== To see them, rerun with: --leak-check=full --show-leak-kinds=all
==24976==
==24976== For counts of detected and suppressed errors, rerun with: -v
==24976== ERROR SUMMARY: 0 errors from 0 contexts (suppressed: 0 from 0)
==24969==
==24969== HEAP SUMMARY:
==24969== in use at exit: 11,049 bytes in 42 blocks
==24969== total heap usage: 92 allocs, 50 frees, 48,587 bytes allocated
==24969==
==24969== LEAK SUMMARY:
==24969== definitely lost: 0 bytes in 0 blocks
==24969== indirectly lost: 0 bytes in 0 blocks
==24969== possibly lost: 0 bytes in 0 blocks
==24969== still reachable: 11,049 bytes in 42 blocks
==24969== suppressed: 0 bytes in 0 blocks
==24969== Reachable blocks (those to which a pointer was found) are not
shown.
==24969== To see them, rerun with: --leak-check=full --show-leak-kinds=all
==24969==
==24969== For counts of detected and suppressed errors, rerun with: -v
==24969== ERROR SUMMARY: 0 errors from 0 contexts (suppressed: 0 from 0)
Parsing config from /etc/xen/auto/asterisk_deb80.cfg
Waiting for domain asterisk_deb80 (domid 31) to die [pid 24967]
Domain 31 has shut down, reason code 1 0x1
Action for shutdown reason code 1 is restart
Domain 31 needs to be cleaned up: destroying the domain
Done. Rebooting now
Waiting for domain asterisk_deb80 (domid 32) to die [pid 24967]
Domain 32 has shut down, reason code 1 0x1
Action for shutdown reason code 1 is restart
Domain 32 needs to be cleaned up: destroying the domain
Done. Rebooting now
Waiting for domain asterisk_deb80 (domid 33) to die [pid 24967]
Domain 33 has shut down, reason code 1 0x1
Action for shutdown reason code 1 is restart
Domain 33 needs to be cleaned up: destroying the domain
Done. Rebooting now
Waiting for domain asterisk_deb80 (domid 34) to die [pid 24967]
Domain 34 has shut down, reason code 0 0x0
Action for shutdown reason code 0 is destroy
Domain 34 needs to be cleaned up: destroying the domain
Done. Exiting now
>
>> Signed-off-by: Ian Campbell <ian.campbell@citrix.com>
>> ---
>> v2: Correct handling in xc_try_lzo1x_decode.
>>
>> This is a bug fix and should go into 4.5.
>>
>> I have a sneaking suspicion this is going to need to backport a very long way...
>> ---
>> tools/libxc/include/xc_dom.h | 10 ++++--
>> tools/libxc/xc_dom_bzimageloader.c | 20 ++++++++++++
>> tools/libxc/xc_dom_core.c | 61 +++++++++++++++++++++++++++--------
>> tools/libxc/xc_dom_decompress_lz4.c | 5 +++
>> 4 files changed, 80 insertions(+), 16 deletions(-)
>>
>> diff --git a/tools/libxc/include/xc_dom.h b/tools/libxc/include/xc_dom.h
>> index 6ae6a9f..07d7224 100644
>> --- a/tools/libxc/include/xc_dom.h
>> +++ b/tools/libxc/include/xc_dom.h
>> @@ -33,8 +33,13 @@ struct xc_dom_seg {
>>
>> struct xc_dom_mem {
>> struct xc_dom_mem *next;
>> - void *mmap_ptr;
>> - size_t mmap_len;
>> + void *ptr;
>> + enum {
>> + XC_DOM_MEM_TYPE_MALLOC_INTERNAL,
>> + XC_DOM_MEM_TYPE_MALLOC_EXTERNAL,
>> + XC_DOM_MEM_TYPE_MMAP,
>> + } type;
>> + size_t len;
>> unsigned char memory[0];
>> };
>>
>> @@ -298,6 +303,7 @@ void xc_dom_log_memory_footprint(struct xc_dom_image *dom);
>> /* --- simple memory pool ------------------------------------------ */
>>
>> void *xc_dom_malloc(struct xc_dom_image *dom, size_t size);
>> +int xc_dom_register_external(struct xc_dom_image *dom, void *ptr, size_t size);
>> void *xc_dom_malloc_page_aligned(struct xc_dom_image *dom, size_t size);
>> void *xc_dom_malloc_filemap(struct xc_dom_image *dom,
>> const char *filename, size_t * size,
>> diff --git a/tools/libxc/xc_dom_bzimageloader.c b/tools/libxc/xc_dom_bzimageloader.c
>> index 2225699..964ebdc 100644
>> --- a/tools/libxc/xc_dom_bzimageloader.c
>> +++ b/tools/libxc/xc_dom_bzimageloader.c
>> @@ -161,6 +161,13 @@ static int xc_try_bzip2_decode(
>>
>> total = (((uint64_t)stream.total_out_hi32) << 32) | stream.total_out_lo32;
>>
>> + if ( xc_dom_register_external(dom, out_buf, total) )
>> + {
>> + DOMPRINTF("BZIP2: Error registering stream output");
>> + free(out_buf);
>> + goto bzip2_cleanup;
>> + }
>> +
>> DOMPRINTF("%s: BZIP2 decompress OK, 0x%zx -> 0x%lx",
>> __FUNCTION__, *size, (long unsigned int) total);
>>
>> @@ -305,6 +312,13 @@ static int _xc_try_lzma_decode(
>> }
>> }
>>
>> + if ( xc_dom_register_external(dom, out_buf, stream->total_out) )
>> + {
>> + DOMPRINTF("%s: Error registering stream output", what);
>> + free(out_buf);
>> + goto lzma_cleanup;
>> + }
>> +
>> DOMPRINTF("%s: %s decompress OK, 0x%zx -> 0x%zx",
>> __FUNCTION__, what, *size, (size_t)stream->total_out);
>>
>> @@ -464,7 +478,13 @@ static int xc_try_lzo1x_decode(
>>
>> dst_len = lzo_read_32(cur);
>> if ( !dst_len )
>> + {
>> + msg = "Error registering stream output";
>> + if ( xc_dom_register_external(dom, out_buf, out_len) )
>> + break;
>> +
>> return 0;
>> + }
>>
>> if ( dst_len > LZOP_MAX_BLOCK_SIZE )
>> {
>> diff --git a/tools/libxc/xc_dom_core.c b/tools/libxc/xc_dom_core.c
>> index baa62a1..ecbf981 100644
>> --- a/tools/libxc/xc_dom_core.c
>> +++ b/tools/libxc/xc_dom_core.c
>> @@ -132,6 +132,7 @@ void *xc_dom_malloc(struct xc_dom_image *dom, size_t size)
>> return NULL;
>> }
>> memset(block, 0, sizeof(*block) + size);
>> + block->type = XC_DOM_MEM_TYPE_MALLOC_INTERNAL;
>> block->next = dom->memblocks;
>> dom->memblocks = block;
>> dom->alloc_malloc += sizeof(*block) + size;
>> @@ -151,23 +152,45 @@ void *xc_dom_malloc_page_aligned(struct xc_dom_image *dom, size_t size)
>> return NULL;
>> }
>> memset(block, 0, sizeof(*block));
>> - block->mmap_len = size;
>> - block->mmap_ptr = mmap(NULL, block->mmap_len,
>> - PROT_READ | PROT_WRITE, MAP_PRIVATE | MAP_ANON,
>> - -1, 0);
>> - if ( block->mmap_ptr == MAP_FAILED )
>> + block->len = size;
>> + block->ptr = mmap(NULL, block->len,
>> + PROT_READ | PROT_WRITE, MAP_PRIVATE | MAP_ANON,
>> + -1, 0);
>> + if ( block->ptr == MAP_FAILED )
>> {
>> DOMPRINTF("%s: mmap failed", __FUNCTION__);
>> free(block);
>> return NULL;
>> }
>> + block->type = XC_DOM_MEM_TYPE_MMAP;
>> block->next = dom->memblocks;
>> dom->memblocks = block;
>> dom->alloc_malloc += sizeof(*block);
>> - dom->alloc_mem_map += block->mmap_len;
>> + dom->alloc_mem_map += block->len;
>> if ( size > (100 * 1024) )
>> print_mem(dom, __FUNCTION__, size);
>> - return block->mmap_ptr;
>> + return block->ptr;
>> +}
>> +
>> +int xc_dom_register_external(struct xc_dom_image *dom, void *ptr, size_t size)
>> +{
>> + struct xc_dom_mem *block;
>> +
>> + block = malloc(sizeof(*block));
>> + if ( block == NULL )
>> + {
>> + DOMPRINTF("%s: allocation failed", __FUNCTION__);
>> + return -1;
>> + }
>> + memset(block, 0, sizeof(*block));
>> + block->ptr = ptr;
>> + block->len = size;
>> + block->type = XC_DOM_MEM_TYPE_MALLOC_EXTERNAL;
>> + block->next = dom->memblocks;
>> + dom->memblocks = block;
>> + dom->alloc_malloc += sizeof(*block);
>> + dom->alloc_mem_map += block->len;
>> + return 0;
>> }
>>
>> void *xc_dom_malloc_filemap(struct xc_dom_image *dom,
>> @@ -212,24 +235,25 @@ void *xc_dom_malloc_filemap(struct xc_dom_image *dom,
>> }
>>
>> memset(block, 0, sizeof(*block));
>> - block->mmap_len = *size;
>> - block->mmap_ptr = mmap(NULL, block->mmap_len, PROT_READ,
>> + block->len = *size;
>> + block->ptr = mmap(NULL, block->len, PROT_READ,
>> MAP_SHARED, fd, 0);
>> - if ( block->mmap_ptr == MAP_FAILED ) {
>> + if ( block->ptr == MAP_FAILED ) {
>> xc_dom_panic(dom->xch, XC_INTERNAL_ERROR,
>> "failed to mmap file: %s",
>> strerror(errno));
>> goto err;
>> }
>>
>> + block->type = XC_DOM_MEM_TYPE_MMAP;
>> block->next = dom->memblocks;
>> dom->memblocks = block;
>> dom->alloc_malloc += sizeof(*block);
>> - dom->alloc_file_map += block->mmap_len;
>> + dom->alloc_file_map += block->len;
>> close(fd);
>> if ( *size > (100 * 1024) )
>> print_mem(dom, __FUNCTION__, *size);
>> - return block->mmap_ptr;
>> + return block->ptr;
>>
>> err:
>> if ( fd != -1 )
>> @@ -246,8 +270,17 @@ static void xc_dom_free_all(struct xc_dom_image *dom)
>> while ( (block = dom->memblocks) != NULL )
>> {
>> dom->memblocks = block->next;
>> - if ( block->mmap_ptr )
>> - munmap(block->mmap_ptr, block->mmap_len);
>> + switch ( block->type )
>> + {
>> + case XC_DOM_MEM_TYPE_MALLOC_INTERNAL:
>> + break;
>> + case XC_DOM_MEM_TYPE_MALLOC_EXTERNAL:
>> + free(block->ptr);
>> + break;
>> + case XC_DOM_MEM_TYPE_MMAP:
>> + munmap(block->ptr, block->len);
>> + break;
>> + }
>> free(block);
>> }
>> }
>> diff --git a/tools/libxc/xc_dom_decompress_lz4.c b/tools/libxc/xc_dom_decompress_lz4.c
>> index 490ec56..b6a33f2 100644
>> --- a/tools/libxc/xc_dom_decompress_lz4.c
>> +++ b/tools/libxc/xc_dom_decompress_lz4.c
>> @@ -103,6 +103,11 @@ int xc_try_lz4_decode(
>>
>> if (size == 0)
>> {
>> + if ( xc_dom_register_external(dom, output, out_len) )
>> + {
>> + msg = "Error registering stream output";
>> + goto exit_2;
>> + }
>> *blob = output;
>> *psize = out_len;
>> return 0;
>> --
>> 1.7.10.4
>>
Information forwarded
to debian-bugs-dist@lists.debian.org, Debian Xen Team <pkg-xen-devel@lists.alioth.debian.org>:
Bug#767295; Package xen-utils-4.4.
(Fri, 21 Nov 2014 03:21:05 GMT) (full text, mbox, link).
Acknowledgement sent
to Gedalya <gedalya@gedalya.net>:
Extra info received and forwarded to list. Copy sent to Debian Xen Team <pkg-xen-devel@lists.alioth.debian.org>.
(Fri, 21 Nov 2014 03:21:05 GMT) (full text, mbox, link).
Message #60 received at 767295@bugs.debian.org (full text, mbox, reply):
[Message part 1 (text/plain, inline)]
On 11/20/2014 10:13 PM, Gedalya wrote: > On 11/20/2014 03:21 PM, Konrad Rzeszutek Wilk wrote: >> On Thu, Nov 20, 2014 at 03:48:47PM +0000, Ian Campbell wrote: >>> The libxc xc_dom_* infrastructure uses a very simple malloc memory >>> pool which >>> is freed by xc_dom_release. However the various xc_try_*_decode >>> routines (other >>> than the gzip one) just use plain malloc/realloc and therefore the >>> buffer ends >>> up leaked. >>> >>> The memory pool currently supports mmap'd buffers as well as a directly >>> allocated buffers, however the try decode routines make use of >>> realloc and do >>> not fit well into this model. Introduce a concept of an external >>> memory block >>> to the memory pool and provide an interface to register such memory. >>> >>> The mmap_ptr and mmap_len fields of the memblock tracking struct >>> lose their >>> mmap_ prefix since they are now also used for external memory blocks. >>> >>> We are only seeing this now because the gzip decoder doesn't leak >>> and it's only >>> relatively recently that kernels in the wild have switched to better >>> compression. >>> >>> This is https://bugs.debian.org/767295 >>> >>> Reported by: Gedalya <gedalya@gedalya.net> >> Gedelya, >> >> Could you also test this patch to make sure it does fix the >> reported issue please? > > So here's what happens now. > 1. Starts up tiny > 2. reboot: leak > 3. reboot: freed (process larger, but the delta is all/mostly shared > pages) > 4. reboot: leak > 5. reboot: freed > etc.. > For the record, I applied it again the xen package currently in debian, 4.4.1-3, see attached patch. The only problem was that tools/libxc/include/xc_dom.h is in tools/libxc/xc_dom.h, otherwise it applied fine.
[0039-libxc-dont-leak-buffer-containing-the-uncompressed-pv-kernel (text/plain, attachment)]
Information forwarded
to debian-bugs-dist@lists.debian.org, Debian Xen Team <pkg-xen-devel@lists.alioth.debian.org>:
Bug#767295; Package xen-utils-4.4.
(Fri, 21 Nov 2014 11:15:05 GMT) (full text, mbox, link).
Acknowledgement sent
to Ian Campbell <Ian.Campbell@citrix.com>:
Extra info received and forwarded to list. Copy sent to Debian Xen Team <pkg-xen-devel@lists.alioth.debian.org>.
(Fri, 21 Nov 2014 11:15:05 GMT) (full text, mbox, link).
Message #65 received at 767295@bugs.debian.org (full text, mbox, reply):
On Fri, 2014-11-21 at 11:03 +0000, Ian Campbell wrote: > http://man7.org/linux/man-pages/man3/mallopt.3.html also talks about > various dynamic thresholds for growing and shrinking the heap. My guess > is that we are bouncing up and down over some threshold with every other > reboot. IOW I'm not overly concerned with this apparent bi-modality, so long as the amount isn't increasing in the long term... I think the original patch should go in. Ian.
Information forwarded
to debian-bugs-dist@lists.debian.org, Debian Xen Team <pkg-xen-devel@lists.alioth.debian.org>:
Bug#767295; Package xen-utils-4.4.
(Fri, 21 Nov 2014 11:15:08 GMT) (full text, mbox, link).
Acknowledgement sent
to Ian Campbell <Ian.Campbell@citrix.com>:
Extra info received and forwarded to list. Copy sent to Debian Xen Team <pkg-xen-devel@lists.alioth.debian.org>.
(Fri, 21 Nov 2014 11:15:08 GMT) (full text, mbox, link).
Message #70 received at 767295@bugs.debian.org (full text, mbox, reply):
On Thu, 2014-11-20 at 22:13 -0500, Gedalya wrote: > On 11/20/2014 03:21 PM, Konrad Rzeszutek Wilk wrote: > > On Thu, Nov 20, 2014 at 03:48:47PM +0000, Ian Campbell wrote: > >> The libxc xc_dom_* infrastructure uses a very simple malloc memory pool which > >> is freed by xc_dom_release. However the various xc_try_*_decode routines (other > >> than the gzip one) just use plain malloc/realloc and therefore the buffer ends > >> up leaked. > >> > >> The memory pool currently supports mmap'd buffers as well as a directly > >> allocated buffers, however the try decode routines make use of realloc and do > >> not fit well into this model. Introduce a concept of an external memory block > >> to the memory pool and provide an interface to register such memory. > >> > >> The mmap_ptr and mmap_len fields of the memblock tracking struct lose their > >> mmap_ prefix since they are now also used for external memory blocks. > >> > >> We are only seeing this now because the gzip decoder doesn't leak and it's only > >> relatively recently that kernels in the wild have switched to better > >> compression. > >> > >> This is https://bugs.debian.org/767295 > >> > >> Reported by: Gedalya <gedalya@gedalya.net> > > Gedelya, > > > > Could you also test this patch to make sure it does fix the > > reported issue please? > > So here's what happens now. > 1. Starts up tiny > 2. reboot: leak > 3. reboot: freed (process larger, but the delta is all/mostly shared pages) > 4. reboot: leak > 5. reboot: freed > etc.. WTF, how very strange! > root@xen:~/xen-pkgs# xl cr /etc/xen/auto/asterisk_deb80.cfg > Parsing config from /etc/xen/auto/asterisk_deb80.cfg > root@xen:~/xen-pkgs# ps aux | grep asterisk_deb80 > root 22981 0.0 0.0 95968 588 ? SLsl 21:55 0:00 /usr/lib/xen-4.4/bin/xl cr /etc/xen/auto/asterisk_deb80.cfg > root@xen:~/xen-pkgs# pmap -x 22981 > 22981: /usr/lib/xen-4.4/bin/xl cr /etc/xen/auto/asterisk_deb80.cfg > Address Kbytes RSS Dirty Mode Mapping > 0000000000400000 144 128 0 r-x-- xl > 0000000000623000 4 4 4 r---- xl > 0000000000624000 8 8 8 rw--- xl > 0000000000626000 4 4 4 rw--- [ anon ] > 00000000009a6000 288 240 240 rw--- [ anon ] > 00007f14d4000000 132 8 8 rw--- [ anon ] > 00007f14d4021000 65404 0 0 ----- [ anon ] > << snip >> > ---------------- ------- ------- ------- > total kB 95968 2728 596 > > --- reboot domu --- > > root@xen:~/xen-pkgs# ps aux | grep asterisk_deb80 > root 22981 0.6 3.3 131652 20008 ? SLsl 21:55 0:00 /usr/lib/xen-4.4/bin/xl cr /etc/xen/auto/asterisk_deb80.cfg > root@xen:~/xen-pkgs# pmap -x 22981 > 22981: /usr/lib/xen-4.4/bin/xl cr /etc/xen/auto/asterisk_deb80.cfg > Address Kbytes RSS Dirty Mode Mapping > 0000000000400000 144 144 0 r-x-- xl > 0000000000623000 4 4 4 r---- xl > 0000000000624000 8 8 8 rw--- xl > 0000000000626000 4 4 4 rw--- [ anon ] > 00000000009a6000 288 288 288 rw--- [ anon ] > 00000000009ee000 35676 16772 16772 rw--- [ anon ] This is the (temporarily) leaked mapping, right? > Tried valgrind, it doesn't look like it was able to see what was going on Indeed. The values for total heap usage at exist and still reachable etc also don't seem to account for the ~3M of mapping on each iteration. I don't know how glibc's allocator works, but I suppose it isn't impossible that it is retaining some mappings of free regions and collecting them to free later somehow, which just happens to only trigger every other reboot (e.g. perhaps it is based on some threshold of free memory). ...investigates... So, http://man7.org/linux/man-pages/man3/malloc.3.html talks about special behaviour using mmap for allocations above MMAP_THRESHOLD (128K by default), which we will be hitting here I think. That explains the anon mapping. http://man7.org/linux/man-pages/man3/mallopt.3.html also talks about various dynamic thresholds for growing and shrinking the heap. My guess is that we are bouncing up and down over some threshold with every other reboot. Ian.
Information forwarded
to debian-bugs-dist@lists.debian.org, Debian Xen Team <pkg-xen-devel@lists.alioth.debian.org>:
Bug#767295; Package xen-utils-4.4.
(Fri, 21 Nov 2014 20:21:09 GMT) (full text, mbox, link).
Acknowledgement sent
to Gedalya <gedalya@gedalya.net>:
Extra info received and forwarded to list. Copy sent to Debian Xen Team <pkg-xen-devel@lists.alioth.debian.org>.
(Fri, 21 Nov 2014 20:21:09 GMT) (full text, mbox, link).
Message #75 received at 767295@bugs.debian.org (full text, mbox, reply):
On 11/21/2014 06:03 AM, Ian Campbell wrote: > >> So here's what happens now. >> 1. Starts up tiny >> 2. reboot: leak >> 3. reboot: freed (process larger, but the delta is all/mostly shared pages) >> 4. reboot: leak >> 5. reboot: freed >> etc.. > WTF, how very strange! :-) > >> --- reboot domu --- >> >> root@xen:~/xen-pkgs# ps aux | grep asterisk_deb80 >> root 22981 0.6 3.3 131652 20008 ? SLsl 21:55 0:00 /usr/lib/xen-4.4/bin/xl cr /etc/xen/auto/asterisk_deb80.cfg >> root@xen:~/xen-pkgs# pmap -x 22981 >> 22981: /usr/lib/xen-4.4/bin/xl cr /etc/xen/auto/asterisk_deb80.cfg >> Address Kbytes RSS Dirty Mode Mapping >> 0000000000400000 144 144 0 r-x-- xl >> 0000000000623000 4 4 4 r---- xl >> 0000000000624000 8 8 8 rw--- xl >> 0000000000626000 4 4 4 rw--- [ anon ] >> 00000000009a6000 288 288 288 rw--- [ anon ] >> 00000000009ee000 35676 16772 16772 rw--- [ anon ] > This is the (temporarily) leaked mapping, right? Yea that's the one that popped in after the reboot.. About 16 MB. > >> Tried valgrind, it doesn't look like it was able to see what was going on > Indeed. The values for total heap usage at exist and still reachable etc > also don't seem to account for the ~3M of mapping on each iteration. > > I don't know how glibc's allocator works, but I suppose it isn't > impossible that it is retaining some mappings of free regions and > collecting them to free later somehow, which just happens to only > trigger every other reboot (e.g. perhaps it is based on some threshold > of free memory). > > ...investigates... > > So, http://man7.org/linux/man-pages/man3/malloc.3.html talks about > special behaviour using mmap for allocations above MMAP_THRESHOLD (128K > by default), which we will be hitting here I think. That explains the > anon mapping. > > http://man7.org/linux/man-pages/man3/mallopt.3.html also talks about > various dynamic thresholds for growing and shrinking the heap. My guess > is that we are bouncing up and down over some threshold with every other > reboot. > > Ian. OK this is way over my head.. I don't have a full and precise understanding of all of the above, but let me try to comment nevertheless. There are two issues here. The added mappings (as I understand, files, non-anon, shared) do happen only on reboot, but they're not a real memory leak issue because they are shared with other processes, so no matter ho many xl processes we have, it's only another 2.6 or so MB added to the total memory usage of the server, right? On the other hand we have this anon area, 16 MB, that pops in on one reboot and gets freed on the next. That's the real issue, as I see it..! And all that only starts happening after the last line of valgrind output. Valgrind only had output up to the first boot of the VM, none later. For a fresh, non-rebooted domu, the xl process shows up as in top as ~588 KB RES, 0 SHR, how can the latter be anyway?? I don't understand.
Information forwarded
to debian-bugs-dist@lists.debian.org, Debian Xen Team <pkg-xen-devel@lists.alioth.debian.org>:
Bug#767295; Package xen-utils-4.4.
(Fri, 21 Nov 2014 20:27:04 GMT) (full text, mbox, link).
Acknowledgement sent
to Gedalya <gedalya@gedalya.net>:
Extra info received and forwarded to list. Copy sent to Debian Xen Team <pkg-xen-devel@lists.alioth.debian.org>.
(Fri, 21 Nov 2014 20:27:04 GMT) (full text, mbox, link).
Message #80 received at 767295@bugs.debian.org (full text, mbox, reply):
On 11/21/2014 06:12 AM, Ian Campbell wrote: > On Fri, 2014-11-21 at 11:03 +0000, Ian Campbell wrote: >> http://man7.org/linux/man-pages/man3/mallopt.3.html also talks about >> various dynamic thresholds for growing and shrinking the heap. My guess >> is that we are bouncing up and down over some threshold with every other >> reboot. > IOW I'm not overly concerned with this apparent bi-modality, so long as > the amount isn't increasing in the long term... > > I think the original patch should go in. > > Ian. > > It's an improvement, but consider this: Someone has a xen host running wheezy, 40 domu's, with 768MB for dom0, worked fine so far. Tries upgrading to jessie, and lo, each domu process takes up only 588 KB on dom0, great! Then a new kernel package is released, all domu's get rebooted once. All host memory is now full. Dude might have had other plans for that memory... This is dead memory so I guess it can be swapped out, not easily a scenario where the server totally crashes, but it's a bit ugly, we're talking about memory usage leaping from 0.6 to 16 MB per domu.
Information forwarded
to debian-bugs-dist@lists.debian.org, Debian Xen Team <pkg-xen-devel@lists.alioth.debian.org>:
Bug#767295; Package xen-utils-4.4.
(Mon, 24 Nov 2014 10:45:10 GMT) (full text, mbox, link).
Acknowledgement sent
to Ian Campbell <Ian.Campbell@citrix.com>:
Extra info received and forwarded to list. Copy sent to Debian Xen Team <pkg-xen-devel@lists.alioth.debian.org>.
(Mon, 24 Nov 2014 10:45:10 GMT) (full text, mbox, link).
Message #85 received at 767295@bugs.debian.org (full text, mbox, reply):
On Fri, 2014-11-21 at 15:25 -0500, Gedalya wrote:
> On 11/21/2014 06:12 AM, Ian Campbell wrote:
> > On Fri, 2014-11-21 at 11:03 +0000, Ian Campbell wrote:
> >> http://man7.org/linux/man-pages/man3/mallopt.3.html also talks about
> >> various dynamic thresholds for growing and shrinking the heap. My guess
> >> is that we are bouncing up and down over some threshold with every other
> >> reboot.
> > IOW I'm not overly concerned with this apparent bi-modality, so long as
> > the amount isn't increasing in the long term...
> >
> > I think the original patch should go in.
> >
> > Ian.
> >
> >
> It's an improvement, but consider this:
> Someone has a xen host running wheezy, 40 domu's, with 768MB for dom0,
> worked fine so far. Tries upgrading to jessie, and lo, each domu process
> takes up only 588 KB on dom0, great!
> Then a new kernel package is released, all domu's get rebooted once. All
> host memory is now full. Dude might have had other plans for that
> memory... This is dead memory so I guess it can be swapped out, not
> easily a scenario where the server totally crashes, but it's a bit ugly,
> we're talking about memory usage leaping from 0.6 to 16 MB per domu.
Unfortunately this is down to the behaviour of the libc and not
something which appears to be under application control.
The following program demonstrates the same behaviour and is certainly
not leaking anything. Notice that at "Freed block at XXXXX. Everything
is now freed, end of day" there is still an anon mapping of that
address. Notice also that the "in use" figures are zero.
If this concerns you then you should probably take a look at mallopt(3)
and/or be talking to the libc folks about it. It's not an xl issue
AFAICT.
Ian.
#include <sys/types.h>
#include <stdio.h>
#include <stdlib.h>
#include <unistd.h>
#include <malloc.h>
#define KB 196
int main(int argc, char **argv)
{
void *p;
char buf[1000];
snprintf(buf, 1000, "pmap -x %d", getpid());
printf("Start of day\n");
system(buf);
malloc_stats();
printf("\n=========================\n\n");
p = malloc(KB*0x1000);
printf("allocated %dKB block at %p\n", KB, p);
system(buf);
malloc_stats();
printf("\n=========================\n\n");
free(p);
printf("Freed block at %p\n", p);
system(buf);
malloc_stats();
printf("\n=========================\n\n");
p = malloc(KB*0x1000);
printf("Allocated another %dKB block at %p\n", KB, p);
system(buf);
malloc_stats();
printf("\n=========================\n\n");
free(p);
printf("Freed block at %p. Everything is now freed, end of day\n", p);
system(buf);
malloc_stats();
printf("\n=========================\n\n");
return 0;
}
Information forwarded
to debian-bugs-dist@lists.debian.org, Debian Xen Team <pkg-xen-devel@lists.alioth.debian.org>:
Bug#767295; Package xen-utils-4.4.
(Tue, 25 Nov 2014 07:18:04 GMT) (full text, mbox, link).
Acknowledgement sent
to Gedalya <gedalya@gedalya.net>:
Extra info received and forwarded to list. Copy sent to Debian Xen Team <pkg-xen-devel@lists.alioth.debian.org>.
(Tue, 25 Nov 2014 07:18:05 GMT) (full text, mbox, link).
Message #90 received at 767295@bugs.debian.org (full text, mbox, reply):
On 11/24/2014 05:37 AM, Ian Campbell wrote: > Unfortunately this is down to the behaviour of the libc and not > something which appears to be under application control. > > The following program demonstrates the same behaviour and is certainly > not leaking anything. Notice that at "Freed block at XXXXX. Everything > is now freed, end of day" there is still an anon mapping of that > address. Notice also that the "in use" figures are zero. > > If this concerns you then you should probably take a look at mallopt(3) > and/or be talking to the libc folks about it. It's not an xl issue > AFAICT. Firstly, thank you very much for explaining this in such clear detail, above all this has been quite educational for me :-) After reading the man page, it looks like glibc's behavior here is indeed by design. I'm unable to form, much less advocate an opinion about how libc should behave, any discussion about libc must be very broad. Stepping away from the technical details, I still think that any future enhancement to make xl go out of its way to free this memory would definitely be nice. Right now we have memory that is allocated for a single, momentary use and it can stay allocated for the lifetime of a domu unnecessarily, taking the size of the xl process to another order of magnitude. Even if no longer technically a bug / memory leak, and the implied priority, this could still merit someone's attention.
Severity set to 'important' from 'normal'
Request was from Bastian Blank <bastian.blank@credativ.de>
to control@bugs.debian.org.
(Wed, 26 Nov 2014 07:51:09 GMT) (full text, mbox, link).
Reply sent
to Bastian Blank <waldi@debian.org>:
You have taken responsibility.
(Thu, 27 Nov 2014 21:24:43 GMT) (full text, mbox, link).
Notification sent
to Gedalya <gedalya@gedalya.net>:
Bug acknowledged by developer.
(Thu, 27 Nov 2014 21:24:43 GMT) (full text, mbox, link).
Message #97 received at 767295-close@bugs.debian.org (full text, mbox, reply):
Source: xen
Source-Version: 4.4.1-4
We believe that the bug you reported is fixed in the latest version of
xen, which is due to be installed in the Debian FTP archive.
A summary of the changes between this version and the previous one is
attached.
Thank you for reporting the bug, which will now be closed. If you
have further comments please address them to 767295@bugs.debian.org,
and the maintainer will reopen the bug report if appropriate.
Debian distribution maintenance software
pp.
Bastian Blank <waldi@debian.org> (supplier of updated xen package)
(This message was generated automatically at their request; if you
believe that there is a problem with it please contact the archive
administrators by mailing ftpmaster@ftp-master.debian.org)
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA512
Format: 1.8
Date: Thu, 27 Nov 2014 20:17:36 +0100
Source: xen
Binary: libxen-4.4 libxenstore3.0 libxen-dev xenstore-utils xen-utils-common xen-utils-4.4 xen-hypervisor-4.4-amd64 xen-system-amd64 xen-hypervisor-4.4-arm64 xen-system-arm64 xen-hypervisor-4.4-armhf xen-system-armhf
Architecture: source all amd64
Version: 4.4.1-4
Distribution: unstable
Urgency: medium
Maintainer: Debian Xen Team <pkg-xen-devel@lists.alioth.debian.org>
Changed-By: Bastian Blank <waldi@debian.org>
Description:
libxen-4.4 - Public libs for Xen
libxen-dev - Public headers and libs for Xen
libxenstore3.0 - Xenstore communications library for Xen
xen-hypervisor-4.4-amd64 - Xen Hypervisor on AMD64
xen-hypervisor-4.4-arm64 - Xen Hypervisor on ARM64
xen-hypervisor-4.4-armhf - Xen Hypervisor on ARMHF
xen-system-amd64 - Xen System on AMD64 (meta-package)
xen-system-arm64 - Xen System on ARM64 (meta-package)
xen-system-armhf - Xen System on ARMHF (meta-package)
xen-utils-4.4 - XEN administrative tools
xen-utils-common - Xen administrative tools - common files
xenstore-utils - Xenstore command line utilities for Xen
Closes: 759384 767295 769543 770460
Changes:
xen (4.4.1-4) unstable; urgency=medium
.
[ Bastian Blank ]
* Make operations pre-emptible.
CVE-2014-5146, CVE-2014-5149
* Don't allow page table updates from non-PV page tables.
CVE-2014-8594
* Enforce privilege level while loading code segment.
CVE-2014-8595
* Fix reference counter leak.
CVE-2014-9030
* Use linux 3.16.0-4 stuff.
* Fix memory leak in xl. (closes: #767295)
.
[ Ian Campbell ]
* Add licensing for tools/python/logging to debian/copyright.
(Closes: #759384)
* Correctly include xen-init-name in xen-utils-common. (Closes: #769543)
* xen-utils recommends grub-xen-host package (Closes: #770460)
Checksums-Sha1:
23075ae54e1a8ea49864c871be036f7c9fc88e12 2600 xen_4.4.1-4.dsc
598a7e9f0c4279bad0765e5964f13f1c266a30aa 59772 xen_4.4.1-4.debian.tar.xz
7fdd633935a4ad26294929e345f74c69f63a2336 120666 xen-utils-common_4.4.1-4_all.deb
902851a1eec687232b068effc4770db8cf95e586 1671010 xen-hypervisor-4.4-amd64_4.4.1-4_amd64.deb
0f5016527f38126a6687f755c9fcda904148ffc7 19586 xen-system-amd64_4.4.1-4_amd64.deb
add47ed1b2f26940e181b347d79e6424723f63c0 30458 libxenstore3.0_4.4.1-4_amd64.deb
a4a19ae136ffddd6b1cf0124102abe1e3d287650 295062 libxen-4.4_4.4.1-4_amd64.deb
bb97828b8e38b608f14fa0565cb1d570fe1f190e 476442 libxen-dev_4.4.1-4_amd64.deb
b107f1d59c8d4b76856a21840c9d8f4867b37537 393300 xen-utils-4.4_4.4.1-4_amd64.deb
a91952017a6a419e53e62c39e91914aa08da0abd 26120 xenstore-utils_4.4.1-4_amd64.deb
Checksums-Sha256:
28da1bed741e73d0646e90eed19fe189599d1cf1e293d6e5c2c5b42947514596 2600 xen_4.4.1-4.dsc
8b81cde7cd1e0066ee0d2b4225dbc997e42991f6242b4d37754faa04d14d3333 59772 xen_4.4.1-4.debian.tar.xz
e47df6cc38c8c7368478022b0c562b41f5a7e12cb90511a8069b7f4d1f403acf 120666 xen-utils-common_4.4.1-4_all.deb
9b34a6bc3195559ff27c4d8430cd1961cd0717a50070f210ccfe93f2b3f28e43 1671010 xen-hypervisor-4.4-amd64_4.4.1-4_amd64.deb
9e27b24f7858f1c4e2d8ebd4b5f6026cfa1edfec64523d9d58b806398d7d857e 19586 xen-system-amd64_4.4.1-4_amd64.deb
24735aaf1754f5adfe24d282e913ece79c12cdbfbb38a77bbc4417da5597d133 30458 libxenstore3.0_4.4.1-4_amd64.deb
c5ab637175197592c8f46490ec6629850e818e597193e1867dbaeb28fdb7e858 295062 libxen-4.4_4.4.1-4_amd64.deb
26463a88f4e2543d7980777cbb50f52dd599c082111fb976594f47b5dd8aa065 476442 libxen-dev_4.4.1-4_amd64.deb
74e1c08489418db06d736df02cb24d71bbe254627b0d7a91a42e121b778ac250 393300 xen-utils-4.4_4.4.1-4_amd64.deb
f4228ff85519ddf9b014349c12e24e950b9dc9a81fda8f57432317e9588f7064 26120 xenstore-utils_4.4.1-4_amd64.deb
Files:
509ecc20c9d5fb8eb14fabfcf2d9ac23 2600 kernel optional xen_4.4.1-4.dsc
aa51a2398499174dc462a7bfd639dedf 59772 kernel optional xen_4.4.1-4.debian.tar.xz
882642769cd3e6763cddf07980d4cecf 120666 kernel optional xen-utils-common_4.4.1-4_all.deb
ce3749fbc82c2537cad6fe440ed795cc 1671010 kernel optional xen-hypervisor-4.4-amd64_4.4.1-4_amd64.deb
774d0ea604b46ca6b9784d6639d7ef6f 19586 kernel optional xen-system-amd64_4.4.1-4_amd64.deb
f0d445b9bd059f3cfbc3d9325eb191c5 30458 libs optional libxenstore3.0_4.4.1-4_amd64.deb
f5a1a954eb030711ba25c82b78b3000f 295062 libs optional libxen-4.4_4.4.1-4_amd64.deb
c5ad87daae3a9c894c21d2046e07cc69 476442 libdevel optional libxen-dev_4.4.1-4_amd64.deb
249c3d7b27ed2601b429b3252d4dc564 393300 kernel optional xen-utils-4.4_4.4.1-4_amd64.deb
f9c006b04fb36b5ff1783b8af56b4246 26120 admin optional xenstore-utils_4.4.1-4_amd64.deb
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1
iQEcBAEBCgAGBQJUd4TMAAoJEG2TiIWKaf5RAXsH+wY8YlPwdnGwZKl+UBP6j6uG
w53j6d6SEv3lAOIGYsQwwtByBpFJzE/Gt7AzsSXrS7Lv9pcHnd7+wgIROeWRDJ9a
iT2ui/z9e2G3eYP1pj9CgyQzfXfl/WbV01V7YivzD3QHDCHSb03PX7IsOFtBsomE
+Ja6BbPSxGYG+kqG4og3bfMrllFf6tVTc3Nw4LUOgEkct1uhVK619xvzHJNPk4Yk
4Yj1gopSQjGzTpkA3htxa6IE3/IjvvQu9QBxcGwehSK4dEjJqTGgkw5ZwWEEDiSK
EPSg3MdZ5MK7CFuWe72oN4P24DNThC4ySbL5QuiUtBS6xBPicJL4dGy1Pe/ED6s=
=zRox
-----END PGP SIGNATURE-----
Information forwarded
to debian-bugs-dist@lists.debian.org, Debian Xen Team <pkg-xen-devel@lists.alioth.debian.org>:
Bug#767295; Package xen-utils-4.4.
(Fri, 28 Nov 2014 13:27:05 GMT) (full text, mbox, link).
Acknowledgement sent
to Ian Campbell <ijc@debian.org>:
Extra info received and forwarded to list. Copy sent to Debian Xen Team <pkg-xen-devel@lists.alioth.debian.org>.
(Fri, 28 Nov 2014 13:27:05 GMT) (full text, mbox, link).
Message #102 received at 767295@bugs.debian.org (full text, mbox, reply):
reopen 767295
thanks
The 4.4.1-4 release only included one related fix there are actually two
more:
commit 379b351889a8f02abe30a06e2ce9ba8b381b91ab
Author: Ian Campbell <ian.campbell@citrix.com>
Date: Thu Nov 6 13:00:31 2014 +0000
tools: libxl: do not leak diskpath during local disk attach
libxl__device_disk_local_initiate_attach is assigning dls->diskpath with a
strdup of the device path. This is then passed to the callback, e.g.
parse_bootloader_result but bootloader_cleanup will not free it.
Since the callback is within the scope of the (e)gc and therefore doesn't need
to be malloc'd, a gc'd alloc will do. All other assignments to this field use
the gc.
https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=767295
Reported-by: Gedalya <gedalya@gedalya.net>
Signed-off-by: Ian Campbell <ian.campbell@citrix.com>
Acked-by: Ian Jackson <ian.jackson@eu.citrix.com>
Acked-by: Wei Liu <wei.liu2@citrix.com>
commit 8f4023dd7d77de7b2c1af77e86637202a33f948a
Author: Ian Campbell <ian.campbell@citrix.com>
Date: Thu Nov 20 15:48:47 2014 +0000
libxc: don't leak buffer containing the uncompressed PV kernel
The libxc xc_dom_* infrastructure uses a very simple malloc memory pool which
is freed by xc_dom_release. However the various xc_try_*_decode routines (other
than the gzip one) just use plain malloc/realloc and therefore the buffer ends
up leaked.
The memory pool currently supports mmap'd buffers as well as a directly
allocated buffers, however the try decode routines make use of realloc and do
not fit well into this model. Introduce a concept of an external memory block
to the memory pool and provide an interface to register such memory.
The mmap_ptr and mmap_len fields of the memblock tracking struct lose their
mmap_ prefix since they are now also used for external memory blocks.
We are only seeing this now because the gzip decoder doesn't leak and it's only
relatively recently that kernels in the wild have switched to better
compression.
This is https://bugs.debian.org/767295
Reported by: Gedalya <gedalya@gedalya.net>
Signed-off-by: Ian Campbell <ian.campbell@citrix.com>
Reviewed-by: Wei Liu <wei.liu2@citrix.com>
Which the actual leaks discovered here (of which the second is the far
larger).
Bug reopened
Request was from Ian Campbell <ijc@debian.org>
to control@bugs.debian.org.
(Fri, 28 Nov 2014 13:27:11 GMT) (full text, mbox, link).
No longer marked as fixed in versions xen/4.4.1-4.
Request was from Ian Campbell <ijc@debian.org>
to control@bugs.debian.org.
(Fri, 28 Nov 2014 13:27:12 GMT) (full text, mbox, link).
Reply sent
to Bastian Blank <waldi@debian.org>:
You have taken responsibility.
(Sun, 30 Nov 2014 19:51:14 GMT) (full text, mbox, link).
Notification sent
to Gedalya <gedalya@gedalya.net>:
Bug acknowledged by developer.
(Sun, 30 Nov 2014 19:51:14 GMT) (full text, mbox, link).
Message #111 received at 767295-close@bugs.debian.org (full text, mbox, reply):
Source: xen
Source-Version: 4.4.1-5
We believe that the bug you reported is fixed in the latest version of
xen, which is due to be installed in the Debian FTP archive.
A summary of the changes between this version and the previous one is
attached.
Thank you for reporting the bug, which will now be closed. If you
have further comments please address them to 767295@bugs.debian.org,
and the maintainer will reopen the bug report if appropriate.
Debian distribution maintenance software
pp.
Bastian Blank <waldi@debian.org> (supplier of updated xen package)
(This message was generated automatically at their request; if you
believe that there is a problem with it please contact the archive
administrators by mailing ftpmaster@ftp-master.debian.org)
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1
Format: 1.8
Date: Sun, 30 Nov 2014 20:13:32 +0100
Source: xen
Binary: libxen-4.4 libxenstore3.0 libxen-dev xenstore-utils xen-utils-common xen-utils-4.4 xen-hypervisor-4.4-amd64 xen-system-amd64 xen-hypervisor-4.4-arm64 xen-system-arm64 xen-hypervisor-4.4-armhf xen-system-armhf
Architecture: source all amd64
Version: 4.4.1-5
Distribution: unstable
Urgency: medium
Maintainer: Debian Xen Team <pkg-xen-devel@lists.alioth.debian.org>
Changed-By: Bastian Blank <waldi@debian.org>
Description:
libxen-4.4 - Public libs for Xen
libxen-dev - Public headers and libs for Xen
libxenstore3.0 - Xenstore communications library for Xen
xen-hypervisor-4.4-amd64 - Xen Hypervisor on AMD64
xen-hypervisor-4.4-arm64 - Xen Hypervisor on ARM64
xen-hypervisor-4.4-armhf - Xen Hypervisor on ARMHF
xen-system-amd64 - Xen System on AMD64 (meta-package)
xen-system-arm64 - Xen System on ARM64 (meta-package)
xen-system-armhf - Xen System on ARMHF (meta-package)
xen-utils-4.4 - XEN administrative tools
xen-utils-common - Xen administrative tools - common files
xenstore-utils - Xenstore command line utilities for Xen
Closes: 767295
Changes:
xen (4.4.1-5) unstable; urgency=medium
.
* Fix excessive checks of hypercall arguments.
CVE-2014-8866
* Fix boundary checks of emulated MMIO access.
CVE-2014-8867
* Fix additional memory leaks in xl. (closes: #767295)
Checksums-Sha1:
224033f6dca82c3717349ad2b882d4d13ff0faa3 2598 xen_4.4.1-5.dsc
5cb42d61c090e17c8e1d07abb4eec370402e7abf 64792 xen_4.4.1-5.debian.tar.xz
310d9cec42512d3fc4fd8f68bb6e17adba3c7154 120772 xen-utils-common_4.4.1-5_all.deb
d50cd6debff16965ff9914cfee862f5eb1751c5a 1671186 xen-hypervisor-4.4-amd64_4.4.1-5_amd64.deb
9134dcbd64b9204a33ea09201221f6399a61d236 19660 xen-system-amd64_4.4.1-5_amd64.deb
c84c686aaa82a5f142db0fabb94b55cb37157a54 30572 libxenstore3.0_4.4.1-5_amd64.deb
a5850c6bb093e915664086a99a6555985a785f40 295086 libxen-4.4_4.4.1-5_amd64.deb
37e9aa25ad1f8f1398bf1fcc9210a063d1a4c93f 476346 libxen-dev_4.4.1-5_amd64.deb
3d215f6a52ef0db6afc4bfb6454ff41194f3cc4a 393280 xen-utils-4.4_4.4.1-5_amd64.deb
f8661ece2b83acfd1cac6d5a565022319b05e983 26198 xenstore-utils_4.4.1-5_amd64.deb
Checksums-Sha256:
fc66cdd908f3bb12e02327bd29a887b06d7b21ab38edacd3c9e712494c56996c 2598 xen_4.4.1-5.dsc
5f2d2137fa68ff63545a5e8a2e45d836fea481cb0f7a854bcff5f882cbeb881f 64792 xen_4.4.1-5.debian.tar.xz
1bfb6b49e713f2807fddf7dbb0c3c8ee556a963fc940bf25e22f75263aa4ab08 120772 xen-utils-common_4.4.1-5_all.deb
959dcce1cfc70d823a7f84adb1f1d2ccbbed042f5d2160d5e910bd7152008c54 1671186 xen-hypervisor-4.4-amd64_4.4.1-5_amd64.deb
5e414a9f08df0e701e73576020aedf3029490a20407f83c27a51432cf9b9b1bf 19660 xen-system-amd64_4.4.1-5_amd64.deb
93f593202d5d1b3e4ad059c5930b9cd94e4f4ff7724081d60a6f6c5277422cff 30572 libxenstore3.0_4.4.1-5_amd64.deb
b9340b5e6b9378b557402a2201f8e2068a78bcd948cfe2991bccc312d064bde1 295086 libxen-4.4_4.4.1-5_amd64.deb
c17513d46a9f8c32f6bfd0f83ff2e243142e93b10bf3a4d4d0706c284e3f48f0 476346 libxen-dev_4.4.1-5_amd64.deb
77bead8f2f6414068ada22218f306cb5b2d974dc82744043330bbfc0c6e696b2 393280 xen-utils-4.4_4.4.1-5_amd64.deb
098abd59aa380e18ec8ff1c28e8944cf1b2e340d3331986395390efad0639053 26198 xenstore-utils_4.4.1-5_amd64.deb
Files:
f1a2af9f9651dbeabff5d8fc062b6412 2598 kernel optional xen_4.4.1-5.dsc
34f0a6c8c8d8cff1903789ce68f30f51 64792 kernel optional xen_4.4.1-5.debian.tar.xz
0e8be9db756ed5614109fc1a23063506 120772 kernel optional xen-utils-common_4.4.1-5_all.deb
1804a09b1aec022711b0dadc1d95cacb 1671186 kernel optional xen-hypervisor-4.4-amd64_4.4.1-5_amd64.deb
a73268649f67307496ff5257ea4027f9 19660 kernel optional xen-system-amd64_4.4.1-5_amd64.deb
39a901402a540ddb1a02050abd46d90e 30572 libs optional libxenstore3.0_4.4.1-5_amd64.deb
3e5672de5fe4240e210f5e49310f7b8f 295086 libs optional libxen-4.4_4.4.1-5_amd64.deb
7a22ff82b1c665afc7a9bcced6babb2b 476346 libdevel optional libxen-dev_4.4.1-5_amd64.deb
5c15d6d7ec7d2d0fa9158465cf9e5b22 393280 kernel optional xen-utils-4.4_4.4.1-5_amd64.deb
4e80677ec103a2fccfbeca1c3dda847a 26198 admin optional xenstore-utils_4.4.1-5_amd64.deb
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1
iQEcBAEBAgAGBQJUe3GMAAoJEG2TiIWKaf5RjOsH/0NTgWrDPBhffzY0/0+pjphr
rjpsw/W1merwZgO5mx+BUxTLwmxSbVOkqBxU46QyxNlw/QoErolezKO06zJ/jepP
A4+2z8un6wyvMRKwWznsNVwkGYozwA14rpPH74O9h4aswrjZB4p/BeS2jkJKRH8b
jLoy0TdpLpSZdeRHVcEQKUakZYmST49MAwB8bvH/UKgd/PGwezfGEXhrrayvNSoM
b0hdChatSTufnjeX7qtOCR89Rt+vFwcO91upvjwih8a9STGsboGGhdLw3Jc9S6tj
8kwEY45k3J2+hnsFTDMqD1bV1O4Wdee+KqitbUtd67+Wx3azkl12tSoatdFWAdc=
=lBub
-----END PGP SIGNATURE-----
Bug archived.
Request was from Debbugs Internal Request <owner@bugs.debian.org>
to internal_control@bugs.debian.org.
(Sat, 03 Jan 2015 07:35:31 GMT) (full text, mbox, link).
Send a report that this bug log contains spam.
Debbugs is free software and licensed under the terms of the GNU Public License version 2. The current version can be obtained from https://bugs.debian.org/debbugs-source/.
Copyright © 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson, 2005-2017 Don Armstrong, and many other contributors.