Debian Bug report logs - #372618
mdadm --monitor consumes much memory and idle cpu

Toggle useless messages

Message #5 received at submit@bugs.debian.org (full text, mbox, reply):

From: Elimar Riesebieter <riesebie@lxtec.de>

To: Debian Bug Tracking System <submit@bugs.debian.org>

Subject: mdadm --monitor consumes much memory and idle cpu

Date: Sat, 10 Jun 2006 19:18:52 +0200

Package: mdadm
Version: 2.5-4
Severity: important

The memory and cpu consumption grows up to around 50% running 
/sbin/mdadm --monitor --pid-file /var/run/mdadm.pid --mail root
--daemonise --scan

2.4.1-6 doesn't.

Elimar


-- Package-specific info:
--- mount output
/dev/md6 on / type ext3 (rw)
proc on /proc type proc (rw)
sysfs on /sys type sysfs (rw)
usbfs on /proc/bus/usb type usbfs (rw)
tmpfs on /dev/shm type tmpfs (rw)
devpts on /dev/pts type devpts (rw,gid=5,mode=620)
/dev/md3 on /boot type ext3 (rw)
/dev/md7 on /usr type ext3 (rw)
/dev/md8 on /usr/local type ext3 (rw)
/dev/md9 on /var type ext3 (rw)
/dev/md10 on /tmp type ext3 (rw)
/dev/md11 on /burn type ext3 (rw)
/dev/md12 on /source type ext3 (rw)
/dev/sda13 on /working1 type ext3 (rw)
/dev/sdb13 on /working2 type ext3 (rw)
tmpfs on /dev type tmpfs (rw,size=10M,mode=0755)

--- mdadm.conf
###############################################################################
# Description:   Configuration of RAID1 on frodo.home.lxtec.de
# Author:        Elimar Riesebieter <riesebie@lxtec.de>
# Created:       Thu, 25 Aug 2005 17:15:35 +0200
# Last modified: Thu, 01 Jun 2006 11:34:23 +0200
# File:          /etc/mdadm/mdadm.conf
###############################################################################
DEVICE /dev/sd[ab]3
DEVICE /dev/sd[ab]6
DEVICE /dev/sd[ab]7
DEVICE /dev/sd[ab]8
DEVICE /dev/sd[ab]9
DEVICE /dev/sd[ab]10
DEVICE /dev/sd[ab]11
DEVICE /dev/sd[ab]12
# Scanned via mdadm --examine --scan
ARRAY /dev/md3 level=raid1 num-devices=2 UUID=3839298f:c9093687:393025dc:1b3c919d
   devices=/dev/sda3,/dev/sdb3
ARRAY /dev/md6 level=raid1 num-devices=2 UUID=f43d835e:86c053a4:21502bb0:9290dca2
   devices=/dev/sda6,/dev/sdb6
ARRAY /dev/md7 level=raid1 num-devices=2 UUID=f2491527:d422e6dd:ca9c999a:c2e87612
   devices=/dev/sda7,/dev/sdb7
ARRAY /dev/md8 level=raid1 num-devices=2 UUID=ee0ab0c0:01f0ed2e:14bce312:b34123c7
   devices=/dev/sda8,/dev/sdb8
ARRAY /dev/md9 level=raid1 num-devices=2 UUID=b587ddf4:30418600:5a3f237d:d3a43abc
   devices=/dev/sda9,/dev/sdb9
ARRAY /dev/md10 level=raid1 num-devices=2 UUID=542db8fd:7d7e1671:7d40bad8:97aeea91
   devices=/dev/sda10,/dev/sdb10
ARRAY /dev/md11 level=raid1 num-devices=2 UUID=503375ef:b5bf75e0:a88d90c5:6aba10d8
   devices=/dev/sda11,/dev/sdb11
ARRAY /dev/md12 level=raid1 num-devices=2 UUID=bd216359:c851284f:48244fa0:4ab50427
   devices=/dev/sda12,/dev/sdb12

# Introduced in mdadm 2.5
CREATE owner=root group=disk mode=0660 auto=yes

--- /proc/mdstat:
Personalities : [raid0] [raid1] 
md3 : active raid1 sdb3[0] sda3[1]
      56128 blocks [2/2] [UU]
      
md6 : active raid1 sdb6[0] sda6[1]
      192640 blocks [2/2] [UU]
      
md7 : active raid1 sdb7[0] sda7[1]
      7815488 blocks [2/2] [UU]
      
md8 : active raid1 sdb8[0] sda8[1]
      979840 blocks [2/2] [UU]
      
md9 : active raid1 sdb9[0] sda9[1]
      3903680 blocks [2/2] [UU]
      
md10 : active raid1 sdb10[0] sda10[1]
      979840 blocks [2/2] [UU]
      
md11 : active raid1 sdb11[0] sda11[1]
      1461760 blocks [2/2] [UU]
      
md12 : active raid1 sdb12[0] sda12[1]
      9767424 blocks [2/2] [UU]
      
unused devices: <none>

--- /proc/partitions:
major minor  #blocks  name

   8     0  117220824 sda
   8     1   14651248 sda1
   8     2    9767520 sda2
   8     3      56227 sda3
   8     4          1 sda4
   8     5     979933 sda5
   8     6     192748 sda6
   8     7    7815591 sda7
   8     8     979933 sda8
   8     9    3903763 sda9
   8    10     979933 sda10
   8    11    1461883 sda11
   8    12    9767488 sda12
   8    13   66661686 sda13
   8    16  117220824 sdb
   8    17   14651248 sdb1
   8    18    9767520 sdb2
   8    19      56227 sdb3
   8    20          1 sdb4
   8    21     979933 sdb5
   8    22     192748 sdb6
   8    23    7815591 sdb7
   8    24     979933 sdb8
   8    25    3903763 sdb9
   8    26     979933 sdb10
   8    27    1461883 sdb11
   8    28    9767488 sdb12
   8    29   66661686 sdb13
   9    12    9767424 md12
   9    11    1461760 md11
   9    10     979840 md10
   9     9    3903680 md9
   9     8     979840 md8
   9     7    7815488 md7
   9     6     192640 md6
   9     3      56128 md3

--- initrd.img-2.6.17-rc6-frodo:

--- /proc/modules:

-- System Information:
Debian Release: testing/unstable
  APT prefers unstable
  APT policy: (990, 'unstable'), (500, 'testing')
Architecture: i386 (i686)
Shell:  /bin/sh linked to /bin/bash
Kernel: Linux 2.6.17-rc6-frodo
Locale: LANG=de_DE@euro, LC_CTYPE=de_DE@euro (charmap=ISO-8859-15)

Versions of packages mdadm depends on:
ii  debconf [debconf-2.0]         1.5.1      Debian configuration management sy
ii  libc6                         2.3.6-15   GNU C Library: Shared libraries
ii  lsb-base                      3.1-10     Linux Standard Base 3.1 init scrip
ii  makedev                       2.3.1-81   creates device files in /dev

Versions of packages mdadm recommends:
ii  exim4-daemon-light [mail-tr 4.62-1lxtec1 lightweight exim MTA (v4) daemon

-- debconf information:
* mdadm/autostart: true
* mdadm/mail_to: root
* mdadm/warning:
* mdadm/start_daemon: true

-- 
  Alles was viel bedacht wird ist bedenklich!;-)
         Friedrich Nietzsche

Message #14 received at 372618-submitter@bugs.debian.org (full text, mbox, reply):

From: martin f krafft <madduck@debian.org>

To: 372618-submitter@bugs.debian.org

Subject: Re: Bug#372618: mdadm --monitor consumes much memory and idle cpu

Date: Sun, 11 Jun 2006 08:01:41 +0200

[Message part 1 (text/plain, inline)]

tags 372618 confirmed
thanks

Before:
  root      7068  0.0  0.0   3036   788 ?        Ss   Jun09   0:00
  /sbin/mdadm --monitor --pid-file /var/run/mdadm.pid --mail root
  --daemonise --scan

After upgrade:
  root     27042 39.5  0.5  16828 10200 ?        Ss   07:59   0:01
  /sbin/mdadm --monitor --pid-file /var/run/mdadm.pid --mail root
  --daemonise --scan

So I can definitely confirm this and have sent it upstream. Thanks
for your report!

-- 
 .''`.     martin f. krafft <madduck@debian.org>
: :'  :    proud Debian developer and author: http://debiansystem.info
`. `'`
  `-  Debian - when you have better things to do than fixing a system
 
"zwei monologe, die sich gegenseitig
 immer und immer wieder störend unterbrechen,
 nennt man eine diskussion."
                                                    -- charles tschopp

[signature.asc (application/pgp-signature, inline)]

Message #19 received at 372618@bugs.debian.org (full text, mbox, reply):

From: Neil Brown <neilb@suse.de>

To: 372618@bugs.debian.org

Subject: Re: Fwd: Bug#372618: mdadm --monitor consumes much memory and idle cpu

Date: Sun, 11 Jun 2006 19:49:32 +1000

On Saturday June 10, madduck@debian.org wrote:
> 
> Neil, this one should be of interest to you. Please reply to
> 372618@bugs.debian.org (reply-to set).
> 
> ----- Forwarded message from Elimar Riesebieter <riesebie@lxtec.de> -----
> 
...
> 
> The memory and cpu consumption grows up to around 50% running 
> /sbin/mdadm --monitor --pid-file /var/run/mdadm.pid --mail root
> --daemonise --scan
> 

Yeah, thanks...

See patch.

NeilBrown



Fix memory leak in monitor mode

When rescanning /dev, we didn't free the old list.
Also don't search for device with a number of 0,0

Signed-off-by: Neil Brown <neilb@suse.de>

### Diffstat output
 ./util.c |   13 +++++++++++++
 1 file changed, 13 insertions(+)

diff ./util.c~current~ ./util.c
--- ./util.c~current~	2006-06-11 19:48:02.000000000 +1000
+++ ./util.c	2006-06-11 19:39:27.000000000 +1000
@@ -416,10 +416,23 @@ char *map_dev(int major, int minor, int 
 	struct devmap *p;
 	char *std = NULL, *nonstd=NULL;
 	int did_check = 0;
+
+	if (major == 0 && minor == 0) {
+		if (!create)
+			return NULL;
+		else
+			return "0:0";
+	}
  retry:
 	if (!devlist_ready) {
 		char *dev = "/dev";
 		struct stat stb;
+		while(devlist) {
+			struct devmap *d = devlist;
+			devlist = d->next;
+			free(d->name);
+			free(d);
+		}
 		if (lstat(dev, &stb)==0 &&
 		    S_ISLNK(stb.st_mode))
 			dev = "/dev/.";

Message #28 received at 372618@bugs.debian.org (full text, mbox, reply):

From: Elimar Riesebieter <riesebie@lxtec.de>

To: Debian Bug Tracking System <372618@bugs.debian.org>

Subject: mdadm: fixed in new upstream 2.5.1

Date: Sun, 25 Jun 2006 16:54:13 +0200

Package: mdadm
Followup-For: Bug #372618

IMHO 60-endian-lkh-fix.patch isn't needed in 2.5.1.

Elimar

-- 
  Experience is something you don't get until 
  just after you need it!

Message #37 received at 372618-done@bugs.debian.org (full text, mbox, reply):

From: martin f krafft <madduck@debian.org>

To: 361408-done@bugs.debian.org, 318230-done@bugs.debian.org, 352798-done@bugs.debian.org, 310247-done@bugs.debian.org, 323950-done@bugs.debian.org, 333486-done@bugs.debian.org, 344714-done@bugs.debian.org, 360290-done@bugs.debian.org, 361116-done@bugs.debian.org, 337903-done@bugs.debian.org, 363592-done@bugs.debian.org, 356153-done@bugs.debian.org, 369779-done@bugs.debian.org, 372618-done@bugs.debian.org, 367901-done@bugs.debian.org, 373802-done@bugs.debian.org, 310241-done@bugs.debian.org, 375876-done@bugs.debian.org, 376009-done@bugs.debian.org, 354144-done@bugs.debian.org, 376181-done@bugs.debian.org, 377071-done@bugs.debian.org, 377968-done@bugs.debian.org, 299623-done@bugs.debian.org

Subject: Fixed by mdadm 2.5.2-7 upload to unstable

Date: Tue, 18 Jul 2006 10:03:49 +0200

[Message part 1 (text/plain, inline)]

Version: 2.5.2-7

I believe this mdadm bug has been solved in the 2.5.2-7 upload
I just made to unstable. If it persists, please reopen the bug and
give details.

Since I forgot to create proper changes file for the upload,
I include the appropriate changelog entries below:

 mdadm (2.5.2-7) unstable; urgency=low
 .
   * Release to unstable.
 .
 mdadm (2.5.2-6) experimental; urgency=low
 .
   * Adding mdrun to generated udeb. I will only remove mdrun after etch.
   * Updated debconf translations:
     - Dutch, thanks to Frans "Franzerl" Pop! (closes: #377412)
     - French, thanks to Florentin Duneau! (closes: #377968)
 .
 mdadm (2.5.2-5) UNRELEASED; urgency=low
 .
   * Remove the check for the lvm prereq in the initramfs hook, as #369617 is
     now fixed. Thus conflicts against initramfs-tools (<< 0.65).
   * Updated debconf translations:
     - German, thanks to Mario Joußen!
   * Added short note about maintenance in SVN to README.Debian.
 .
 mdadm (2.5.2-4) UNRELEASED; urgency=low
 .
   * The "it takes two to swing" release.
   * Now does not stop arrays on upgrade or remove. Thanks (and sorry) to
     Christian Pernegger (and hopefully no others).
   * Fixed small problem in debconf configuration script related to unbound
     MAIL_TO variable. Thanks to Christian Pernegger.
 .
 mdadm (2.5.2-3) UNRELEASED; urgency=low
 .
   * The initramfs now gets all RAID modules installed. It's a lot safer to
     have them all around, the size difference is negligible, and we still only
     load the needed ones at boot time.
   * Added /usr/share/mdadm/checkarray, which can be used to check arrays for
     parity. Also added a debconf question to let the user choose whether
     cron should run these checks (closes: #377071).
   * Only shut down arrays automatically when they've been automatically
     started (closes: #376009).
   * Make sure the user has a chance to choose the autostart feature by
     elevating the debconf priority to high (see #376009). The warning about
     reuse of RAID components has also been elevated to debconf priority high.
   * The MAIL_TO setting from /etc/default/mdadm has been removed. Instead, use
     MAILADDR in /etc/mdadm/mdadm.conf. See mdadm.conf(5). Your setting should
     be automatically migrated.
   * Now rewrites /etc/default/mdadm (but preserves settings) instead of trying
     to patch it with changes.
   * Added note to README.Debian to ensure users know that only the devices
     listed in mdadm.conf will be autostarted (see #376009).
   * Now includes latest md.txt from kernel documentation in
     /usr/share/doc/mdadm/md.txt.gz.
   * Added some more recipes to /usr/share/doc/mdadm/README.recipes.gz.
   * Updated debconf translations:
     - Swedish, thanks to Daniel Nylander!
     - Brazilian Portuguese, thanks to Felipe Augusto van de Wiel!
     - Czech, thanks to Miroslav Kure!
     - Russian, thanks to Yuri Kozlov (closes: #376181).
   * Further updates to the debconf templates; I hope the translators aren't
     going to kill me.
 .
 mdadm (2.5.2-2) experimental; urgency=low
 .
   * The "if it weren't for Munich's wheat beer, there'd be no" release.
   * Removed -fno-strict-aliasing from compiler options, after upstream fixed
     the bug that led to its use (see #369779, #356153). Thanks to Elimar
     Riesebieter for pointing this out (closes: #375876).
   * Moved detection of RAID devices from initramfs hook to debconf control
     file, and added a (low-priority) debconf question as to which devices
     should be started early in the boot sequence. For the cases where we
     failed to auto-detect previously (e.g. root on LVM on RAID), it's paranoid
     and suggests to start them all (closes: #375879). Thanks to Alec Berryman
     for spotting this.
   * Fixed a typo in README.experimental, which could lead to an unbootable
     system with initramfs-tools 0.64 or before. Again, thanks to Alec for
     spotting this.
   * Extended bug script to include --examine output for all components (at
     least if called by root, which hopefully should never happen. Err,
     wait...)
   * Disabled deprecation warning in mdrun until the transition is complete.
   * Reworded the debconf templates due to a new question, and also for
     readability.
 .
 mdadm (2.5.2-1) experimental; urgency=low
 .
   * New upstream release.
   * Implemented checks in the initramfs hooks and scripts for compatibility
     with initramfs-tools. Now we do not need a conflict anymore because
     mdadm's hooks and scripts will simply do nothing while the ones provided
     by initramfs-tools are still present.
   * Not using /bin/bash for mdrun, which I thought we'd need for read timeout
     support (for the deprecation warning). Since the -n and -t flags to the
     read shell builtin are non-POSIX, I dropped them, they were merely
     cosmetic anyway.
 .
 mdadm (2.5.1-2) experimental; urgency=low
 .
   * Updating dependency on initramfs-tools, which has not yet adopted to mdadm
     taking over the hooks.
 .
 mdadm (2.5.1-1) experimental; urgency=low
 .
   * New upstream release:
     - Really fixes return status of examine (closes: #367901).
     - Fixes a memory leak in monitor mode (closes: #372618).
     - Fixes compiler warnings and errors (closes: #373802, #356153, #369779).
     - Fix byte swapping issues (closes: #369765).
     - Now lists devices it stops (closes: #369850). This also leads to
       beautification of the init.d script's stop action.
   * Fixed RAID init script to not complain about missing logger command.
 .
 mdadm (2.5-4) experimental; urgency=low
 .
   * The "would you like fries with your parasite?" release.
   * Now does not require RAID support from the kernel just for package
     installation; that was silly of me, sorry (closes: Bug#370115).
   * Added version to Replaces: initramfs-tools dependency.
   * Further init.d script improvements.
   * Recommends mail-transport-agent, or the monitor daemon won't be able to
     send anything.
   * Ignores failures from modprobe in postinst when RAID modules are not
     available (closes: #370582).
 .
 mdadm (2.5-3) experimental; urgency=low
 .
   * Added /usr/share/doc/mdadm/README.recipes with some common usage examples.
   * Vastly improved the mdadm-raid init.d script output, and removed bashisms.
 .
 mdadm (2.5-2) experimental; urgency=low
 .
   * The "on her majesty's secret service" release.
   * Enabled -DDEBIAN during build, which will take care of default permissions
     on devices created by mdadm. Together with the CREATE configuration
     directive in 2.5, this now certainly closes: Bug#310241.
   * Added a patch (incorporating lib/mm/xlate.h from lvm2) to prevent direct
     access to kernel headers from userspace (closes: Bug#369765).
   * Disabled strict aliasing compiler checks until we find a better
     implementation for linked lists in C (closes: Bug#369779, Bug#356153).
   * Actually decreased the size of the udeb mdadm binary with -Os
     -fomit-frame-pointer (as suggested by Joey Hess) (closes: Bug#314370)
   * Added Replaces: initramfs-tools to communicate that we're not conflicting
     but replacing instead (see Bug#367567)
   * Updated conflict with initramfs-tools to (<< 0.63) per suggestion by the
     maintainers.
 .
 mdadm (2.5-1) experimental; urgency=low
 .
   * The "show me the way to the next whiskey bar" release.
   * See /usr/share/doc/mdadm/README.experimental or
     http://madduck.net/~madduck/scratch/README.experimental
   * New upstream release:
     - mails include /proc/mdstat output (closes: Bug#355882)
     - allows specification of device permissions in config (closes: Bug#310241)
   * /sbin/mdrun has been deprecated and replaced by calls to /sbin/mdadm;
     a proper deprecation warning is in place (see NEWS).
   * Moved initramfs hook and script into the package, and switched it to mdadm
     (from mdadm. Thanks to Stephen Frost for his help (closes: Bug#354144).
     This should make sure that the right minor numbers are chosen during boot
     (mainly because mdadm takes care of it all) (closes: Bug#361408).
   * Removing mdrun from the udeb (d-i patch submitted to debian-boot mailing
     list)
   * Upstream links against openssl for SHA1 support (homehost feature), which
     is a problem. An internal SHA1 implementation is provided, however, so
     I just link against that.
   * Switched init.d scripts to use LSB-compliant output.
   * Enhanced init.d script output.

-- 
Please do not send copies of list mail to me; I read the list!
 
 .''`.     martin f. krafft <madduck@debian.org>
: :'  :    proud Debian developer and author: http://debiansystem.info
`. `'`
  `-  Debian - when you have better things to do than fixing a system

[signature.asc (application/pgp-signature, inline)]

Send a report that this bug log contains spam.