Debian Bug report logs - #612915
Random GFS2 partition locking problem

Toggle useless messages

Message #5 received at submit@bugs.debian.org (full text, mbox, reply):

From: Olaf Reitmaier Veracierta <olafrv@gmail.com>

To: Debian Bug Tracking System <submit@bugs.debian.org>

Subject: Random GFS2 partition locking problem

Date: Fri, 11 Feb 2011 09:57:35 -0430

Package: gfs2-tools
Version: 3.0.12-2
Severity: critical
Tags: squeeze

My mail server (cid02) uses NFS shares mounted from a NFS server (sane02) which is exporting GFS2 cluster partition.

On random times my mail server get IO blocked process over NFS shares, when I start to investigate the problem I found
it is NOT problem with NFS server, network or Mail server (postfix, imap, dovecot).

It appears to be a strange GFS2 filesystem block out.

Any help is appreciated.

Events happens as follow and messages begin to throtle...

On server cid02 file /var/log/kern.log

Feb 11 04:57:49 ccsdtfdemcid02 kernel: [93435.428896] nfs: server 10.0.18.35 not responding, still trying

On server sane02 file /var/log/kern.log

Feb 11 04:56:12 ccsdtfdemsane02 kernel: [63721.168102] INFO: task nfsd:5340 blocked for more than 120 seconds.
Feb 11 04:56:12 ccsdtfdemsane02 kernel: [63721.168159] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
Feb 11 04:56:12 ccsdtfdemsane02 kernel: [63721.168214] nfsd          D 0000000000000002     0  5340      2 0x00000000
Feb 11 04:56:12 ccsdtfdemsane02 kernel: [63721.168221]  ffff88012c19f810 0000000000000046 0000000000000000 ffffffffa036293b
Feb 11 04:56:12 ccsdtfdemsane02 kernel: [63721.168228]  0000000000000000 ffffffffa0362495 000000000000f9e0 ffff880121ec1fd8
Feb 11 04:56:12 ccsdtfdemsane02 kernel: [63721.168233]  0000000000015780 0000000000015780 ffff88012c092a60 ffff88012c092d58
Feb 11 04:56:12 ccsdtfdemsane02 kernel: [63721.168239] Call Trace:
Feb 11 04:56:12 ccsdtfdemsane02 kernel: [63721.168261]  [<ffffffffa036293b>] ? dlm_put_lockspace+0x10/0x1c [dlm]
Feb 11 04:56:12 ccsdtfdemsane02 kernel: [63721.168271]  [<ffffffffa0362495>] ? dlm_lock+0x135/0x149 [dlm]
Feb 11 04:56:12 ccsdtfdemsane02 kernel: [63721.168291]  [<ffffffffa04b87c0>] ? gdlm_ast+0x0/0xc2 [gfs2]
Feb 11 04:56:12 ccsdtfdemsane02 kernel: [63721.168301]  [<ffffffffa04b8731>] ? gdlm_bast+0x0/0x3d [gfs2]
Feb 11 04:56:12 ccsdtfdemsane02 kernel: [63721.168309]  [<ffffffffa04a1dcc>] ? gfs2_glock_holder_wait+0x0/0xd [gfs2]
Feb 11 04:56:12 ccsdtfdemsane02 kernel: [63721.168317]  [<ffffffffa04a1dd5>] ? gfs2_glock_holder_wait+0x9/0xd [gfs2]
Feb 11 04:56:12 ccsdtfdemsane02 kernel: [63721.168327]  [<ffffffff812fae26>] ? __wait_on_bit+0x41/0x70
Feb 11 04:56:12 ccsdtfdemsane02 kernel: [63721.168335]  [<ffffffffa04a1dcc>] ? gfs2_glock_holder_wait+0x0/0xd [gfs2]
Feb 11 04:56:12 ccsdtfdemsane02 kernel: [63721.168340]  [<ffffffff812faec0>] ? out_of_line_wait_on_bit+0x6b/0x77
Feb 11 04:56:12 ccsdtfdemsane02 kernel: [63721.168346]  [<ffffffff81064d58>] ? wake_bit_function+0x0/0x23
Feb 11 04:56:12 ccsdtfdemsane02 kernel: [63721.168355]  [<ffffffffa04a1dba>] ? gfs2_glock_wait+0x23/0x28 [gfs2]
Feb 11 04:56:12 ccsdtfdemsane02 kernel: [63721.168364]  [<ffffffffa04a4703>] ? gfs2_glock_nq_init+0x16/0x2a [gfs2]
Feb 11 04:56:12 ccsdtfdemsane02 kernel: [63721.168373]  [<ffffffffa04a505d>] ? gfs2_inode_lookup+0x16c/0x1d1 [gfs2]
Feb 11 04:56:12 ccsdtfdemsane02 kernel: [63721.168382]  [<ffffffffa04a46fb>] ? gfs2_glock_nq_init+0xe/0x2a [gfs2]
Feb 11 04:56:12 ccsdtfdemsane02 kernel: [63721.168392]  [<ffffffffa04b4670>] ? try_rgrp_unlink+0xa3/0xcc [gfs2]
Feb 11 04:56:12 ccsdtfdemsane02 kernel: [63721.168401]  [<ffffffffa04b5105>] ? gfs2_inplace_reserve_i+0x285/0x640 [gfs2]
Feb 11 04:56:12 ccsdtfdemsane02 kernel: [63721.168409]  [<ffffffffa049a243>] ? gfs2_dirent_find+0x0/0x7 [gfs2]
Feb 11 04:56:12 ccsdtfdemsane02 kernel: [63721.168416]  [<ffffffffa0499b50>] ? gfs2_dirent_search+0x13b/0x167 [gfs2]
Feb 11 04:56:12 ccsdtfdemsane02 kernel: [63721.168426]  [<ffffffffa04a521d>] ? gfs2_createi+0x15b/0x912 [gfs2]
Feb 11 04:56:12 ccsdtfdemsane02 kernel: [63721.168430]  [<ffffffff81064c4b>] ? bit_waitqueue+0x10/0xa0
Feb 11 04:56:12 ccsdtfdemsane02 kernel: [63721.168438]  [<ffffffffa04a1c5f>] ? do_promote+0x15b/0x1f3 [gfs2]
Feb 11 04:56:12 ccsdtfdemsane02 kernel: [63721.168447]  [<ffffffffa04a2ed4>] ? run_queue+0xa5/0x127 [gfs2]
Feb 11 04:56:12 ccsdtfdemsane02 kernel: [63721.168456]  [<ffffffffa04ae156>] ? gfs2_create+0x67/0x131 [gfs2]
Feb 11 04:56:12 ccsdtfdemsane02 kernel: [63721.168465]  [<ffffffffa04a513e>] ? gfs2_createi+0x7c/0x912 [gfs2]
Feb 11 04:56:12 ccsdtfdemsane02 kernel: [63721.168472]  [<ffffffff810f78b4>] ? vfs_create+0x6d/0x89
Feb 11 04:56:12 ccsdtfdemsane02 kernel: [63721.168484]  [<ffffffffa044eb27>] ? nfsd_create_v3+0x20a/0x358 [nfsd]
Feb 11 04:56:12 ccsdtfdemsane02 kernel: [63721.168493]  [<ffffffffa0457a14>] ? do_open_lookup+0x91/0x240 [nfsd]
Feb 11 04:56:12 ccsdtfdemsane02 kernel: [63721.168503]  [<ffffffffa0457d7a>] ? nfsd4_open+0x1b7/0x339 [nfsd]
Feb 11 04:56:12 ccsdtfdemsane02 kernel: [63721.168533]  [<ffffffffa0458db1>] ? nfsd4_encode_operation+0x5b/0x135 [nfsd]
Feb 11 04:56:12 ccsdtfdemsane02 kernel: [63721.168548]  [<ffffffffa0457716>] ? nfsd4_proc_compound+0x1fd/0x3b3 [nfsd]
Feb 11 04:56:12 ccsdtfdemsane02 kernel: [63721.168559]  [<ffffffffa0449329>] ? nfsd_dispatch+0xdd/0x1b9 [nfsd]
Feb 11 04:56:12 ccsdtfdemsane02 kernel: [63721.168577]  [<ffffffffa03894ff>] ? svc_process+0x403/0x627 [sunrpc]
Feb 11 04:56:12 ccsdtfdemsane02 kernel: [63721.168588]  [<ffffffffa0449772>] ? nfsd+0x0/0x12e [nfsd]
Feb 11 04:56:12 ccsdtfdemsane02 kernel: [63721.168598]  [<ffffffffa0449857>] ? nfsd+0xe5/0x12e [nfsd]
Feb 11 04:56:12 ccsdtfdemsane02 kernel: [63721.168605]  [<ffffffff81064a5d>] ? kthread+0x79/0x81
Feb 11 04:56:12 ccsdtfdemsane02 kernel: [63721.168615]  [<ffffffff81011baa>] ? child_rip+0xa/0x20
Feb 11 04:56:12 ccsdtfdemsane02 kernel: [63721.168622]  [<ffffffff810649e4>] ? kthread+0x0/0x81
Feb 11 04:56:12 ccsdtfdemsane02 kernel: [63721.168628]  [<ffffffff81011ba0>] ? child_rip+0x0/0x20



-- System Information:
Debian Release: 6.0
  APT prefers oldstable
  APT policy: (500, 'oldstable'), (500, 'stable')
Architecture: amd64 (x86_64)

Kernel: Linux 2.6.32-5-amd64 (SMP w/8 CPU cores)
Locale: LANG=en_US.UTF-8, LC_CTYPE=en_US.UTF-8 (charmap=UTF-8)
Shell: /bin/sh linked to /bin/bash

Versions of packages gfs2-tools depends on:
ii  cman                      3.0.12-2       Red Hat cluster suite - cluster ma
ii  libc6                     2.11.2-10      Embedded GNU C Library: Shared lib
ii  libldap-2.4-2             2.4.23-7       OpenLDAP libraries
ii  libncurses5               5.7+20100313-5 shared libraries for terminal hand
ii  libxml2                   2.7.8.dfsg-2   GNOME XML library
ii  psmisc                    22.11-1        utilities that use the proc file s

gfs2-tools recommends no packages.

gfs2-tools suggests no packages.

-- no debconf information

Message #12 received at 612915@bugs.debian.org (full text, mbox, reply):

From: Frederik Schüler <fs@debian.org>

To: 612915@bugs.debian.org, control@bugs.debian.org

Subject: Random GFS2 partition locking problem

Date: Wed, 16 Nov 2011 21:44:20 +0100

severity 612915 normal
reassign 612915 linux-2.6
thanks

Hi,

this is a bug in gfs2. I suggest you try a newer kernel, for example the 
latest linux-2.6 found on backports.debian.org.

Best regards
Frederik Schüler


-- 
ENOSIG

Message #23 received at 612915-done@bugs.debian.org (full text, mbox, reply):

From: Moritz Mühlenhoff <jmm@inutil.org>

To: 612744-done@bugs.debian.org, 612839-done@bugs.debian.org, 612915-done@bugs.debian.org, 613052-done@bugs.debian.org, 613225-done@bugs.debian.org, 477377-done@bugs.debian.org, 496903-done@bugs.debian.org, 497185-done@bugs.debian.org

Subject: Closing

Date: Wed, 19 Jun 2013 18:56:37 +0200

Hi,
your bug has been filed against the "linux-2.6" source package and was filed for
a kernel older than the recently released Debian 7.x / Wheezy with a severity
less than important.

We don't have the ressources to reproduce the complete backlog of all older kernel
bugs, so we're closing this bug for now. If you can reproduce the bug with Debian Wheezy
or a more recent kernel from testing or unstable, please reopen the bug by sending
a mail to control@bugs.debian.org with the following three commands included in the
mail:

reopen BUGNUMBER
reassign BUGNUMBER src:linux
thanks

Cheers,
        Moritz

Send a report that this bug log contains spam.