Debian Bug report logs - #483216
sqlite3: sqlite fails to update dbs used from a CIFS mount

version graph

Package: sqlite3; Maintainer for sqlite3 is Laszlo Boszormenyi (GCS) <gcs@debian.org>; Source for sqlite3 is src:sqlite3.

Reported by: Daniel Kahn Gillmor <dkg@fifthhorseman.net>

Date: Tue, 27 May 2008 20:24:01 UTC

Severity: normal

Tags: wontfix

Merged with 509654

Found in version sqlite3/3.5.8-4

Reply or subscribe to this bug.

Toggle useless messages

View this report as an mbox folder, status mbox, maintainer mbox


Report forwarded to debian-bugs-dist@lists.debian.org, Laszlo Boszormenyi (GCS) <gcs@debian.hu>:
Bug#483216; Package sqlite3. Full text and rfc822 format available.

Acknowledgement sent to Daniel Kahn Gillmor <dkg-debian.org@fifthhorseman.net>:
New Bug report received and forwarded. Copy sent to Laszlo Boszormenyi (GCS) <gcs@debian.hu>. Full text and rfc822 format available.

Message #5 received at submit@bugs.debian.org (full text, mbox):

From: Daniel Kahn Gillmor <dkg-debian.org@fifthhorseman.net>
To: Debian Bug Tracking System <submit@bugs.debian.org>
Subject: sqlite3: sqlite fails to update dbs used from a CIFS mount
Date: Tue, 27 May 2008 16:23:22 -0400
Package: sqlite3
Version: 3.5.8-4
Severity: normal

SQLite works fine when used on most filesystems, but it seems to choke
when used on a cifs-mounted filesystem.

In the test below, i'm connecting to a debian etch (3.0.24-6etch9)
samba server running on "bob" (the leading number in the prompt is the
return code from the previous command):

[0 dkg@squeak ~]$ mount | egrep '/(tmp|mnt)'
/tmp on /tmp type tmpfs (rw,nosuid,nodev)
//bob/dmz on /mnt type cifs (rw,mand,unc=\\bob\dmz,username=,uid=1000,posixpaths,rsize=16384,wsize=57344)
[0 dkg@squeak ~]$ rm -f /tmp/testdb 
[0 dkg@squeak ~]$ echo 'create table foo (x int); insert into foo (x) values (1);' | sqlite3 /tmp/testdb
[0 dkg@squeak ~]$ echo 'select * from foo;' | sqlite3 /tmp/testdb
1
[0 dkg@squeak ~]$ cp /tmp/testdb /mnt/testdb
cp: overwrite `/mnt/testdb'? y
[0 dkg@squeak ~]$ echo 'select * from foo;' | sqlite3 /mnt/testdb
1
[0 dkg@squeak ~]$ echo 'insert into foo (x) values (2);' | sqlite3 /tmp/testdb
[0 dkg@squeak ~]$ echo 'insert into foo (x) values (2);' | sqlite3 /mnt/testdb
SQL error near line 1: database is locked
[1 dkg@squeak ~]$ echo 'select * from foo;' | sqlite3 /tmp/testdb
1
2
[0 dkg@squeak ~]$ echo 'select * from foo;' | sqlite3 /mnt/testdb
1
[0 dkg@squeak ~]$ 


I'm using mount.cifs on the client from lenny:

[0 dkg@squeak ~]$ dpkg -l smbfs
Desired=Unknown/Install/Remove/Purge/Hold
| Status=Not/Inst/Cfg-files/Unpacked/Failed-cfg/Half-inst/trig-aWait/Trig-pend
|/ Err?=(none)/Hold/Reinst-required/X=both-problems (Status,Err: uppercase=bad)
||/ Name           Version        Description
+++-==============-==============-============================================
ii  smbfs          1:3.0.28a-3    mount and umount commands for the smbfs (for
[0 dkg@squeak ~]$ 


This combination is a bad one as sqlite starts to be used more widely.
In particular, firefox v3 uses sqlite internally, so any institutions
using CIFS-mounted home directories will find firefox3 failing for
their users.

This bug is also reported to ubuntu, fwiw:

  https://bugs.launchpad.net/ubuntu/+source/samba/+bug/117730

Thanks for maintaining sqlite3 in debian,

       --dkg

-- System Information:
Debian Release: lenny/sid
  APT prefers testing
  APT policy: (500, 'testing'), (200, 'unstable'), (101, 'experimental')
Architecture: i386 (i686)

Kernel: Linux 2.6.24-1-686 (SMP w/1 CPU core)
Locale: LANG=en_US.UTF-8, LC_CTYPE=en_US.UTF-8 (charmap=UTF-8)
Shell: /bin/sh linked to /bin/bash

Versions of packages sqlite3 depends on:
ii  libc6                         2.7-10     GNU C Library: Shared libraries
ii  libreadline5                  5.2-3      GNU readline and history libraries
ii  libsqlite3-0                  3.5.8-4    SQLite 3 shared library

sqlite3 recommends no packages.

-- no debconf information




Information forwarded to debian-bugs-dist@lists.debian.org, Laszlo Boszormenyi (GCS) <gcs@debian.hu>:
Bug#483216; Package sqlite3. Full text and rfc822 format available.

Acknowledgement sent to Florian Weimer <fw@deneb.enyo.de>:
Extra info received and forwarded to list. Copy sent to Laszlo Boszormenyi (GCS) <gcs@debian.hu>. Full text and rfc822 format available.

Message #10 received at 483216@bugs.debian.org (full text, mbox):

From: Florian Weimer <fw@deneb.enyo.de>
To: Daniel Kahn Gillmor <dkg-debian.org@fifthhorseman.net>
Cc: 483216@bugs.debian.org
Subject: Re: Bug#483216: sqlite3: sqlite fails to update dbs used from a CIFS mount
Date: Wed, 28 May 2008 21:08:36 +0200
* Daniel Kahn Gillmor:

> This bug is also reported to ubuntu, fwiw:
>
>   https://bugs.launchpad.net/ubuntu/+source/samba/+bug/117730

This looks like cifs not providing POSIX file system semantics.  I dont
know if this ia bug, or an inherent protocol limitation.




Information forwarded to debian-bugs-dist@lists.debian.org, Laszlo Boszormenyi (GCS) <gcs@debian.hu>:
Bug#483216; Package sqlite3. Full text and rfc822 format available.

Acknowledgement sent to Daniel Kahn Gillmor <dkg-debian.org@fifthhorseman.net>:
Extra info received and forwarded to list. Copy sent to Laszlo Boszormenyi (GCS) <gcs@debian.hu>. Full text and rfc822 format available.

Message #15 received at 483216@bugs.debian.org (full text, mbox):

From: Daniel Kahn Gillmor <dkg-debian.org@fifthhorseman.net>
To: Florian Weimer <fw@deneb.enyo.de>
Cc: Daniel Kahn Gillmor <dkg-debian.org@fifthhorseman.net>, 483216@bugs.debian.org
Subject: Re: Bug#483216: sqlite3: sqlite fails to update dbs used from a CIFS mount
Date: Wed, 28 May 2008 15:44:03 -0400
[Message part 1 (text/plain, inline)]
On Wed 2008-05-28 15:08:36 -0400, Florian Weimer wrote:

> * Daniel Kahn Gillmor:
>
>> This bug is also reported to ubuntu, fwiw:
>>
>>   https://bugs.launchpad.net/ubuntu/+source/samba/+bug/117730
>
> This looks like cifs not providing POSIX file system semantics.  I
> dont know if this ia bug, or an inherent protocol limitation.

I agree that this is one possible interpretation of the problem, but
SQLite does *not* exhibit the same problem when it is run on Windows
clients using data directories that are "network mapped" via CIFS from
the same samba server.  Instead, it is able to read and write the
testdb without a problem.

So either the sqlite build for Windows doesn't internally depend on
those filesystem semantics, or there's a way to work around them on
Windows.  Maybe it's possible for SQLite in debian (and other unices)
to use the same workaround if it detects that it's working on CIFS?
Or maybe this is a bug in cifs.mount?  Or maybe a bug in the kernel's
cifs.ko?  Or in samba itself somehow?  I'd be happy to get pointed in
the right direction about where to get started fixing this.  (i'd be
even happier to be able to ditch CIFS, but that's not politically
feasible)

This bug is pretty serious because it's likely to strand users with
CIFS-exported home directories once Firefox 3 replaces Firefox 2.
It's already causing breakages in CIFS-homedir environments under
Ubuntu 8.04, and in debian with other apps that use SQLite
(e.g. f-spot, django, etc).

Thanks for the feedback and for any other direction or advice you can
give.

Regards,

       --dkg
[Message part 2 (application/pgp-signature, inline)]

Information forwarded to debian-bugs-dist@lists.debian.org, Laszlo Boszormenyi (GCS) <gcs@debian.hu>:
Bug#483216; Package sqlite3. Full text and rfc822 format available.

Acknowledgement sent to Laszlo Boszormenyi <gcs@debian.hu>:
Extra info received and forwarded to list. Copy sent to Laszlo Boszormenyi (GCS) <gcs@debian.hu>. Full text and rfc822 format available.

Message #20 received at 483216@bugs.debian.org (full text, mbox):

From: Laszlo Boszormenyi <gcs@debian.hu>
To: Daniel Kahn Gillmor <dkg-debian.org@fifthhorseman.net>, Florian Weimer <fw@deneb.enyo.de>, 483216@bugs.debian.org
Cc: control@bugs.debian.org
Subject: Re: Bug#483216: sqlite3: sqlite fails to update dbs used from a CIFS mount
Date: Wed, 28 May 2008 22:14:41 +0200
package sqlite3
tags 483216 + wontfix
thanks

Hi,

On Tue, 2008-05-27 at 16:23 -0400, Daniel Kahn Gillmor wrote:
> SQLite works fine when used on most filesystems, but it seems to choke
> when used on a cifs-mounted filesystem.
 This is a known 'bug'[1] in SQLite3. There is a workaround, quoting:
"You can work around the problem by compiling with
-DSQLITE_DISABLE_DIRSYNC=1".

But the webpage also states: "SQLite appears to be doing the right thing
here. The bug appears to be OpenWRT-Linux, not in SQLite."

Using the mentioned workaround is very risky:
"That macro disables the directory syncing logic. But then if you lose
power at exactly the wrong moment and your rollback journal vanishes (or
gets moved to /lost+found) and your database is corrupted - don't
complain. I better solution, it seems, would be to fix OpenWRT-Linux.".

As for now, I don't want to make this risk to all of our users, tagging
this bug as wontfix.

Regards,
Laszlo/GCS
[1] http://sqlite.org/cvstrac/tktview?tn=1751





Tags added: wontfix Request was from Laszlo Boszormenyi <gcs@debian.hu> to control@bugs.debian.org. (Wed, 28 May 2008 20:30:03 GMT) Full text and rfc822 format available.

Information forwarded to debian-bugs-dist@lists.debian.org, Laszlo Boszormenyi (GCS) <gcs@debian.hu>:
Bug#483216; Package sqlite3. Full text and rfc822 format available.

Acknowledgement sent to Daniel Kahn Gillmor <dkg-debian.org@fifthhorseman.net>:
Extra info received and forwarded to list. Copy sent to Laszlo Boszormenyi (GCS) <gcs@debian.hu>. Full text and rfc822 format available.

Message #27 received at 483216@bugs.debian.org (full text, mbox):

From: Daniel Kahn Gillmor <dkg-debian.org@fifthhorseman.net>
To: Laszlo Boszormenyi <gcs@debian.hu>
Cc: Daniel Kahn Gillmor <dkg-debian.org@fifthhorseman.net>, Florian Weimer <fw@deneb.enyo.de>, 483216@bugs.debian.org
Subject: Re: Bug#483216: sqlite3: sqlite fails to update dbs used from a CIFS mount
Date: Wed, 28 May 2008 19:13:16 -0400
[Message part 1 (text/plain, inline)]
On Wed 2008-05-28 16:14:41 -0400, Laszlo Boszormenyi wrote:

> http://sqlite.org/cvstrac/tktview?tn=1751

Thanks for the followup, but i don't think this is the same bug. In ,
the error reported by sqlite was: "Error: disk I/O error", but i'm
getting "SQL error near line 1: database is locked"

I tried compiling sqlite3 with -DSQLITE_DISABLE_DIRSYNC=1 as a
workaround, and repeated my same experiment, but it still fails.

The basic experiment starts with a copied sqlite database that was
created with:

 echo 'create table foo (x int); insert into foo (x) values (1);' | sqlite3 /tmp/testdb

A copy of this database file was then experimented on over a tmpfs and
a cifsmount twice: once each with the stock libsqlite3, and once each
with a libsqlite3 built with -DSQLITE_DISABLE_DIRSYNC=1.

The command to generate the strace was (for example):

 echo 'insert into foo (x) values (2);' | strace -o tmpfs.nodirsync.strace sqlite3 /tmp/testdb

The error seems to be happening in the second call that attempts to
set a write lock on the file in a range that it already holds a read
lock on:


[0 dkg@squeak sqlite3]$ grep '^fcntl64(3, F_SETLK64' tmpfs.nodirsync.strace
fcntl64(3, F_SETLK64, {type=F_RDLCK, whence=SEEK_SET, start=1073741824, len=1}, 0xbfbd9034) = 0
fcntl64(3, F_SETLK64, {type=F_RDLCK, whence=SEEK_SET, start=1073741826, len=510}, 0xbfbd9034) = 0
fcntl64(3, F_SETLK64, {type=F_UNLCK, whence=SEEK_SET, start=1073741824, len=1}, 0xbfbd9034) = 0
fcntl64(3, F_SETLK64, {type=F_UNLCK, whence=SEEK_SET, start=0, len=0}, 0xbfbd9144) = 0
fcntl64(3, F_SETLK64, {type=F_RDLCK, whence=SEEK_SET, start=1073741824, len=1}, 0xbfbd90b4) = 0
fcntl64(3, F_SETLK64, {type=F_RDLCK, whence=SEEK_SET, start=1073741826, len=510}, 0xbfbd90b4) = 0
fcntl64(3, F_SETLK64, {type=F_UNLCK, whence=SEEK_SET, start=1073741824, len=1}, 0xbfbd90b4) = 0
fcntl64(3, F_SETLK64, {type=F_WRLCK, whence=SEEK_SET, start=1073741825, len=1}, 0xbfbd9174) = 0
fcntl64(3, F_SETLK64, {type=F_WRLCK, whence=SEEK_SET, start=1073741824, len=1}, 0xbfbd8fa4) = 0
fcntl64(3, F_SETLK64, {type=F_WRLCK, whence=SEEK_SET, start=1073741826, len=510}, 0xbfbd8fa4) = 0
fcntl64(3, F_SETLK64, {type=F_RDLCK, whence=SEEK_SET, start=1073741826, len=510}, 0xbfbd90f4) = 0
fcntl64(3, F_SETLK64, {type=F_UNLCK, whence=SEEK_SET, start=1073741824, len=2}, 0xbfbd90f4) = 0
fcntl64(3, F_SETLK64, {type=F_UNLCK, whence=SEEK_SET, start=0, len=0}, 0xbfbd90e4) = 0
[0 dkg@squeak sqlite3]$ grep '^fcntl64(3, F_SETLK64' cifs.nodirsync.strace
fcntl64(3, F_SETLK64, {type=F_RDLCK, whence=SEEK_SET, start=1073741824, len=1}, 0xbfb9bff4) = 0
fcntl64(3, F_SETLK64, {type=F_RDLCK, whence=SEEK_SET, start=1073741826, len=510}, 0xbfb9bff4) = 0
fcntl64(3, F_SETLK64, {type=F_UNLCK, whence=SEEK_SET, start=1073741824, len=1}, 0xbfb9bff4) = 0
fcntl64(3, F_SETLK64, {type=F_UNLCK, whence=SEEK_SET, start=0, len=0}, 0xbfb9c104) = 0
fcntl64(3, F_SETLK64, {type=F_RDLCK, whence=SEEK_SET, start=1073741824, len=1}, 0xbfb9c074) = 0
fcntl64(3, F_SETLK64, {type=F_RDLCK, whence=SEEK_SET, start=1073741826, len=510}, 0xbfb9c074) = 0
fcntl64(3, F_SETLK64, {type=F_UNLCK, whence=SEEK_SET, start=1073741824, len=1}, 0xbfb9c074) = 0
fcntl64(3, F_SETLK64, {type=F_WRLCK, whence=SEEK_SET, start=1073741825, len=1}, 0xbfb9c134) = 0
fcntl64(3, F_SETLK64, {type=F_WRLCK, whence=SEEK_SET, start=1073741824, len=1}, 0xbfb9bf64) = 0
fcntl64(3, F_SETLK64, {type=F_WRLCK, whence=SEEK_SET, start=1073741826, len=510}, 0xbfb9bf64) = -1 EACCES (Permission denied)
fcntl64(3, F_SETLK64, {type=F_RDLCK, whence=SEEK_SET, start=1073741826, len=510}, 0xbfb9c564) = 0
fcntl64(3, F_SETLK64, {type=F_UNLCK, whence=SEEK_SET, start=1073741824, len=2}, 0xbfb9c564) = 0
fcntl64(3, F_SETLK64, {type=F_UNLCK, whence=SEEK_SET, start=0, len=0}, 0xbfb9c634) = 0
[0 dkg@squeak sqlite3]$ 

Attached, you should be able to find all 4 straces.  if you compare
the normal straces with the nodirsync straces, you can see that the
directory sync is indeed disabled, but the failure still occurs.  If i
can help debug this in any other way, please let me know.

Regards,

        --dkg

[tmpfs.strace (text/plain, attachment)]
[cifs.strace (text/plain, attachment)]
[tmpfs.nodirsync.strace (text/plain, attachment)]
[cifs.nodirsync.strace (text/plain, attachment)]
[Message part 6 (application/pgp-signature, inline)]

Information forwarded to debian-bugs-dist@lists.debian.org, Laszlo Boszormenyi (GCS) <gcs@debian.hu>:
Bug#483216; Package sqlite3. Full text and rfc822 format available.

Acknowledgement sent to Daniel Kahn Gillmor <dkg-debian.org@fifthhorseman.net>:
Extra info received and forwarded to list. Copy sent to Laszlo Boszormenyi (GCS) <gcs@debian.hu>. Full text and rfc822 format available.

Message #32 received at 483216@bugs.debian.org (full text, mbox):

From: Daniel Kahn Gillmor <dkg-debian.org@fifthhorseman.net>
To: 483216@bugs.debian.org
Cc: Laszlo Boszormenyi <gcs@debian.hu>, Florian Weimer <fw@deneb.enyo.de>, Daniel Kahn Gillmor <dkg-debian.org@fifthhorseman.net>
Subject: Re: Bug#483216: sqlite3: sqlite fails to update dbs used from a CIFS mount
Date: Wed, 28 May 2008 23:06:41 -0400
[Message part 1 (text/plain, inline)]
Ah: a bit more information after reading fcntl64(2) and mount.cifs(8),
noticing that the "mand" flag was present in /proc/mounts for the CIFS
mount, and some more testing:

I'm able to successfully perform the record insertion if i mount the
CIFS share with the "nobrl" option.  When i mount it with "nobrl", the
"mand" flag in the /proc/mounts line disappears.  The record insertion
works whether or not i've build libsqlite3 with
-DSQLITE_DISABLE_DIRSYNC=1, so that particular concern appears to be
unrelated (note that the fsync() call still fails with DIRSYNC
enabled, but sqlite appears to just plow ahead).

From mount.cifs(8):

      nobrl
          Do not send byte range lock requests to the server. This is
          necessary for certain applications that break with cifs
          style mandatory byte range locks (and most cifs servers do
          not yet support requesting advisory byte range locks).

Does this mean that SQLite (and any application which uses it) are
those "certain applications"?

Or is there a bug in SQLite in the way that it uses locks?  Or a bug
in the cifs kernel module?

   --dkg
[Message part 2 (application/pgp-signature, inline)]

Forcibly Merged 483216 509654. Request was from Mike Hommey <mh@glandium.org> to control@bugs.debian.org. (Wed, 24 Dec 2008 08:57:07 GMT) Full text and rfc822 format available.

Changed Bug submitter from Daniel Kahn Gillmor <dkg-debian.org@fifthhorseman.net> to Daniel Kahn Gillmor <dkg@fifthhorseman.net>. Request was from Daniel Kahn Gillmor <dkg@fifthhorseman.net> to control@bugs.debian.org. (Thu, 26 Mar 2009 13:42:26 GMT) Full text and rfc822 format available.

Send a report that this bug log contains spam.


Debian bug tracking system administrator <owner@bugs.debian.org>. Last modified: Sat Apr 19 10:50:28 2014; Machine Name: buxtehude.debian.org

Debian Bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.