Debian Bug report logs - #584254
dpkg should support a --force-unsafe-io option or such

Toggle useless messages

Message #5 received at submit@bugs.debian.org (full text, mbox, reply):

From: LaMont Jones <lamont@debian.org>

To: submit@bugs.debian.org

Subject: dpkg should support a --force-unsafe-io option or such

Date: Wed, 2 Jun 2010 11:14:38 -0600

Package: dpkg
Version: 1.15.6
Severity: normal

With the introduction of fsync() calls to protect data, applications
that do potentially large apt-get install invocations may not want
to incur the penalty of fsync() calls from dpkg.

In the case of building a livecd, this can be the difference between
a 10 minute apt invocation, and 21 minutes.

The use cases where --force-unsafe-io would make sense are those
where apt is modifiying an underlying (possibly empty) base chroot
and the results will be thrown away before the next apt invocation
such that a crash just means "start from the known good base".  The
two that jump to mind are:
  1. livecd builds
  2. buildd chroots where we are using lvm snapshots (or pbuilder or
     whatever) and will be discarding the resulting chroot upon either
     completion or machine reboot).

Obviously, we want the default to be safe, and the option name to be
scary enough that end users don't think it's a good option to use in
daily life.  I spoke with Colin Watson, and he suggested the option
be called --force-unsafe-io.

thanks,
lamont

Message #10 received at 584254@bugs.debian.org (full text, mbox, reply):

From: Jonathan Nieder <jrnieder@gmail.com>

To: LaMont Jones <lamont@debian.org>

Cc: 584254@bugs.debian.org

Subject: Re: dpkg should support a --force-unsafe-io option or such

Date: Wed, 2 Jun 2010 13:30:30 -0500

Hi LaMont,

LaMont Jones wrote:

> With the introduction of fsync() calls to protect data, applications
> that do potentially large apt-get install invocations may not want
> to incur the penalty of fsync() calls from dpkg.
> 
> In the case of building a livecd, this can be the difference between
> a 10 minute apt invocation, and 21 minutes.

Makes sense.

> The use cases where --force-unsafe-io would make sense are those
> where apt is modifiying an underlying (possibly empty) base chroot
> and the results will be thrown away before the next apt invocation
> such that a crash just means "start from the known good base".

As a hackish workaround, it would be nice to get libeatmydata into
Debian (see <http://bugs.debian.org/512588>).  That would prevent
fsync() calls in maintainer scripts, too.  Of course a
--force-unsafe-io option is probably better, since among other things
it expresses the intent more clearly.

Regards,
Jonathan

Message #15 received at 584254@bugs.debian.org (full text, mbox, reply):

From: Guillem Jover <guillem@debian.org>

To: LaMont Jones <lamont@debian.org>, 584254@bugs.debian.org

Subject: Re: Bug#584254: dpkg should support a --force-unsafe-io option or such

Date: Wed, 2 Jun 2010 21:02:25 +0200

On Wed, 2010-06-02 at 11:14:38 -0600, LaMont Jones wrote:
> Package: dpkg
> Version: 1.15.6
> Severity: normal
> 
> With the introduction of fsync() calls to protect data, applications
> that do potentially large apt-get install invocations may not want
> to incur the penalty of fsync() calls from dpkg.
> 
> In the case of building a livecd, this can be the difference between
> a 10 minute apt invocation, and 21 minutes.
> 
> The use cases where --force-unsafe-io would make sense are those
> where apt is modifiying an underlying (possibly empty) base chroot
> and the results will be thrown away before the next apt invocation
> such that a crash just means "start from the known good base".  The
> two that jump to mind are:
>   1. livecd builds
>   2. buildd chroots where we are using lvm snapshots (or pbuilder or
>      whatever) and will be discarding the resulting chroot upon either
>      completion or machine reboot).
> 
> Obviously, we want the default to be safe, and the option name to be
> scary enough that end users don't think it's a good option to use in
> daily life.  I spoke with Colin Watson, and he suggested the option
> be called --force-unsafe-io.

Right, that's what we discussed with Colin at some point, as a last
resort "solution", but I added support for using sync() instead of
fsync() on Linux to avoid the massive slow down. Have you tested
1.15.7.2 (which includes that change), and do the performance issues
persist there. We actually got pretty good results from several testers
on ext4, so if there's no need I'd just avoid adding such option.

thanks,
guillem

Message #20 received at 584254@bugs.debian.org (full text, mbox, reply):

From: Jonathan Nieder <jrnieder@gmail.com>

To: Guillem Jover <guillem@debian.org>, 584254@bugs.debian.org

Cc: LaMont Jones <lamont@debian.org>

Subject: Re: Bug#584254: dpkg should support a --force-unsafe-io option or such

Date: Wed, 2 Jun 2010 14:38:39 -0500

Guillem Jover wrote:

> Have you tested
> 1.15.7.2 (which includes that change), and do the performance issues
> persist there. We actually got pretty good results from several testers
> on ext4

Right, though still significantly (about 20%) worse in some cases than
without any fsync[1].  So I can imagine why a person might still want
to disable it where the sync does not buy anything.

Jonathan

[1] http://bugs.debian.org/578635

Message #27 received at 584254@bugs.debian.org (full text, mbox, reply):

From: Modestas Vainius <modax@debian.org>

To: LaMont Jones <lamont@debian.org>, 584254@bugs.debian.org, Guillem Jover <guillem@debian.org>

Cc: debian-boot@lists.debian.org

Subject: Re: dpkg should support a --force-unsafe-io option or such

Date: Sat, 9 Oct 2010 02:55:52 +0300

[Message part 1 (text/plain, inline)]

Hello,

On trečiadienis 02 Birželis 2010 20:14:38 LaMont Jones wrote:
> With the introduction of fsync() calls to protect data, applications
> that do potentially large apt-get install invocations may not want
> to incur the penalty of fsync() calls from dpkg.
> 
> In the case of building a livecd, this can be the difference between
> a 10 minute apt invocation, and 21 minutes.
> 
> The use cases where --force-unsafe-io would make sense are those
> where apt is modifiying an underlying (possibly empty) base chroot
> and the results will be thrown away before the next apt invocation
> such that a crash just means "start from the known good base".  The
> two that jump to mind are:
>   1. livecd builds
>   2. buildd chroots where we are using lvm snapshots (or pbuilder or
>      whatever) and will be discarding the resulting chroot upon either
>      completion or machine reboot).
> 
> Obviously, we want the default to be safe, and the option name to be
> scary enough that end users don't think it's a good option to use in
> daily life.  I spoke with Colin Watson, and he suggested the option
> be called --force-unsafe-io.

Has there been any progress on this?

The problem is that not only dpkg takes longer, but there might be real world 
costs involved due to this uber-paranoid dpkg behaviour. I've been installing 
squeeze to microsd card (on guruplug) and debian installer has tortured 
(literally) my card for 45+ minutes at the stage of unpacking base system. I'm 
really concerned that at such a pace I will have to throw away my card very 
soon due to wear level.

It does not make much sense for dpkg to be in this uber-paranoid mode at 
debian-installer time. If power fails, install process will probably have to 
be started from scratch anyway. What's more, obviously I have no choice to use 
libeatmydata or similar to fight this dpkg behaviour at debian installer time.

In my opinion, dpkg should provide a way to turn off those offensive *sync() 
calls and debian installer should make use of it.

-- 
Modestas Vainius <modestas@vainius.eu>

[signature.asc (application/pgp-signature, inline)]

Message #32 received at 584254@bugs.debian.org (full text, mbox, reply):

From: Otavio Salvador <otavio@ossystems.com.br>

To: Modestas Vainius <modax@debian.org>

Cc: LaMont Jones <lamont@debian.org>, 584254@bugs.debian.org, Guillem Jover <guillem@debian.org>, debian-boot@lists.debian.org

Subject: Re: dpkg should support a --force-unsafe-io option or such

Date: Sat, 9 Oct 2010 19:06:41 -0300

Hello,

On Fri, Oct 8, 2010 at 8:55 PM, Modestas Vainius <modax@debian.org> wrote:
...
> It does not make much sense for dpkg to be in this uber-paranoid mode at
> debian-installer time. If power fails, install process will probably have to
> be started from scratch anyway. What's more, obviously I have no choice to use
> libeatmydata or similar to fight this dpkg behaviour at debian installer time.
>
> In my opinion, dpkg should provide a way to turn off those offensive *sync()
> calls and debian installer should make use of it.

I fully agree that d-i doesn't need and shouldn't use it by default.
In fact some embedded systems shouldn't use it either in production
mode if the media suffer of wearing.

As previously said in this thread, d-i needs to be restarted in case
of power outage or something broken and fsync won't help on that
except make it taking longer.

-- 
Otavio Salvador                  O.S. Systems
E-mail: otavio@ossystems.com.br  http://www.ossystems.com.br
Mobile: +55 53 9981-7854         http://projetos.ossystems.com.br

Message #37 received at 584254@bugs.debian.org (full text, mbox, reply):

From: Modestas Vainius <modax@debian.org>

To: Otavio Salvador <otavio@ossystems.com.br>

Cc: LaMont Jones <lamont@debian.org>, 584254@bugs.debian.org, Guillem Jover <guillem@debian.org>, debian-boot@lists.debian.org

Subject: Re: dpkg should support a --force-unsafe-io option or such

Date: Sun, 10 Oct 2010 12:37:27 +0300

[Message part 1 (text/plain, inline)]

Hello,

On sekmadienis 10 Spalis 2010 01:06:41 Otavio Salvador wrote:
> On Fri, Oct 8, 2010 at 8:55 PM, Modestas Vainius <modax@debian.org> wrote:
> ...
> 
> > It does not make much sense for dpkg to be in this uber-paranoid mode at
> > debian-installer time. If power fails, install process will probably have
> > to be started from scratch anyway. What's more, obviously I have no
> > choice to use libeatmydata or similar to fight this dpkg behaviour at
> > debian installer time.
> > 
> > In my opinion, dpkg should provide a way to turn off those offensive
> > *sync() calls and debian installer should make use of it.
> 
> I fully agree that d-i doesn't need and shouldn't use it by default.
> In fact some embedded systems shouldn't use it either in production
> mode if the media suffer of wearing.
> 
> As previously said in this thread, d-i needs to be restarted in case
> of power outage or something broken and fsync won't help on that
> except make it taking longer.

I feel rather strong about this issue because I keep running and running into 
it in different situations. I truly believe that it should be high priority 
for squeeze but I won't bump severity myself. To sum up:

1) current dpkg is bog slow on modern file systems (brtfs, ext4). What is 
more, due to frequent sync() calls, it is a very bad idea to run dpkg when 
other unrelated disk I/O (esp. on modern FSes) is in the background. It just 
slows everything down.

2) current dpkg is arguably not suitable for flash media (i.e. embedded 
devices). This hurts the "universal" part of Debian OS.

3) dpkg is pointlessly slow in such use cases as buildds where *sync() is not 
important at all.

4) while typical dpkg could be work arounded with libeatmydata, there is no 
cure for debian-installer.

And all this is to protect against power failure? Sure, the purpose is noble 
and maybe it's a good default but due to negative side effects I find it 
unacceptable that this behaviour is not configurable.

-- 
Modestas Vainius <modax@debian.org>

[signature.asc (application/pgp-signature, inline)]

Message #42 received at 584254@bugs.debian.org (full text, mbox, reply):

From: Guillem Jover <guillem@debian.org>

To: Modestas Vainius <modax@debian.org>, 584254@bugs.debian.org

Cc: Otavio Salvador <otavio@ossystems.com.br>, LaMont Jones <lamont@debian.org>, debian-boot@lists.debian.org

Subject: Re: Bug#584254: dpkg should support a --force-unsafe-io option or such

Date: Sun, 10 Oct 2010 14:21:47 +0200

Hi!

On Sun, 2010-10-10 at 12:37:27 +0300, Modestas Vainius wrote:
> On sekmadienis 10 Spalis 2010 01:06:41 Otavio Salvador wrote:
> > On Fri, Oct 8, 2010 at 8:55 PM, Modestas Vainius <modax@debian.org> wrote:
> > > It does not make much sense for dpkg to be in this uber-paranoid mode at
> > > debian-installer time. If power fails, install process will probably have
> > > to be started from scratch anyway. What's more, obviously I have no
> > > choice to use libeatmydata or similar to fight this dpkg behaviour at
> > > debian installer time.
> > > 
> > > In my opinion, dpkg should provide a way to turn off those offensive
> > > *sync() calls and debian installer should make use of it.
> > 
> > I fully agree that d-i doesn't need and shouldn't use it by default.
> > In fact some embedded systems shouldn't use it either in production
> > mode if the media suffer of wearing.
> > 
> > As previously said in this thread, d-i needs to be restarted in case
> > of power outage or something broken and fsync won't help on that
> > except make it taking longer.
> 
> I feel rather strong about this issue because I keep running and running into 
> it in different situations. I truly believe that it should be high priority 
> for squeeze but I won't bump severity myself. To sum up:

I had planned to include changes for this for squeeze, but then the
sudden freeze happened, and after the initial reaction from the
release team on the first exception request I didn't feel like begging
for this and other changes.

> 1) current dpkg is bog slow on modern file systems (brtfs, ext4). What is 
> more, due to frequent sync() calls, it is a very bad idea to run dpkg when 
> other unrelated disk I/O (esp. on modern FSes) is in the background. It just 
> slows everything down.

Right, for some time now I've considered the switch from fsync() to
sync() (on Linux) to be a mistake, as it will affect unrelated file
systems, which might be disruptive in case there's background work being
done.

But this was done because those "modern file systems" didn't perform
adequately with fsync(), and they are the ones which require the syncs
or they will actually lose data.

> 3) dpkg is pointlessly slow in such use cases as buildds where *sync() is not 
> important at all.

Well, even if the buildd chroot supposedly should be able to be recreated
easily, if the zero-lenght file issues appear on it, then it might not
be obvious something is wrong, and might produce bogus packages.

> 2) current dpkg is arguably not suitable for flash media (i.e. embedded 
> devices). This hurts the "universal" part of Debian OS.

> 4) while typical dpkg could be work arounded with libeatmydata, there is no 
> cure for debian-installer.

Sure, I agree the user should be able to disable this, at their own
risk. Or on specific cases, like on d-i.

> And all this is to protect against power failure? Sure, the purpose is noble 
> and maybe it's a good default but due to negative side effects I find it 
> unacceptable that this behaviour is not configurable.

Not only power failures, any abrupt system crash, say the bus getting
locked up, or a user assisted reboot, can produce for example
zero-lenght files on at least ext4 file systems, I don't know about
btrfs. The worst though is that the performance issues affect the file
systems which really need those safety measures. Also take into account
these are not anecdotal cases, Ubuntu who has offered ext4 on installation
for some time now had *hundreds* of duped bug reports on broken systems
due to this.

Anyway, I actually had the changes around last July, and I'll run them
through the release team to see what they say. I've rebased them now,
and will polish them a bit, but they are essentially these:

  <http://git.hadrons.org/?p=debian/dpkg/dpkg.git;a=commitdiff;h=87740373>
  <http://git.hadrons.org/?p=debian/dpkg/dpkg.git;a=commitdiff;h=0484cd7d>

regards,
guillem

Message #47 received at 584254@bugs.debian.org (full text, mbox, reply):

From: Goswin von Brederlow <goswin-v-b@web.de>

To: Guillem Jover <guillem@debian.org>

Cc: Modestas Vainius <modax@debian.org>, 584254@bugs.debian.org, Otavio Salvador <otavio@ossystems.com.br>, LaMont Jones <lamont@debian.org>, debian-boot@lists.debian.org

Subject: Re: Bug#584254: dpkg should support a --force-unsafe-io option or such

Date: Wed, 13 Oct 2010 10:56:39 +0200

Guillem Jover <guillem@debian.org> writes:

>> 3) dpkg is pointlessly slow in such use cases as buildds where *sync() is not 
>> important at all.
>
> Well, even if the buildd chroot supposedly should be able to be recreated
> easily, if the zero-lenght file issues appear on it, then it might not
> be obvious something is wrong, and might produce bogus packages.

Nah. If the buildd crashes while creating the chroot then it will just
have to remove and recreate the chroot when it comes back. In configs
where lvm snapshots, btrfs snapshots or cow hardlink farms are used this
is the default. And only in such chroots would you use the unsafe IO
option.

>> 2) current dpkg is arguably not suitable for flash media (i.e. embedded 
>> devices). This hurts the "universal" part of Debian OS.
>
>> 4) while typical dpkg could be work arounded with libeatmydata, there is no 
>> cure for debian-installer.
>
> Sure, I agree the user should be able to disable this, at their own
> risk. Or on specific cases, like on d-i.

Maybe in unsafe mode dpkg should create /var/lib/dpkg/unsafe, sync(), do
everything else unsafe, sync(), remove /var/lib/dpkg/unsafe. And when
/var/lib/dpkg/unsafe exists already then it should start SCREAMING its
head of. :)

MfG
        Goswin

Message #52 received at 584254@bugs.debian.org (full text, mbox, reply):

From: Modestas Vainius <modax@debian.org>

To: 584254@bugs.debian.org, Guillem Jover <guillem@debian.org>

Cc: Otavio Salvador <otavio@ossystems.com.br>, LaMont Jones <lamont@debian.org>, debian-boot@lists.debian.org

Subject: Re: Bug#584254: dpkg should support a --force-unsafe-io option or such

Date: Wed, 20 Oct 2010 22:20:31 +0300

[Message part 1 (text/plain, inline)]

Hello,

On sekmadienis 10 Spalis 2010 15:21:47 Guillem Jover wrote:
> Anyway, I actually had the changes around last July, and I'll run them
> through the release team to see what they say. I've rebased them now,
> and will polish them a bit, but they are essentially these:
> 
>   <http://git.hadrons.org/?p=debian/dpkg/dpkg.git;a=commitdiff;h=87740373>
>   <http://git.hadrons.org/?p=debian/dpkg/dpkg.git;a=commitdiff;h=0484cd7d>

But these do not cover fsync()s for /var/lib/dpkg/status and 
/var/lib/dpkg/updates/*, do they? Those are very frequent as well and slow the 
whole process down. I think you need to expand --force-unsafe-io scope.

Btw, how will I be able to enable --force-unsafe-io for dpkg when it's run 
under apt/aptitude? Maybe environment variable and/or /etc/dpkg/dpkg.cfg 
option is a better solution then?

-- 
Modestas Vainius <modax@debian.org>

[signature.asc (application/pgp-signature, inline)]

Message #57 received at 584254@bugs.debian.org (full text, mbox, reply):

From: Julien Cristau <jcristau@debian.org>

To: Modestas Vainius <modax@debian.org>

Cc: 584254@bugs.debian.org, Guillem Jover <guillem@debian.org>, Otavio Salvador <otavio@ossystems.com.br>, LaMont Jones <lamont@debian.org>, debian-boot@lists.debian.org

Subject: Re: Bug#584254: dpkg should support a --force-unsafe-io option or such

Date: Wed, 20 Oct 2010 21:33:34 +0200

[Message part 1 (text/plain, inline)]

On Wed, Oct 20, 2010 at 22:20:31 +0300, Modestas Vainius wrote:

> Btw, how will I be able to enable --force-unsafe-io for dpkg when it's run 
> under apt/aptitude? Maybe environment variable and/or /etc/dpkg/dpkg.cfg 
> option is a better solution then?
> 
-o DPkg::options="--force-unsafe-io"?

Cheers,
Julien

[signature.asc (application/pgp-signature, inline)]

Message #62 received at 584254@bugs.debian.org (full text, mbox, reply):

From: Modestas Vainius <modax@debian.org>

To: 584254@bugs.debian.org, Julien Cristau <jcristau@debian.org>

Cc: Guillem Jover <guillem@debian.org>, Otavio Salvador <otavio@ossystems.com.br>, LaMont Jones <lamont@debian.org>, debian-boot@lists.debian.org

Subject: Re: Bug#584254: dpkg should support a --force-unsafe-io option or such

Date: Wed, 20 Oct 2010 22:48:38 +0300

[Message part 1 (text/plain, inline)]

Hello,

On trečiadienis 20 Spalis 2010 22:33:34 Julien Cristau wrote:
> On Wed, Oct 20, 2010 at 22:20:31 +0300, Modestas Vainius wrote:
> > Btw, how will I be able to enable --force-unsafe-io for dpkg when it's
> > run under apt/aptitude? Maybe environment variable and/or
> > /etc/dpkg/dpkg.cfg option is a better solution then?
> 
> -o DPkg::options="--force-unsafe-io"?

DPkg::options does not seem to work with neither apt or aptitude.

-- 
Modestas Vainius <modax@debian.org>

[signature.asc (application/pgp-signature, inline)]

Message #67 received at 584254@bugs.debian.org (full text, mbox, reply):

From: Raphael Hertzog <hertzog@debian.org>

To: Modestas Vainius <modax@debian.org>, 584254@bugs.debian.org

Cc: Julien Cristau <jcristau@debian.org>, Guillem Jover <guillem@debian.org>, Otavio Salvador <otavio@ossystems.com.br>, LaMont Jones <lamont@debian.org>, debian-boot@lists.debian.org

Subject: Re: Bug#584254: dpkg should support a --force-unsafe-io option or such

Date: Wed, 20 Oct 2010 22:08:36 +0200

Hi,

On Wed, 20 Oct 2010, Modestas Vainius wrote:
> On trečiadienis 20 Spalis 2010 22:33:34 Julien Cristau wrote:
> > On Wed, Oct 20, 2010 at 22:20:31 +0300, Modestas Vainius wrote:
> > > Btw, how will I be able to enable --force-unsafe-io for dpkg when it's
> > > run under apt/aptitude? Maybe environment variable and/or
> > > /etc/dpkg/dpkg.cfg option is a better solution then?
> > 
> > -o DPkg::options="--force-unsafe-io"?
> 
> DPkg::options does not seem to work with neither apt or aptitude.

It does but it's “-o DPkg::options::="--force-unsafe-io"”.

And /etc/dpkg/dpkg.cfg can be used to enable any command-line parameter
so you should be able to put "force-unsafe-io" in that file as well.

Cheers,
-- 
Raphaël Hertzog ◈ Debian Developer ◈ [Flattr=20693]

Follow my Debian News ▶ http://RaphaelHertzog.com (English)
                      ▶ http://RaphaelHertzog.fr (Français)

Message #72 received at 584254@bugs.debian.org (full text, mbox, reply):

From: Modestas Vainius <modax@debian.org>

To: Raphael Hertzog <hertzog@debian.org>

Cc: 584254@bugs.debian.org, Julien Cristau <jcristau@debian.org>, Guillem Jover <guillem@debian.org>, Otavio Salvador <otavio@ossystems.com.br>, LaMont Jones <lamont@debian.org>, debian-boot@lists.debian.org

Subject: Re: Bug#584254: dpkg should support a --force-unsafe-io option or such

Date: Wed, 20 Oct 2010 23:26:02 +0300

[Message part 1 (text/plain, inline)]

Hello,

On trečiadienis 20 Spalis 2010 23:08:36 Raphael Hertzog wrote:
> It does but it's “-o DPkg::options::="--force-unsafe-io"”.

Indeed, this syntax works. Thanks.

> And /etc/dpkg/dpkg.cfg can be used to enable any command-line parameter
> so you should be able to put "force-unsafe-io" in that file as well.

Great. Thanks for this info.

-- 
Modestas Vainius <modax@debian.org>

[signature.asc (application/pgp-signature, inline)]

Message #77 received at 584254@bugs.debian.org (full text, mbox, reply):

From: Guillem Jover <guillem@debian.org>

To: Modestas Vainius <modax@debian.org>, 584254@bugs.debian.org

Subject: Re: Bug#584254: dpkg should support a --force-unsafe-io option or such

Date: Fri, 22 Oct 2010 11:12:05 +0200

On Wed, 2010-10-20 at 22:20:31 +0300, Modestas Vainius wrote:
> On sekmadienis 10 Spalis 2010 15:21:47 Guillem Jover wrote:
> > Anyway, I actually had the changes around last July, and I'll run them
> > through the release team to see what they say. I've rebased them now,
> > and will polish them a bit, but they are essentially these:
> > 
> >   <http://git.hadrons.org/?p=debian/dpkg/dpkg.git;a=commitdiff;h=87740373>
> >   <http://git.hadrons.org/?p=debian/dpkg/dpkg.git;a=commitdiff;h=0484cd7d>
> 
> But these do not cover fsync()s for /var/lib/dpkg/status and 
> /var/lib/dpkg/updates/*, do they? Those are very frequent as well and slow
> the whole process down. I think you need to expand --force-unsafe-io scope.

Those fsync() have always been there, thus it cannot be part of any
regression. I added few fsync()s for directories at the time, and for
few other db files but nothing that amounts to much.

The status file should be synced generally only once at the end of the
run or when enough updates have been accumulated (currently 250), and
each update file is fsync()ed once, those happen on state changes which
should not be many per package and are pretty small files.

If your file system is having problems keeping up with the db fsync()s
then I'd urge you to complain to the file system developers, as I'd
consider those buggy if they don't allow for the use of safe fsync() +
rename() when they actually require it or make users suffer data loss.

regards,
guillem

Message #82 received at 584254-submitter@bugs.debian.org (full text, mbox, reply):

From: Guillem Jover <guillem@debian.org>

To: 584254-submitter@bugs.debian.org

Subject: Bug#584254 marked as pending

Date: Thu, 25 Nov 2010 06:50:22 +0000

tag 584254 pending
thanks

Hello,

Bug #584254 reported by you has been fixed in the Git repository. You can
see the changelog below, and you can check the diff of the fix at:

    http://git.debian.org/?p=dpkg/dpkg.git;a=commitdiff;h=929a9c4

---
commit 929a9c4808c79781469987585f78f07df7f1d484
Author: Guillem Jover <guillem@debian.org>
Date:   Thu Jul 29 08:59:09 2010 +0200

    Add new --force-unsafe-io to disable safe I/O operations on unpack
    
    This allows to not perform file system syncs before file renames
    to guarantee its atomicity, which is known to cause substantial
    performance degradation on some file systems, unfortunately the ones
    that require the safe I/O on the first place due to their unreliable
    behaviour causing zero-length files on abrupt system crashes (sudden
    reboot, bus locks, pulling the plug, etc).
    
    Using this option might improve performance at the cost of losing
    data, and should thus be used with care, but that's ultimately
    something for the user of the affected file systems to decide.
    
    Closes: #584254

diff --git a/debian/changelog b/debian/changelog
index 4fcd521..9c97f0c 100644
--- a/debian/changelog
+++ b/debian/changelog
@@ -14,6 +14,8 @@ dpkg (1.15.8.6) UNRELEASED; urgency=low
   [ Guillem Jover ]
   * Disable by default usage of synchronous sync(2), as it causes undesired
     I/O on unrelated file systems. Closes: #588339, #595927, #600075
+  * Add new --force-unsafe-io to disable safe I/O operations on unpack.
+    Closes: #584254
 
   [ Updated man page translations ]
   * French (Christian Perrier). Including a typo fix

Message #87 received at 584254-close@bugs.debian.org (full text, mbox, reply):

From: Guillem Jover <guillem@debian.org>

To: 584254-close@bugs.debian.org

Subject: Bug#584254: fixed in dpkg 1.15.8.6

Date: Thu, 25 Nov 2010 07:02:13 +0000

Source: dpkg
Source-Version: 1.15.8.6

We believe that the bug you reported is fixed in the latest version of
dpkg, which is due to be installed in the Debian FTP archive:

dpkg-dev_1.15.8.6_all.deb
  to main/d/dpkg/dpkg-dev_1.15.8.6_all.deb
dpkg_1.15.8.6.dsc
  to main/d/dpkg/dpkg_1.15.8.6.dsc
dpkg_1.15.8.6.tar.bz2
  to main/d/dpkg/dpkg_1.15.8.6.tar.bz2
dpkg_1.15.8.6_amd64.deb
  to main/d/dpkg/dpkg_1.15.8.6_amd64.deb
dselect_1.15.8.6_amd64.deb
  to main/d/dpkg/dselect_1.15.8.6_amd64.deb
libdpkg-dev_1.15.8.6_amd64.deb
  to main/d/dpkg/libdpkg-dev_1.15.8.6_amd64.deb
libdpkg-perl_1.15.8.6_all.deb
  to main/d/dpkg/libdpkg-perl_1.15.8.6_all.deb



A summary of the changes between this version and the previous one is
attached.

Thank you for reporting the bug, which will now be closed.  If you
have further comments please address them to 584254@bugs.debian.org,
and the maintainer will reopen the bug report if appropriate.

Debian distribution maintenance software
pp.
Guillem Jover <guillem@debian.org> (supplier of updated dpkg package)

(This message was generated automatically at their request; if you
believe that there is a problem with it please contact the archive
administrators by mailing ftpmaster@debian.org)


-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Format: 1.8
Date: Thu, 25 Nov 2010 07:10:48 +0100
Source: dpkg
Binary: libdpkg-dev dpkg dpkg-dev libdpkg-perl dselect
Architecture: source amd64 all
Version: 1.15.8.6
Distribution: unstable
Urgency: low
Maintainer: Dpkg Developers <debian-dpkg@lists.debian.org>
Changed-By: Guillem Jover <guillem@debian.org>
Description: 
 dpkg       - Debian package management system
 dpkg-dev   - Debian package development tools
 dselect    - Debian package management front-end
 libdpkg-dev - Debian package management static library
 libdpkg-perl - Dpkg perl modules
Closes: 584254 588339 595455 595927 596168 596519 597023 597651 598473 599923 600075 600240 601852 602518 604769
Changes: 
 dpkg (1.15.8.6) unstable; urgency=low
 .
   [ Raphaël Hertzog ]
   * Ensure debian/source/local-options is always excluded from the source
     package even if the user provides customized -i or -I options.
     Closes: #597023
   * Fix Dpkg::Version's handling of version with a debian revision but an
     empty version (e.g. "-0.1"). Thanks to James Vega <jamessan@debian.org>
     for the patch. Closes: #597651
   * With "3.0 (quilt)" source package, create .pc/.quilt_series with the
     correct series file if the source package provides vendor specific patch
     sets.
 .
   [ Guillem Jover ]
   * Disable by default usage of synchronous sync(2), as it causes undesired
     I/O on unrelated file systems. Closes: #588339, #595927, #600075
   * Add new --force-unsafe-io to disable safe I/O operations on unpack.
     Closes: #584254
 .
   [ Updated man page translations ]
   * French (Christian Perrier). Including a typo fix and a typographical
     change reported by Vincent Danjean. Closes: #601852
   * Spanish (Omar Campagne). Closes: #596519
 .
   [ Updated programs translations ]
   * Basque (Iñaki Larrañaga Murgoitio). Closes: #599923
   * Catalan (Jordi Mallach).
   * Danish (Ask Hjorth Larsen). Closes: #600240
   * German (Sven Joachim). Improved by Holger Wansing.
   * Italian (Pietro Battiston). Fix translation of "however". Closes: #602518
   * Portuguese (Miguel Figueiredo). Closes: #596168
   * Romanian (Andrei Popescu). Closes: #604769
   * Russian (Yuri Kozlov). Closes: #595455
   * Vietnamese (Clytie Siddall). Closes: #598473
 .
   [ Updated scripts translations ]
   * Catalan (Jordi Mallach).
   * German (Sven Joachim).
 .
   [ Updated dselect translations ]
   * Catalan (Jordi Mallach).
   * German (Sven Joachim).
Checksums-Sha1: 
 0ac67a10e335d0ec5375ac523998546403525ff9 1208 dpkg_1.15.8.6.dsc
 ebc9a6087f8f8c56c973f26f9bdb17ef1c570f0c 5222815 dpkg_1.15.8.6.tar.bz2
 7a6e46450c5d8d89746a5ba16a6ed7b9306b2a07 426852 libdpkg-dev_1.15.8.6_amd64.deb
 ba234426fb283f292efc9a0b202a09b5c579b504 2338026 dpkg_1.15.8.6_amd64.deb
 196702535a3a609f134375d5d3c39bb26663f4ca 894426 dselect_1.15.8.6_amd64.deb
 e6affdd25dcdc0b9c73e5059bc95f34531ed52e4 801736 dpkg-dev_1.15.8.6_all.deb
 47cb193e3096afb8035632a1b6bae0da9771c226 682848 libdpkg-perl_1.15.8.6_all.deb
Checksums-Sha256: 
 a4355f87fa1466edcefff224182ae7824d1469d1ddc28126d00cd361c611e0a9 1208 dpkg_1.15.8.6.dsc
 b319621a4d0f9fa7b356b4def978bad0b18b944405f2be9eace5e2713b5f1f49 5222815 dpkg_1.15.8.6.tar.bz2
 eedd636f39cb03a28758558cd6f4b700a2168c1adb9e4c8a59ab4ca07549bad5 426852 libdpkg-dev_1.15.8.6_amd64.deb
 6d3265e9aa6ef2d7ec341687df80e55da5996daa5d581e1864e9466bc7c36321 2338026 dpkg_1.15.8.6_amd64.deb
 b4a073849a3944c4f7b13b67cc6db8a78e8edd38ca50d86134f7aa1977c40f4b 894426 dselect_1.15.8.6_amd64.deb
 3368e9efe1206c720b92c69b89f3f43e8a0a4d13949aa24deceb02b2452ef756 801736 dpkg-dev_1.15.8.6_all.deb
 6a05978ee576c2848aa0f748cd0208547c0f39856a782ba227db331b7cf24bd3 682848 libdpkg-perl_1.15.8.6_all.deb
Files: 
 6d69ce9fd47b97aef50f5ba1209c4b24 1208 admin required dpkg_1.15.8.6.dsc
 4102648a08a4416bfc3e4f4275a438e4 5222815 admin required dpkg_1.15.8.6.tar.bz2
 1c45d231edef121f36a575a6322e928b 426852 libdevel optional libdpkg-dev_1.15.8.6_amd64.deb
 4525021598d6810dcb3fa114e5433ba1 2338026 admin required dpkg_1.15.8.6_amd64.deb
 b5662cfacfb969c3db3668cab230729b 894426 admin optional dselect_1.15.8.6_amd64.deb
 85db60da1da45941fd1789d156bdbd01 801736 utils optional dpkg-dev_1.15.8.6_all.deb
 00816223970f7700379ef13d2110878b 682848 perl optional libdpkg-perl_1.15.8.6_all.deb

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.10 (GNU/Linux)

iEYEARECAAYFAkzuA9IACgkQuW9ciZ2SjJtWeACgw/Em7Z5+xaFqj6lJk3DpESzc
y/EAnA0oig96OBJD4pOI1fM62qXmzP00
=ugvO
-----END PGP SIGNATURE-----

Send a report that this bug log contains spam.