Debian Bug report logs -
#317815
/usr/bin/updatedb: sets TMPDIR to /var/tmp/ for sort's buffers
Reported by: Arthur Korn <arthur@debian.org>
Date: Mon, 11 Jul 2005 19:03:03 UTC
Severity: minor
Found in version 4.1.20-6
Done: Andreas Metzler <ametzler@debian.org>
Bug is archived. No further changes may be made.
Toggle useless messages
Report forwarded to debian-bugs-dist@lists.debian.org, Andreas Metzler <ametzler@debian.org>:
Bug#317815; Package findutils.
(full text, mbox, link).
Acknowledgement sent to Arthur Korn <arthur@wal.sourcepole>:
New Bug report received and forwarded. Copy sent to Andreas Metzler <ametzler@debian.org>.
(full text, mbox, link).
Message #5 received at submit@bugs.debian.org (full text, mbox, reply):
Package: findutils
Version: 4.1.20-6
Severity: minor
File: /usr/bin/updatedb
Hi
We had a problem today on one system where /var would be full every
night. We figured out that /var/tmp/ was full with hundreds of megabytes
of files with names like 'sortXb87kd'. Some inspection led me to the
assumption that those are buffer files from sort, which it places in
TMPDIR, which is set to /var/tmp/ here:
------/usr/bin/updatedb
# Directory to hold intermediate files.
if test -d /var/tmp; then
: ${TMPDIR=/var/tmp}
elif test -d /usr/tmp; then
: ${TMPDIR=/usr/tmp}
else
: ${TMPDIR=/tmp}
fi
export TMPDIR
--------
Since those are absolutely temporary files, and can grow quite large on
systems with many files, I suggest to prefer /tmp over /var/tmp or to
reset TMPDIR to /tmp when calling sort.
Overflowing /var is bad enough, but when it happens like this it can be
pretty hard to track down and will lead to bizarre malfunctions.
HTH, 2ri
-- System Information:
Debian Release: 3.1
Architecture: i386 (i686)
Kernel: Linux 2.4.26
Locale: LANG=de_CH, LC_CTYPE=de_CH (charmap=ISO-8859-1)
Versions of packages findutils depends on:
ii libc6 2.3.2.ds1-22 GNU C Library: Shared libraries an
-- no debconf information
Changed Bug submitter from Arthur Korn <arthur@wal.sourcepole> to Arthur Korn <arthur@debian.org>.
Request was from Arthur Korn <arthur@debian.org>
to control@bugs.debian.org.
(full text, mbox, link).
Message sent on to Arthur Korn <arthur@debian.org>:
Bug#317815.
(full text, mbox, link).
Message #10 received at 317815-submitter@bugs.debian.org (full text, mbox, reply):
URL:
<http://savannah.gnu.org/bugs/?func=detailitem&item_id=13774>
Summary: Why is updatedb prefering /var/tmp over /tmp for
temporary files?
Project: findutils
Submitted by: None
Submitted on: Tue 07/12/05 at 13:05
Category: updatedb
Severity: 3 - Normal
Item Group: None
Status: None
Privacy: Public
Assigned to: None
Originator Name: Arthur Korn
Originator Email: 317815-submitter@bugs.debian.org
Open/Closed: Open
Release: None
Fixed Release: None
_______________________________________________________
Details:
This was originally reported by Arthur Korn in http://bugs.debian.org/317815
updatedb by default uses /var/tmp for temporary files.
| We had a problem today on one system where /var would be full every
| night. We figured out that /var/tmp/ was full with hundreds of megabytes
| of files with names like 'sortXb87kd'. Some inspection led me to |
assumption that those are buffer files from sort, which it places | in TMPDIR,
which is set to /var/tmp/ here:
Afaict from reading FHS /var/tmp should only be used instead of /tmp if the
temporary files must not be deleted on reboot. I fail to see how this applies
to updatedb.
thanks, cu andreas
PS: Sorry, I have not taken my savannah pasword with me.
_______________________________________________________
Carbon-Copy List:
CC Address | Comment
------------------------------------+-----------------------------
317815-submitter --AT-- bugs --DOT-- debian --DOT-- org | Originator Email
_______________________________________________________
Reply to this item at:
<http://savannah.gnu.org/bugs/?func=detailitem&item_id=13774>
_______________________________________________
Message sent via/by Savannah
http://savannah.gnu.org/
Message sent on to Arthur Korn <arthur@debian.org>:
Bug#317815.
(full text, mbox, link).
Message #13 received at 317815-submitter@bugs.debian.org (full text, mbox, reply):
> This was originally reported by Arthur Korn in http://bugs.debian.org/317815
>
> updatedb by default uses /var/tmp for temporary files.
>
> | We had a problem today on one system where /var would be full every
> | night. We figured out that /var/tmp/ was full with hundreds of megabytes
> | of files with names like 'sortXb87kd'. Some inspection led me to |
> assumption that those are buffer files from sort, which it places |
> in TMPDIR, which is set to /var/tmp/ here:
>
> Afaict from reading FHS /var/tmp should only be used instead of /tmp if the
> temporary files must not be deleted on reboot. I fail to see how this applies
> to updatedb.
I can't speak as to findutils's original use of /var/tmp but in the
old days when disks were small and filesystems limited in size the
/tmp and /usr/tmp directories would frequently be on different disks.
Frequently the /usr disk was larger than the / disk. So /tmp was
limited in size for usable files and /usr/tmp available for possibly
much larger files. It was typical for applications that used large
disk space for temporary files to use /usr/tmp for that purpose so
that they would not be limited by the sometimes quite small /tmp
filesystem.
> | Since those are absolutely temporary files, and can grow quite large on
> | systems with many files, I suggest to prefer /tmp over /var/tmp or to
> | reset TMPDIR to /tmp when calling sort.
Since those files may be quite large it is probably necessary on many
systems to use /var/tmp over /tmp or they won't fit.
This is probably something that should be a system configurable
parameter. Probably something in /etc/default/findutils with the
default remaining /var/tmp.
> | Overflowing /var is bad enough, but when it happens like this it can be
> | pretty hard to track down and will lead to bizarre malfunctions.
The statement that it would lead to bizarre malfunctions confuses me
because the entire purpose of having a separate a filesystem is to
avoid breaking other parts of the system when a particular filesystem
fills up. By putting /var on a separate filesystem if something fills
it up it should not bring down the other use of the machine.
On server configurations /var is frequently on a separate filesystem
specifically to contain the disk space growth of it. Of course some
people do not like hard partitioned filesystems. One is always free
to create the entire system as one partition in that case.
Bob
Message sent on to Arthur Korn <arthur@debian.org>:
Bug#317815.
(full text, mbox, link).
Message #16 received at 317815-submitter@bugs.debian.org (full text, mbox, reply):
Follow-up Comment #1, bug #13774 (project findutils):
Traditionally /var/tmp has greater capacity, and is not memory-backed. GNU
sort uses a mixture of RAM and temporary files, spilling out to disk for
datasets larger than "--buffer-size". Therefore it's especially silly to
use /tmp for scratch files on systems where this is actually stored in VM
(e.g. Solaris). There have also been bug reports in the past caused by
people running out of space in /tmp (or /). Some systems have a relatively
vanilla root filesystem but vast numbers of files in other filsystems.
In short, /var/tmp is traditionally bigger and that can sometimes make the
difference between working and not working.
If the "sort" command is leaving temporary files behind, that might be a bug
in sort (unless perhaps it has been killed).
If you want sort to use a different subdirectory I suggest you set TMPDIR in
/etc/updatedb.conf. I will modify updatedb itself not to override TMPDIR if
it is already set.
_______________________________________________________
Reply to this item at:
<http://savannah.gnu.org/bugs/?func=detailitem&item_id=13774>
_______________________________________________
Message sent via/by Savannah
http://savannah.gnu.org/
Message sent on to Arthur Korn <arthur@debian.org>:
Bug#317815.
(full text, mbox, link).
Message #19 received at 317815-submitter@bugs.debian.org (full text, mbox, reply):
Update of bug #13774 (project findutils):
Severity: 3 - Normal => 2 - Minor
Status: None => In Progress
Assigned to: None => jay
Release: None => 4.2.23
_______________________________________________________
Reply to this item at:
<http://savannah.gnu.org/bugs/?func=detailitem&item_id=13774>
_______________________________________________
Message sent via/by Savannah
http://savannah.gnu.org/
Message sent on to Arthur Korn <arthur@debian.org>:
Bug#317815.
(full text, mbox, link).
Message #22 received at 317815-submitter@bugs.debian.org (full text, mbox, reply):
Follow-up Comment #2, bug #13774 (project findutils):
Jay wrote:
> If you want sort to use a different subdirectory I suggest you
> set TMPDIR in /etc/updatedb.conf. I will modify updatedb itself
> not to override TMPDIR if it is already set.
There is no change is necessary, this is already supported
cu andreas
_______________________________________________________
Reply to this item at:
<http://savannah.gnu.org/bugs/?func=detailitem&item_id=13774>
_______________________________________________
Message sent via/by Savannah
http://savannah.gnu.org/
Message sent on to Arthur Korn <arthur@debian.org>:
Bug#317815.
(full text, mbox, link).
Message #25 received at 317815-submitter@bugs.debian.org (full text, mbox, reply):
Follow-up Comment #3, bug #13774 (project findutils):
Ok, I see the wisdom in this. Maybe this wisdom could be integrated into the
FHS as well.
The combination of millions of hardlinks with long names created by
storebackup and a not _that_ large /var/ partition where numerous other tasks
where working in led to the problems.
Resolved here by setting TMPDIR to /tmp/ and adding the backups to PRUNEPATH
(storebackup has it's own find tool which works on the filelists which it
maintains anyway).
I consider this issue closed, thanks.
_______________________________________________________
Reply to this item at:
<http://savannah.gnu.org/bugs/?func=detailitem&item_id=13774>
_______________________________________________
Nachricht geschickt von/durch Savannah
http://savannah.gnu.org/
Message sent on to Arthur Korn <arthur@debian.org>:
Bug#317815.
(full text, mbox, link).
Message #28 received at 317815-submitter@bugs.debian.org (full text, mbox, reply):
Follow-up Comment #4, bug #13774 (project findutils):
> Therefore it's especially silly to use /tmp for scratch files
> on systems where this is actually stored in VM (e.g.
> Solaris). There have also been bug reports in the past caused
> by people running out of space in /tmp (or /). Some systems
> have a relatively vanilla root filesystem but vast numbers of
> files in other filsystems.
>
> In short, /var/tmp is traditionally bigger and that can
> sometimes make the difference between working and not
> working.
You are dictating policy here through updatedb though. This
should be up to the sysadmin. If the sysadmin wants to put /tmp
into VM that's up to him. And if he makes /tmp to small that's up to him as
well. It's the sysadmin who need to make reasonable decisions.
The updatedb behaveour however goes against the letter of the FHS:
http://www.pathname.com/fhs/pub/fhs-2.3.html#VARTMPTEMPORARYFILESPRESERVEDBETWEE
I'm not enough of a FHS expert to tell if it actually violates the standard.
Tomas Pospisek
_______________________________________________________
Reply to this item at:
<http://savannah.gnu.org/bugs/?func=detailitem&item_id=13774>
_______________________________________________
Message sent via/by Savannah
http://savannah.gnu.org/
Message sent on to Arthur Korn <arthur@debian.org>:
Bug#317815.
(full text, mbox, link).
Message #31 received at 317815-submitter@bugs.debian.org (full text, mbox, reply):
Follow-up Comment #5, bug #13774 (project findutils):
It doesn't look to me as if our preference for /var/tmp violates the FHS at
all.
I don't think that we're really dictating policy via updatedb. After all,
/var/tmp is a valid place to put temporary files, and this choice can be
overrridden in /etc/updatedb.conf (on Debian systems).
_______________________________________________________
Reply to this item at:
<http://savannah.gnu.org/bugs/?func=detailitem&item_id=13774>
_______________________________________________
Message sent via/by Savannah
http://savannah.gnu.org/
Message sent on to Arthur Korn <arthur@debian.org>:
Bug#317815.
(full text, mbox, link).
Message #34 received at 317815-submitter@bugs.debian.org (full text, mbox, reply):
Update of bug #13774 (project findutils):
Status: In Progress => Wont Fix
Open/Closed: Open => Closed
_______________________________________________________
Reply to this item at:
<http://savannah.gnu.org/bugs/?func=detailitem&item_id=13774>
_______________________________________________
Message sent via/by Savannah
http://savannah.gnu.org/
Reply sent to Andreas Metzler <ametzler@debian.org>:
You have taken responsibility.
(full text, mbox, link).
Notification sent to Arthur Korn <arthur@debian.org>:
Bug acknowledged by developer.
(full text, mbox, link).
Message #39 received at 317815-done@bugs.debian.org (full text, mbox, reply):
On 2005-07-13 anonymous <savannah-bounces@gnu.org> wrote:
> Follow-up Comment #3, bug #13774 (project findutils):
> Ok, I see the wisdom in this. Maybe this wisdom could be integrated into the
> FHS as well.
> The combination of millions of hardlinks with long names created by
> storebackup and a not _that_ large /var/ partition where numerous other tasks
> where working in led to the problems.
> Resolved here by setting TMPDIR to /tmp/ and adding the backups to PRUNEPATH
> (storebackup has it's own find tool which works on the filelists which it
> maintains anyway).
> I consider this issue closed, thanks.
Ok. Closing the debian bug, too.
cu andras
--
"See, I told you they'd listen to Reason," [SPOILER] Svfurlr fnlf,
fuhggvat qbja gur juveyvat tha.
Neal Stephenson in "Snow Crash"
http://downhill.aus.cc/
Bug archived.
Request was from Debbugs Internal Request <owner@bugs.debian.org>
to internal_control@bugs.debian.org.
(Sun, 17 Jun 2007 15:35:58 GMT) (full text, mbox, link).
Send a report that this bug log contains spam.
Debian bug tracking system administrator <owner@bugs.debian.org>.
Last modified:
Sat Dec 23 16:43:54 2023;
Machine Name:
bembo
Debian Bug tracking system
Debbugs is free software and licensed under the terms of the GNU
Public License version 2. The current version can be obtained
from https://bugs.debian.org/debbugs-source/.
Copyright © 1999 Darren O. Benham,
1997,2003 nCipher Corporation Ltd,
1994-97 Ian Jackson,
2005-2017 Don Armstrong, and many other contributors.