Debian Bug report logs - #759382
do not keep so much logs

Display info messages

Message #5 received at submit@bugs.debian.org (full text, mbox, reply):

From: Antoine Beaupré <anarcat@koumbit.org>

To: Debian Bug Tracking System <submit@bugs.debian.org>

Subject: do not keep so much logs

Date: Tue, 26 Aug 2014 14:28:48 -0700

Package: apache2
Severity: wishlist

Apache, at least in Wheezy, seems to be configured by default to keep 52
log files, rotated on a weekly basis, meaning that logs are kept for a
year.

This is a long time to keep longs. It exposes our users unduly to
surveillance and privacy breaches.

It also means a lot of data to keep on disk for busy webservers. For any
moderately to high traffic webserver, this can actually fill up /var
pretty fast. For example, a server with an average of 12 hits per
second:

http://stats.koumbit.net/koumbit.net/ceres.koumbit.net/apache_accesses.html

... accumulates around 30MB *per day*. That means 11GB per year. I
suspect the default partitionning would not allocate enough space for
/var at all on most systems to cover for that.

I would suggest following the policies set for /var/log/syslog, which
are rotate daily and keey 7 days.

-- System Information:
Debian Release: 7.6
  APT prefers stable
  APT policy: (500, 'stable')
Architecture: amd64 (x86_64)

Kernel: Linux 3.2.0-4-amd64 (SMP w/2 CPU cores)
Locale: LANG=fr_CA.UTF-8, LC_CTYPE=en_US.UTF-8 (charmap=UTF-8)
Shell: /bin/sh linked to /bin/dash

Versions of packages apache2 depends on:
pn  apache2-mpm-worker | apache2-mpm-prefork | apache2-mpm-event | apac  <none>
pn  apache2.2-common                                                     <none>

apache2 recommends no packages.

apache2 suggests no packages.

Message #12 received at 759382@bugs.debian.org (full text, mbox, reply):

From: Matt Taggart <taggart@debian.org>

To: 759382@bugs.debian.org

Cc: bradsmith@debian.org, skirpichev@gmail.com, ctaylor@debian.org, joostvb@debian.org, rfrancoise@debian.org

Subject: webalizer log needs

Date: Fri, 29 Aug 2014 17:34:03 -0700

Hi,

One of the arguments for keeping more apache logs is that web analysis
tools need them. Here is a survey of things I found in Debian and what
they might need. First some general comments:

* ideally the analyzer scans the log, generates stats, and then stores
the stats and doesn't need the log anymore. There still might be
sensitive data in the results, but the data stored is greatly reduced.
* even if logfile analyzers store results and are run often, we need
to consider the case of machines with power management that might mean
they are asleep at times. Hopefully anacron will help deal with that,
but we probably want to be conservative and make sure they have plenty
of opportunity to scan the logs before they are rotated.

analog
======
Has the ability to do incremental processing with a cache file
http://www.analog.cx/docs/cache.html
debian/rules sets CACHEDIR (is that sufficient?) (only DNS?)
Doesn't seem to run via cron.

awffull
=======
Fork of webalizer, see below.

awstats
=======
Gathers stats from logs every 10mins, updates html once a day.

goaccess
========
Interactive user tool. Doesn't appear to store results and so probably
needs the logs to be useful?

lire
====
Appears to use a database, has daily/weekly/monthly cronjobs. Need more
info about if it stores stats.

visitors
========
Doesn't appear to store stats, probably requires logs?

webalizer
=========
Keeps track of where it is in the log files, maintains a
an incremental history file, and runs via cron once a day.


For the things that do store data, I think 7 days should be enough to
ensure that they have a chance to process the logs before they get
rotated.

The above might be interesting for nginx log retention as well.

-- 
Matt Taggart
taggart@debian.org

Message #17 received at 759382@bugs.debian.org (full text, mbox, reply):

From: Stefan Fritsch <sf@sfritsch.de>

To: Matt Taggart <taggart@debian.org>, anarcat@koumbit.org

Cc: 759382@bugs.debian.org

Subject: Re: Bug#759382: webalizer log needs

Date: Sun, 21 Sep 2014 22:41:23 +0200

On Friday 29 August 2014 17:34:03, Matt Taggart wrote:
> For the things that do store data, I think 7 days should be enough
> to ensure that they have a chance to process the logs before they
> get rotated.

Thanks for the detailed analysis. Somehow 7 days seems awfully short 
to me when one has to analyse problems. What do you think about 14 
days?

Message #22 received at 759382@bugs.debian.org (full text, mbox, reply):

From: Matt Taggart <taggart@debian.org>

To: Stefan Fritsch <sf@sfritsch.de>

Cc: Matt Taggart <taggart@debian.org>, anarcat@koumbit.org, 759382@bugs.debian.org

Subject: Re: Bug#759382: webalizer log needs

Date: Sun, 21 Sep 2014 13:45:17 -0700

Stefan Fritsch writes:
> On Friday 29 August 2014 17:34:03, Matt Taggart wrote:
> > For the things that do store data, I think 7 days should be enough
> > to ensure that they have a chance to process the logs before they
> > get rotated.
> 
> Thanks for the detailed analysis. Somehow 7 days seems awfully short 
> to me when one has to analyse problems. What do you think about 14 
> days?

I think that sounds great. If we determine later that it's not needed then 
it could be turned down further.

Thanks,

-- 
Matt Taggart
taggart@debian.org

Message #27 received at 759382-close@bugs.debian.org (full text, mbox, reply):

From: Stefan Fritsch <sf@debian.org>

To: 759382-close@bugs.debian.org

Subject: Bug#759382: fixed in apache2 2.4.10-2

Date: Sun, 21 Sep 2014 21:20:29 +0000

Source: apache2
Source-Version: 2.4.10-2

We believe that the bug you reported is fixed in the latest version of
apache2, which is due to be installed in the Debian FTP archive.

A summary of the changes between this version and the previous one is
attached.

Thank you for reporting the bug, which will now be closed.  If you
have further comments please address them to 759382@bugs.debian.org,
and the maintainer will reopen the bug report if appropriate.

Debian distribution maintenance software
pp.
Stefan Fritsch <sf@debian.org> (supplier of updated apache2 package)

(This message was generated automatically at their request; if you
believe that there is a problem with it please contact the archive
administrators by mailing ftpmaster@ftp-master.debian.org)


-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256

Format: 1.8
Date: Sun, 21 Sep 2014 22:58:33 +0200
Source: apache2
Binary: apache2 apache2-data apache2-bin apache2-mpm-worker apache2-mpm-prefork apache2-mpm-event apache2-mpm-itk apache2.2-bin apache2.2-common libapache2-mod-proxy-html libapache2-mod-macro apache2-utils apache2-suexec apache2-suexec-pristine apache2-suexec-custom apache2-doc apache2-dev apache2-dbg
Architecture: source amd64 all
Version: 2.4.10-2
Distribution: unstable
Urgency: medium
Maintainer: Debian Apache Maintainers <debian-apache@lists.debian.org>
Changed-By: Stefan Fritsch <sf@debian.org>
Description:
 apache2    - Apache HTTP Server
 apache2-bin - Apache HTTP Server (binary files and modules)
 apache2-data - Apache HTTP Server (common files)
 apache2-dbg - Apache debugging symbols
 apache2-dev - Apache HTTP Server (development headers)
 apache2-doc - Apache HTTP Server (on-site documentation)
 apache2-mpm-event - transitional event MPM package for apache2
 apache2-mpm-itk - transitional itk MPM package for apache2
 apache2-mpm-prefork - transitional prefork MPM package for apache2
 apache2-mpm-worker - transitional worker MPM package for apache2
 apache2-suexec - transitional package for apache2-suexec-pristine
 apache2-suexec-custom - Apache HTTP Server configurable suexec program for mod_suexec
 apache2-suexec-pristine - Apache HTTP Server standard suexec program for mod_suexec
 apache2-utils - Apache HTTP Server (utility programs for web servers)
 apache2.2-bin - Transitional package for apache2-bin
 apache2.2-common - Transitional package for apache2
 libapache2-mod-macro - Transitional package for apache2-bin
 libapache2-mod-proxy-html - Transitional package for apache2-bin
Closes: 755976 756361 759382
Changes:
 apache2 (2.4.10-2) unstable; urgency=medium
 .
   * Pull changes from upstream 2.4.x branch up to r1626207
     + Security Fix for CVE-2013-5704: HTTP trailers could be used to
       replace HTTP headers late during request processing, potentially
       undoing or otherwise confusing modules that examined or modified
       request headers earlier.
       Adds "MergeTrailers" directive to restore legacy behavior.
 .
   * Switch to apache2 providing the httpd and httpd-cgi virtual packages.
     The previously providing apache2-bin package lacks the configuration
     files. Closes: #756361
   * Keep fewer logs by default. Instead of 52 weekly logs, keep 14 daily
     logs. The daily graceful restart also has the advantage of regenerating
     things like TLS session ticket keys more often. Closes: #759382
   * Clarify description of apache2 package. Closes: #755976
   * In the maintainer script helper, print out Apache's error message if
     the config check fails.
   * Re-add mod_ident. It has still at least one user. LP: #1333388
Checksums-Sha1:
 e8fc8c8509fb4e9fd861bccc305d58912ab9d3fc 3218 apache2_2.4.10-2.dsc
 ebcf1744d65f908ea699a0a7c29e0fffb2bf115c 507480 apache2_2.4.10-2.debian.tar.xz
 766be593978c61d7038f64235f81b4824d25456d 1510 libapache2-mod-proxy-html_2.4.10-2_amd64.deb
 a702e8dd57a589c9645394f2abc91616a6bc11e3 1498 libapache2-mod-macro_2.4.10-2_amd64.deb
 11038ddbaa5472a498d8e41f884c841086294871 201662 apache2_2.4.10-2_amd64.deb
 4492350f1bb433b27f17c26b642ca7df31865b9a 162450 apache2-data_2.4.10-2_all.deb
 26ff954212b37adb2b7eb985e1bdcbe2b77e8b4f 1020154 apache2-bin_2.4.10-2_amd64.deb
 989aef74cdcbc7c65dff7e5fa07917cbb64fa2e2 1512 apache2-mpm-worker_2.4.10-2_amd64.deb
 c079b1a48f81fe31109fd516527bffd2280bc168 1514 apache2-mpm-prefork_2.4.10-2_amd64.deb
 98b8e1b3de649ee6ff590703254ca7d59d8a5da8 1514 apache2-mpm-event_2.4.10-2_amd64.deb
 75e0b6f6b6cb109324b9d3aa6e2a7018cfaefa7e 1508 apache2-mpm-itk_2.4.10-2_amd64.deb
 a2f9b24620105d617c95268b45f40c0b75c6b655 1524 apache2.2-bin_2.4.10-2_amd64.deb
 f2d9b8cca673c010babbf8266542622c0f6c5910 121694 apache2.2-common_2.4.10-2_amd64.deb
 69fa92349aef0ca6f70d799afa2f84c233d7a78c 192446 apache2-utils_2.4.10-2_amd64.deb
 c417f64aa28367bf479919508eb5a02696cbdd72 1488 apache2-suexec_2.4.10-2_amd64.deb
 72c9e4f66c5ef99528e70102772041713a3ae987 127376 apache2-suexec-pristine_2.4.10-2_amd64.deb
 7dee3662810132d8cac00ae59132555e089791fd 128924 apache2-suexec-custom_2.4.10-2_amd64.deb
 c6d7d2104805ec81067c8b04789874123d87a415 2723582 apache2-doc_2.4.10-2_all.deb
 4167e4490989e34fe4a40b9230f4b4debdae8834 278486 apache2-dev_2.4.10-2_amd64.deb
 db0049259a986ce895aa3fe88643079de047605d 1691174 apache2-dbg_2.4.10-2_amd64.deb
Checksums-Sha256:
 bb823fa274e974eeff553c4d10937540fb0fab4ff65d04f1b8aefe8d2141b6c7 3218 apache2_2.4.10-2.dsc
 87e9f606b368fb1253c283dd0fe39df4b2424b282fb076d753fcbe2a565d6fd5 507480 apache2_2.4.10-2.debian.tar.xz
 40c29d77dc75e45477905976072ced7fed78835dcc684e58c2ca78a73c537b66 1510 libapache2-mod-proxy-html_2.4.10-2_amd64.deb
 6a65d3bcb3421fefc46e90ea605fdef7a5490fba87845a00f73a17058feb5045 1498 libapache2-mod-macro_2.4.10-2_amd64.deb
 3c2acbf1f478c2b50fc54d1aa296377e2f86d8dcc1f95cd29e6b7d3efe72defb 201662 apache2_2.4.10-2_amd64.deb
 87ef3dde6078f9bf3e36a3ef78fe332ed0411b20d4b2239ec05c48e5dce53cef 162450 apache2-data_2.4.10-2_all.deb
 78879ede4247abe15a9d5c5f18c0f79163072e333e367daa0940e298723027f0 1020154 apache2-bin_2.4.10-2_amd64.deb
 dba79ca8a2372ac992ad464ebd75ce1058430bdb3d2e8ab8cff04edf2b78dd4f 1512 apache2-mpm-worker_2.4.10-2_amd64.deb
 c630ed9c2f0267c94cd221984eebbf3e31d10c01e644c12876f98084654dee69 1514 apache2-mpm-prefork_2.4.10-2_amd64.deb
 5390b5f4725429857898c02e625cc477586355926164b1883097c4ab220c2f9c 1514 apache2-mpm-event_2.4.10-2_amd64.deb
 44e3e41fd9d86e3dbd103155b3a9eb3a1426515a278c87ad83449c67e4bd9425 1508 apache2-mpm-itk_2.4.10-2_amd64.deb
 6582b431469000e851e7cf902e3a3c88a38e20c4b19f0b8f8c26f0e0d7d78c79 1524 apache2.2-bin_2.4.10-2_amd64.deb
 2add641e8247a5ca7b6dc8d7e770b55d252e2a98d40f97f585aebd500efaba7b 121694 apache2.2-common_2.4.10-2_amd64.deb
 62c9190303e24be18eaea105836331a7f0bbf06e431d677cc84dc9c428662dc3 192446 apache2-utils_2.4.10-2_amd64.deb
 f6361aa51498e644a8b927275df1a4e142a9a2a73ce00d38c704b5a560c1dc91 1488 apache2-suexec_2.4.10-2_amd64.deb
 c375ca3e3a818aabfc16f5f9cf35394c33b32fc74e19a6cebdf1d7df2990b580 127376 apache2-suexec-pristine_2.4.10-2_amd64.deb
 03aad8a36cce5809cec533ab62fd6906a9a9a825b9c234a563974b760d11a2bd 128924 apache2-suexec-custom_2.4.10-2_amd64.deb
 8db673b1cba359add77543e8739e103f7a2112b2042ef7305bb9532453c1cb5c 2723582 apache2-doc_2.4.10-2_all.deb
 b916eadaffd143ef871f887b522fe53424328a04713b467789a2bcf0ae88ac0a 278486 apache2-dev_2.4.10-2_amd64.deb
 bf76c7e7ff5f336e7e966eb4d0e4a936b5fb0406087b08a829234743daa629aa 1691174 apache2-dbg_2.4.10-2_amd64.deb
Files:
 654185478e4c88d280c735813fdaaedc 1510 oldlibs extra libapache2-mod-proxy-html_2.4.10-2_amd64.deb
 fa217c6d163179a495d0f66566d66d93 1498 oldlibs extra libapache2-mod-macro_2.4.10-2_amd64.deb
 36ebb38612593beb7de5502f9bb360e9 201662 httpd optional apache2_2.4.10-2_amd64.deb
 ae0cf8bc3d1d10daf535e39327d59408 162450 httpd optional apache2-data_2.4.10-2_all.deb
 e5d3d84942924d193fb0c138c0f45927 1020154 httpd optional apache2-bin_2.4.10-2_amd64.deb
 78ca8d6a0939a94a511bc50c048a951f 1512 oldlibs extra apache2-mpm-worker_2.4.10-2_amd64.deb
 747e3119cf991abb5d52761021e85d89 1514 oldlibs extra apache2-mpm-prefork_2.4.10-2_amd64.deb
 a567ce5a1d125cea5996c31bd3024d4e 1514 oldlibs extra apache2-mpm-event_2.4.10-2_amd64.deb
 f239ab5c0ec1af0fc4b2cb5f70f4f3b4 1508 oldlibs extra apache2-mpm-itk_2.4.10-2_amd64.deb
 35e95c1f2e6737af3c489a118d4e5032 1524 oldlibs extra apache2.2-bin_2.4.10-2_amd64.deb
 bc1932b0c94b4b46ec38425e20fa9aaa 121694 oldlibs extra apache2.2-common_2.4.10-2_amd64.deb
 a5daa25d59f91068532c250965c2fe1d 192446 httpd optional apache2-utils_2.4.10-2_amd64.deb
 d3b46ca33f05536d335873fec87635f1 1488 oldlibs extra apache2-suexec_2.4.10-2_amd64.deb
 18f0f28d233939ec05428ba2a2570335 127376 httpd optional apache2-suexec-pristine_2.4.10-2_amd64.deb
 e7fd22c4f84396094488e16eb217c90e 128924 httpd extra apache2-suexec-custom_2.4.10-2_amd64.deb
 cc086c1507e9d45e9fcfad2cbb22984f 2723582 doc optional apache2-doc_2.4.10-2_all.deb
 6f21af8dbb37e585f60f41f31d1b50fc 278486 httpd optional apache2-dev_2.4.10-2_amd64.deb
 60ed5c2c6b05381bb512ab7fbac5610b 1691174 debug extra apache2-dbg_2.4.10-2_amd64.deb
 66d59ee1c3cc64c10ec6a3a1224549d6 3218 httpd optional apache2_2.4.10-2.dsc
 d8ca8a17ce567a2391f3aef76b96b2a3 507480 httpd optional apache2_2.4.10-2.debian.tar.xz

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1

iQIVAwUBVB8+ScaHXzVBzv3gAQg2JA/+JwYXWFGqhAMxVwpcakZbVqrYV2oIAvvS
CMyx7A+mNkrVT4iNYo2BH4A3UT+J5h8EhQzb+LLn8kmd5MRODuA3isVvQIBo2W5d
pywJnzYOfKIH4+1kFmgrF0yMPsBRRePlDcInYmWZNBM4nod2VbfDjVYKQ+xiIfhr
q0XQrop5EqYEraXZx1NipZxU4YroogQ+fVegj5uboZROUgAzcRsSVTYc+FNdwtEf
p85yOdhVDdspASF5ucjvCvMOPpQj7etmXG5tazuK1NsHjH4jCURyMi5lxU4/aZj5
zC5r774JwXjPYPDYJlgRL9Su9Wly2dTR4G6WtBcz5X3ZfUlh4tHQk7unaWqwRkoq
AvjxaE280vTY0ChlYT3LUe1jERKeuG+6ZnJvwZczr6lQiw8inhqX/VTz+jimmMjN
DolwCcrCvz8q3VoWhQ5EZLZ0FwsKh8e59rzpOuXBzgLFB1Wu6xKTYBIqFXeZqIv9
lFF76LKLImqVyp0jhCyNzE+sE5VnGXI2qkqLrQ/zaXGFR/4cL1asOHRoF4Zsj6tm
vnx3Ro3fvozfGX5tdLQrS2FjxduSwqNIvKKY8TJNLqW0RhPiDz1i0KjhAhRn0bgv
JPLv6QdlX4MQNKc0dDO6J81N831lt9k0261mp1k4ZGh4pP6uVHZ62D0lcVS6Jeoz
cdQ02pqY4MA=
=x6CL
-----END PGP SIGNATURE-----

Message #32 received at 759382@bugs.debian.org (full text, mbox, reply):

From: Vincent Lefevre <vincent@vinc17.net>

To: Antoine Beaupré <anarcat@koumbit.org>, 759382@bugs.debian.org

Subject: Re: Bug#759382: do not keep so much logs

Date: Mon, 22 Sep 2014 11:29:10 +0200

On 2014-08-26 14:28:48 -0700, Antoine Beaupré wrote:
> Apache, at least in Wheezy, seems to be configured by default to keep 52
> log files, rotated on a weekly basis, meaning that logs are kept for a
> year.
> 
> This is a long time to keep longs. It exposes our users unduly to
> surveillance and privacy breaches.

Not your users, but people who connect to the web server. But the
French law requires (required?) / advises to keep the logs for one
year. There's a discussion in French here:

  http://forum.ovh.com/archive/index.php/t-47594.html

Basically this is needed when:
  * Users can create contents.
  * In case of security breach, when someone can do bad things
    via Apache only.

> It also means a lot of data to keep on disk for busy webservers. For any
> moderately to high traffic webserver, this can actually fill up /var
> pretty fast. For example, a server with an average of 12 hits per
> second:
> 
> http://stats.koumbit.net/koumbit.net/ceres.koumbit.net/apache_accesses.html
> 
> ... accumulates around 30MB *per day*. That means 11GB per year.

Everyone says that disk space is cheap. So, this is a very poor
argument. Moreover old logs are compressed, so that it isn't 11GB
per year, but much smaller. With gzip compression (which is not
very good), I get more than a 10x compression. So, in practice,
30 MB per day would mean around 1 GB of disk space on the previous
default of one year, possibly less.

> I suspect the default partitionning would not allocate enough space
> for /var at all on most systems to cover for that.

By default, the Debian installer creates a single partition (unless
this has changed recently).

> I would suggest following the policies set for /var/log/syslog, which
> are rotate daily and keey 7 days.

Not everyone has such a busy webserver.

IMHO, the default log rotation should be changed back to 1 year,
at least to protect users in case of legal matters. Alternatively,
size-based log rotation could be used, e.g. with:

        rotate 15
        size 100M

-- 
Vincent Lefèvre <vincent@vinc17.net> - Web: <https://www.vinc17.net/>
100% accessible validated (X)HTML - Blog: <https://www.vinc17.net/blog/>
Work: CR INRIA - computer arithmetic / AriC project (LIP, ENS-Lyon)

Message #37 received at 759382@bugs.debian.org (full text, mbox, reply):

From: Antoine Beaupré <anarcat@koumbit.org>

To: Vincent Lefevre <vincent@vinc17.net>, 759382@bugs.debian.org

Subject: Re: Bug#759382: do not keep so much logs

Date: Mon, 22 Sep 2014 09:23:11 -0400

[Message part 1 (text/plain, inline)]

On 2014-09-22 05:29:10, Vincent Lefevre wrote:
> On 2014-08-26 14:28:48 -0700, Antoine Beaupré wrote:
>> Apache, at least in Wheezy, seems to be configured by default to keep 52
>> log files, rotated on a weekly basis, meaning that logs are kept for a
>> year.
>> 
>> This is a long time to keep longs. It exposes our users unduly to
>> surveillance and privacy breaches.
>
> Not your users, but people who connect to the web server. But the
> French law requires (required?) / advises to keep the logs for one
> year. There's a discussion in French here:
>
>   http://forum.ovh.com/archive/index.php/t-47594.html
>
> Basically this is needed when:
>   * Users can create contents.
>   * In case of security breach, when someone can do bad things
>     via Apache only.

Ouzbekistan law may also require providers to send their logs directly
to the state and install backdoors into their servers, are we going to
do that for all of Debian by default?

It is the provider's responsability to comply with the local laws, not
Debian's, as it is impossible to make a configuration that will work
with every local law.

>> It also means a lot of data to keep on disk for busy webservers. For any
>> moderately to high traffic webserver, this can actually fill up /var
>> pretty fast. For example, a server with an average of 12 hits per
>> second:
>> 
>> http://stats.koumbit.net/koumbit.net/ceres.koumbit.net/apache_accesses.html
>> 
>> ... accumulates around 30MB *per day*. That means 11GB per year.
>
> Everyone says that disk space is cheap.

I don't. Do you?

> So, this is a very poor argument. Moreover old logs are compressed, so
> that it isn't 11GB per year, but much smaller. With gzip compression
> (which is not very good), I get more than a 10x compression. So, in
> practice, 30 MB per day would mean around 1 GB of disk space on the
> previous default of one year, possibly less.

Those logs were compressed. I know about gzip compression, thank you,
and it is actually pretty decent for log files.

>> I suspect the default partitionning would not allocate enough space
>> for /var at all on most systems to cover for that.
>
> By default, the Debian installer creates a single partition (unless
> this has changed recently).

Ah. I am surprised by that, but I would assume that people would create
a separate /var on server installs. In our experience, we've had
webservers run out of space on /var a few times.

>> I would suggest following the policies set for /var/log/syslog, which
>> are rotate daily and keey 7 days.
>
> Not everyone has such a busy webserver.

Not everyone lives in a country that forces their providers to spy on
their users. Yet anyone can be a victim of massive visits on their
website (aka "slashdotting") which will basically fill up the drives,
regardless of the country they live in.

(Arguably, "slashdottings" themselves are a difficult problem to deal
with and may occur only within a day so log rotation will not help much,
but daily log rotation will certainly be better than weekly to deal with
this.)

We don't want to implement policies that make it difficult to run a
popular webserver - it's the whole point of those things anyways.

> IMHO, the default log rotation should be changed back to 1 year,
> at least to protect users in case of legal matters. Alternatively,
> size-based log rotation could be used, e.g. with:
>
>         rotate 15
>         size 100M

I think keeping logs does not protect users, it actually exposes them to
undue surveillance. When speaking of "users" here, I refer also to the
visitors of the website, which never agreed to install debian, choose
how much logs are kept and so on. We have a responsability towards those
as well. I prefer to refer to "operators" or "admins" for people that
have the power to do the above configuration.

Operators that want to comply with legal legislation should get a lawyer
and review their storage and compliance policy accordingly. Just keeping
"52 weeks" of logs is hardly legal compliance, even in france you need
to do more than this to comply with the authorities (for example there
are specific delays by which you need to answer).

Telling our operators that they can just install debian by default and
assume legal compliance worldwide is actually hurting them and exposing
them to undue legal complications.

Also, the above configuration, on small sites, could even mean keeping
logs even longer than the original configuration. On big sites, it will
not respect the legal requirements.

I would have proposed to keep no logs at all if I didn't know this would
have created a huge backlash. Furthermore, I think it makes sense to
keep a minimal amount of logs to help people understand what's happening
on a daily basis, so for me 14 days is a compromise. We'll still deploy
a 5 day retention policy at Koumbit, and I know of a lot of groups that
simply censor IPs in the logs altogether, with no ill legal effect or
managmeent problems.

Please keep the change in. It is a good compromise.

A.

-- 
A man is none the less a slave because he is allowed to choose a new
master once in a term of years.
                         - Lysander Spooner

[Message part 2 (application/pgp-signature, inline)]

Message #42 received at 759382@bugs.debian.org (full text, mbox, reply):

From: Vincent Lefevre <vincent@vinc17.net>

To: Antoine Beaupré <anarcat@koumbit.org>

Cc: 759382@bugs.debian.org

Subject: Re: Bug#759382: do not keep so much logs

Date: Mon, 22 Sep 2014 16:14:34 +0200

On 2014-09-22 09:23:11 -0400, Antoine Beaupré wrote:
> On 2014-09-22 05:29:10, Vincent Lefevre wrote:
> > Not your users, but people who connect to the web server. But the
> > French law requires (required?) / advises to keep the logs for one
> > year. There's a discussion in French here:
> >
> >   http://forum.ovh.com/archive/index.php/t-47594.html
> >
> > Basically this is needed when:
> >   * Users can create contents.
> >   * In case of security breach, when someone can do bad things
> >     via Apache only.
> 
> Ouzbekistan law may also require providers to send their logs directly
> to the state and install backdoors into their servers, are we going to
> do that for all of Debian by default?

I don't care about Ouzbekistan. In most countries, users are
responsible for what their servers do, and keeping logs is a
way to protect them.

Note also that Debian cares about local laws. Otherwise there
would be no problems with patented algorithms.

> > Everyone says that disk space is cheap.
> 
> I don't. Do you?

Debian devs do.

> Not everyone lives in a country that forces their providers to spy on
> their users.

Please could you avoid saying stupid things?

> Yet anyone can be a victim of massive visits on their website (aka
> "slashdotting") which will basically fill up the drives, regardless
> of the country they live in.

In such a case, size based rules would be better than date based ones.

> > IMHO, the default log rotation should be changed back to 1 year,
> > at least to protect users in case of legal matters. Alternatively,
> > size-based log rotation could be used, e.g. with:
> >
> >         rotate 15
> >         size 100M
> 
> I think keeping logs does not protect users,

By "users", I meant here the responsible of web servers.

> it actually exposes them to undue surveillance. When speaking of
> "users" here, I refer also to the visitors of the website, which
> never agreed to install debian, choose how much logs are kept and so
> on. We have a responsability towards those as well.

Wow! Most web servers keep logs for a long time by choice. Visitors
who do not agree with that should not use the web.

> Also, the above configuration, on small sites, could even mean keeping
> logs even longer than the original configuration.

Not a real problem.

> On big sites, it will not respect the legal requirements.

Admins of big sites will probably have a closer look at the config
anyway.

-- 
Vincent Lefèvre <vincent@vinc17.net> - Web: <https://www.vinc17.net/>
100% accessible validated (X)HTML - Blog: <https://www.vinc17.net/blog/>
Work: CR INRIA - computer arithmetic / AriC project (LIP, ENS-Lyon)

Message #47 received at 759382@bugs.debian.org (full text, mbox, reply):

From: Antoine Beaupré <anarcat@koumbit.org>

To: Vincent Lefevre <vincent@vinc17.net>

Cc: 759382@bugs.debian.org

Subject: Re: Bug#759382: do not keep so much logs

Date: Mon, 22 Sep 2014 10:23:05 -0400

[Message part 1 (text/plain, inline)]

On 2014-09-22 10:14:34, Vincent Lefevre wrote:
> On 2014-09-22 09:23:11 -0400, Antoine Beaupré wrote:
>> On 2014-09-22 05:29:10, Vincent Lefevre wrote:
>> > Not your users, but people who connect to the web server. But the
>> > French law requires (required?) / advises to keep the logs for one
>> > year. There's a discussion in French here:
>> >
>> >   http://forum.ovh.com/archive/index.php/t-47594.html
>> >
>> > Basically this is needed when:
>> >   * Users can create contents.
>> >   * In case of security breach, when someone can do bad things
>> >     via Apache only.
>> 
>> Ouzbekistan law may also require providers to send their logs directly
>> to the state and install backdoors into their servers, are we going to
>> do that for all of Debian by default?
>
> I don't care about Ouzbekistan. In most countries, users are
> responsible for what their servers do, and keeping logs is a
> way to protect them.

I care about Ouzbekistan the same way I care about France.

> Note also that Debian cares about local laws. Otherwise there
> would be no problems with patented algorithms.

Because the servers hosting the source code and binaries, not because of
installed systems so much.

>> > Everyone says that disk space is cheap.
>> 
>> I don't. Do you?
>
> Debian devs do.

I'm a debian dev.

>> Not everyone lives in a country that forces their providers to spy on
>> their users.
>
> Please could you avoid saying stupid things?

No, as they are not stupid. I would prefer it if you would refrain from
qualifying what I consider to be reasonable statements as "stupid". That
you disagree doesn't make them stupid.

I do believe that the european logging directives, for example, are a
way to force providers to spy on their users on the behalf of the
state. Other countries do not have such requirements and still have
other legal means of getting to the data they need for criminal
prosecution. Forcing providers to keep logs is a way to force
deanonymisation of our users on the network, and is a fundamental issue
with freedom of speech and association.

>> Yet anyone can be a victim of massive visits on their website (aka
>> "slashdotting") which will basically fill up the drives, regardless
>> of the country they live in.
>
> In such a case, size based rules would be better than date based ones.

It's not a one-sided issues, there are multiple arguments for reducing
the logs we keep, one of them is surveillance, another is disk usage.

>> > IMHO, the default log rotation should be changed back to 1 year,
>> > at least to protect users in case of legal matters. Alternatively,
>> > size-based log rotation could be used, e.g. with:
>> >
>> >         rotate 15
>> >         size 100M
>> 
>> I think keeping logs does not protect users,
>
> By "users", I meant here the responsible of web servers.
>
>> it actually exposes them to undue surveillance. When speaking of
>> "users" here, I refer also to the visitors of the website, which
>> never agreed to install debian, choose how much logs are kept and so
>> on. We have a responsability towards those as well.
>
> Wow! Most web servers keep logs for a long time by choice. Visitors
> who do not agree with that should not use the web.

Webservers that want to choose to keep logs for a long time can do
so. Admins that do not agree can change the policies, visitors cannot.

>> Also, the above configuration, on small sites, could even mean keeping
>> logs even longer than the original configuration.
>
> Not a real problem.

It's actually exactly the problem that was raised in this bug report in
the first place, and it's a real problem.

>> On big sites, it will not respect the legal requirements.
>
> Admins of big sites will probably have a closer look at the config
> anyway.

And we do. But by bringing our (legal) experience to a larger audience,
we hope to share our expertise and hard-learned lessons with everyone.

Besides, the change was done in the package, and unless you are one of
the maintainers, I am not the one you need to argue with, and I do not
need to convince you this is the right way.

A.
-- 
Lorsque l'on range des objets dans des tiroirs, et que l'on a plus
d'objets que de tiroirs, alors un tiroir au moins contient deux
objets.
                        - Lejeune-Dirichlet, Peter Gustav

[Message part 2 (application/pgp-signature, inline)]

Message #52 received at 759382@bugs.debian.org (full text, mbox, reply):

From: Vincent Lefevre <vincent@vinc17.net>

To: Antoine Beaupré <anarcat@koumbit.org>

Cc: 759382@bugs.debian.org

Subject: Re: Bug#759382: do not keep so much logs

Date: Mon, 22 Sep 2014 16:52:48 +0200

On 2014-09-22 10:23:05 -0400, Antoine Beaupré wrote:
> On 2014-09-22 10:14:34, Vincent Lefevre wrote:
> > On 2014-09-22 09:23:11 -0400, Antoine Beaupré wrote:
> >> On 2014-09-22 05:29:10, Vincent Lefevre wrote:
> >> > Not your users, but people who connect to the web server. But the
> >> > French law requires (required?) / advises to keep the logs for one
> >> > year. There's a discussion in French here:
> >> >
> >> >   http://forum.ovh.com/archive/index.php/t-47594.html
> >> >
> >> > Basically this is needed when:
> >> >   * Users can create contents.
> >> >   * In case of security breach, when someone can do bad things
> >> >     via Apache only.
> >> 
> >> Ouzbekistan law may also require providers to send their logs directly
> >> to the state and install backdoors into their servers, are we going to
> >> do that for all of Debian by default?
> >
> > I don't care about Ouzbekistan. In most countries, users are
> > responsible for what their servers do, and keeping logs is a
> > way to protect them.
> 
> I care about Ouzbekistan the same way I care about France.

I don't know where you live, but this is the same in most countries,
except that the period varies.

> >> > Everyone says that disk space is cheap.
> >> 
> >> I don't. Do you?
> >
> > Debian devs do.
> 
> I'm a debian dev.

You may be in the minority.

> >> Not everyone lives in a country that forces their providers to spy on
> >> their users.
> >
> > Please could you avoid saying stupid things?
> 
> No, as they are not stupid. I would prefer it if you would refrain from
> qualifying what I consider to be reasonable statements as "stupid". That
> you disagree doesn't make them stupid.

What you say is a lie. France does not force users to spy on other
users.

> I do believe that the european logging directives, for example, are a
> way to force providers to spy on their users on the behalf of the
> state. Other countries do not have such requirements and still have
> other legal means of getting to the data they need for criminal
> prosecution. Forcing providers to keep logs is a way to force
> deanonymisation of our users on the network, and is a fundamental issue
> with freedom of speech and association.

When someone connects to my web server, this is not my user.
This is someone (human or not) I don't know.

> > Wow! Most web servers keep logs for a long time by choice. Visitors
> > who do not agree with that should not use the web.
> 
> Webservers that want to choose to keep logs for a long time can do
> so.

And webservers that want to choose to keep logs for a short time
can do so. So, there was no reason to change the default period.

-- 
Vincent Lefèvre <vincent@vinc17.net> - Web: <https://www.vinc17.net/>
100% accessible validated (X)HTML - Blog: <https://www.vinc17.net/blog/>
Work: CR INRIA - computer arithmetic / AriC project (LIP, ENS-Lyon)

Message #57 received at 759382@bugs.debian.org (full text, mbox, reply):

From: Antoine Beaupré <anarcat@koumbit.org>

To: Vincent Lefevre <vincent@vinc17.net>

Cc: 759382@bugs.debian.org

Subject: Re: Bug#759382: do not keep so much logs

Date: Mon, 22 Sep 2014 11:13:22 -0400

[Message part 1 (text/plain, inline)]

On 2014-09-22 10:52:48, Vincent Lefevre wrote:
> I don't know where you live, but this is the same in most countries,
> except that the period varies.

Where I live is irrelevant. It is not the same in all countries: some
have more or less strict restrictions, some don't have any at all.

The United States of America, for example, do not enforce logging.

>> >> > Everyone says that disk space is cheap.
>> >> 
>> >> I don't. Do you?
>> >
>> > Debian devs do.
>> 
>> I'm a debian dev.
>
> You may be in the minority.

I may be.

It could also be because I am involved in buying the hardware at work.

>> >> Not everyone lives in a country that forces their providers to spy on
>> >> their users.
>> >
>> > Please could you avoid saying stupid things?
>> 
>> No, as they are not stupid. I would prefer it if you would refrain from
>> qualifying what I consider to be reasonable statements as "stupid". That
>> you disagree doesn't make them stupid.
>
> What you say is a lie. France does not force users to spy on other
> users.

I disagree. I think that forcing logging is forcing webserver operators
to surveil their users, in the ultimate goal of revealing their
activities to the authorities, and therefore spying.

Saying that this is a lie implies that I am deliberately construing
another reality and masking information I would know. That is incorrect:
we are in a disagreement about the purpose of those policies. You think
it's good, I think it's bad. It doesn't make me stupid or a liar.

>> I do believe that the european logging directives, for example, are a
>> way to force providers to spy on their users on the behalf of the
>> state. Other countries do not have such requirements and still have
>> other legal means of getting to the data they need for criminal
>> prosecution. Forcing providers to keep logs is a way to force
>> deanonymisation of our users on the network, and is a fundamental issue
>> with freedom of speech and association.
>
> When someone connects to my web server, this is not my user.
> This is someone (human or not) I don't know.

I disagree. I think the client of a webserver is a user of a webserver.

Aren't people visiting (say) github.com users of github? Isn't there a
"terms of service" policy at the bottom of every freaking website these
days that they assume you have read and that implies you are a user of
their services?

>> > Wow! Most web servers keep logs for a long time by choice. Visitors
>> > who do not agree with that should not use the web.
>> 
>> Webservers that want to choose to keep logs for a long time can do
>> so.
>
> And webservers that want to choose to keep logs for a short time
> can do so. So, there was no reason to change the default period.

I guess there was a compelling reason enough so that the default was
changed. I have given numerous reasons why globally, the default logging
should be reduced (resource usage, privacy, etc). You have given a
single reason why, locally (namely in France), the default should be 52
weeks, and haven't adressed the question as to what to do with
variations in those policies outside of France.

I do not see why we should keep 52 weeks of logs to satisfy the legal
requirements of just one country in the world, especially since that
change isn't sufficient to ensure compliance, and may break compliance
in other countries.

Anyways, I do not need to be CC'd further in your communications here, I
am not the maintainer of this package, I just reported this bug and I'm
not the one you need to convince.

A.
-- 
We have no friends but the mountains.
                        - Kurdish saying

[Message part 2 (application/pgp-signature, inline)]

Message #62 received at 759382@bugs.debian.org (full text, mbox, reply):

From: Antoine Beaupré <anarcat@koumbit.org>

To: Vincent Lefevre <vincent@vinc17.net>

Cc: 759382@bugs.debian.org

Subject: Re: Bug#759382: do not keep so much logs

Date: Mon, 22 Sep 2014 12:40:20 -0400

[Message part 1 (text/plain, inline)]

On 2014-09-22 10:52:48, Vincent Lefevre wrote:
> On 2014-09-22 10:23:05 -0400, Antoine Beaupré wrote:
>> On 2014-09-22 10:14:34, Vincent Lefevre wrote:
>> > On 2014-09-22 09:23:11 -0400, Antoine Beaupré wrote:
>> >> On 2014-09-22 05:29:10, Vincent Lefevre wrote:
>> >> > Not your users, but people who connect to the web server. But the
>> >> > French law requires (required?) / advises to keep the logs for one
>> >> > year. There's a discussion in French here:
>> >> >
>> >> >   http://forum.ovh.com/archive/index.php/t-47594.html
>> >> >
>> >> > Basically this is needed when:
>> >> >   * Users can create contents.
>> >> >   * In case of security breach, when someone can do bad things
>> >> >     via Apache only.
>> >> 
>> >> Ouzbekistan law may also require providers to send their logs directly
>> >> to the state and install backdoors into their servers, are we going to
>> >> do that for all of Debian by default?
>> >
>> > I don't care about Ouzbekistan. In most countries, users are
>> > responsible for what their servers do, and keeping logs is a
>> > way to protect them.
>> 
>> I care about Ouzbekistan the same way I care about France.
>
> I don't know where you live, but this is the same in most countries,
> except that the period varies.

After a little more research, here's an overview of the national data
retention policies in Europe:

http://wiki.vorratsdatenspeicherung.de/Overview_of_national_data_retention_policies

Data retention seems to have been ruled out as inconstitutional in
Germany, to show an example of how important it could be to keep minimal
logs.

According to people well versed in those legal intricaties, there are
currently no restrictions on web server logging. Most of the data
retention directives apply to ISPs (Internet Service Providers) and
require that organisations providing connectivity to the internet need
to keep track of which IP belongs to which customer. There are also
restrictions for phone service providers.

As far as I know, there are no legal requirements for web hosting
providers to keep logs in the european directive. I would be curious to
hear on which basis you claim that french law requires hosting providers
to keep logs at all. The forum post you refer to is vague at best, and
doesn't even seem to be an official position of OVH. It also mentions
that keeping *more* logs is illegal and that most accounts hosted on OVH
probably don't respect the law.

So please provide more references to back up your laim that "most
countries need data retention" if you want to make a proper point here.

Thanks,

A.

-- 
Man really attains the state of complete humanity when he produces,
without being forced by physical need to sell himself as a commodity.
                        - Ernesto "Che" Guevara

[Message part 2 (application/pgp-signature, inline)]

Message #67 received at 759382@bugs.debian.org (full text, mbox, reply):

From: ilf <ilf@zeromail.org>

To: 759382@bugs.debian.org

Subject: Re: Bug#759382: do not keep so much logs

Date: Mon, 22 Sep 2014 19:07:06 +0200

[Message part 1 (text/plain, inline)]

Antoine Beaupré:
> So please provide more references to back up your laim that "most 
> countries need data retention" if you want to make a proper point here.

I will gladly argue about this, but it seems to miss the point.

Debian shouldn't base its software on specific laws of specific 
countries. Debian should provide sane defaults, produced by the 
community, not obscure laws.

Snowden has shown, that if it exists, TLAs will abuse it. So efficient 
data protection is data minimization.

See also this event at DebConf:
https://summit.debconf.org/debconf14/meeting/70/quit-logging-or-data-minimization-in-debian/
http://debconf14-video.debian.net/video/264/quit-logging-or-data-minimization-in-debian

If you want (or need) to log longer that this two-week default: go 
ahead, just do it. But don't impose that on the other 99%.

-- 
ilf

Über 80 Millionen Deutsche benutzen keine Konsole. Klick dich nicht weg!
		-- Eine Initiative des Bundesamtes für Tastaturbenutzung

[signature.asc (application/pgp-signature, inline)]

Message #72 received at 759382@bugs.debian.org (full text, mbox, reply):

From: ilf <ilf@zeromail.org>

To: 759382@bugs.debian.org

Subject: Re: Bug#759382: do not keep so much logs

Date: Mon, 22 Sep 2014 19:50:48 +0200

[Message part 1 (text/plain, inline)]

ilf:
> Debian shouldn't base its software on specific laws of specific 
> countries. Debian should provide sane defaults, produced by the 
> community, not obscure laws.

BTW, the Apache Consortium was "awarded" with a BigBrotherAward back in 
2000: "for the lack of attention of issues of privacy in the default 
configuration file of the Apache web server."

https://www.bigbrotherawards.de/2000/.scene/

More (bad) translation (by me):

> In the default configuration, the Apache web server logs, among other 
> things, the IP address of the user agent and the name of the retrieved 
> web pages. This information is a violation of user privacy, since 
> relatively simple analysis tools can determine in retrospect, which 
> users accessed which pages, in which order and how long the pages were 
> viewed.
> Beyong these basic logging functions, the default configuration file 
> of the Apache web server provides  additional logging options, that 
> are very easy to switch on. Additional information such as the 
> "referer" (the previously visited page) or the used browser software 
> version can be logged.
> In the standard configuration, these logging functions are only 
> described from a technological point-of-view. There is no indication 
> that the use of these functions may violate the privacy of the user.
> The use of extensive logging in a standard software such as the Apache 
> web server must be fundamentally questioned. From an "information 
> hygiene" perspective, logs should only be kept if they are 
> indispensable for the technological operation of the device. The user 
> has to be informed clearly and unambiguously, before he uses the 
> service.
> The Apache web server especially deserves this nomination, because the 
> default configuration almost begs for the collection of completely 
> unnecessary data on a large. This results in globally distributed data 
> slime trails, about which the user is not informed, but provoke law 
> enforcement agencies to go ahead and request them.

14 years after this award and 15 months after Snowden: let's finally fix 
this.

-- 
ilf

Über 80 Millionen Deutsche benutzen keine Konsole. Klick dich nicht weg!
		-- Eine Initiative des Bundesamtes für Tastaturbenutzung

[signature.asc (application/pgp-signature, inline)]

Message #77 received at 759382@bugs.debian.org (full text, mbox, reply):

From: Vincent Lefevre <vincent@vinc17.net>

To: Antoine Beaupré <anarcat@koumbit.org>

Cc: 759382@bugs.debian.org

Subject: Re: Bug#759382: do not keep so much logs

Date: Tue, 23 Sep 2014 10:42:20 +0200

On 2014-09-22 11:13:22 -0400, Antoine Beaupré wrote:
> On 2014-09-22 10:52:48, Vincent Lefevre wrote:
> > I don't know where you live, but this is the same in most countries,
> > except that the period varies.
> 
> Where I live is irrelevant. It is not the same in all countries: some
> have more or less strict restrictions, some don't have any at all.
> 
> The United States of America, for example, do not enforce logging.

On http://forum.ovh.com/archive/index.php/t-47594.html someone said
6 months for the USA. Now I wonder whether one should believe him.
I still wonder whether you can't have any problem with the justice
if someone does something illegal with your machine (e.g. via apache),
and you just say that it's someone else so that your are not
responsible.

> > What you say is a lie. France does not force users to spy on other
> > users.
> 
> I disagree. I think that forcing logging is forcing webserver operators
> to surveil their users, in the ultimate goal of revealing their
> activities to the authorities, and therefore spying.

No, webserver operators are not forced to log. No-one has ever been
convicted just because they didn't keep logs. However if something
illegal is done, the operator can be taken as responsible if he can't
identify the user who did this, or at least if he hasn't kept some
trace to identify him. Users should know that some trace may be kept,
so that this isn't even spying at all (it would be spying if done by
a 3rd party, without the knowledge of the parties).

Now, if a country wants to spy, it doesn't need a law (see the NSA
in the USA, for instance).

> >> I do believe that the european logging directives, for example, are a
> >> way to force providers to spy on their users on the behalf of the
> >> state. Other countries do not have such requirements and still have
> >> other legal means of getting to the data they need for criminal
> >> prosecution. Forcing providers to keep logs is a way to force
> >> deanonymisation of our users on the network, and is a fundamental issue
> >> with freedom of speech and association.
> >
> > When someone connects to my web server, this is not my user.
> > This is someone (human or not) I don't know.
> 
> I disagree. I think the client of a webserver is a user of a webserver.

But it's not *your* user. It's not a user of Debian (the user doesn't
care what OS is at the other side).

> >> > Wow! Most web servers keep logs for a long time by choice. Visitors
> >> > who do not agree with that should not use the web.
> >> 
> >> Webservers that want to choose to keep logs for a long time can do
> >> so.
> >
> > And webservers that want to choose to keep logs for a short time
> > can do so. So, there was no reason to change the default period.
> 
> I guess there was a compelling reason enough so that the default was
> changed. I have given numerous reasons why globally, the default logging
> should be reduced (resource usage, privacy, etc). You have given a
> single reason why, locally (namely in France), the default should be 52
> weeks, and haven't adressed the question as to what to do with
> variations in those policies outside of France.

Actually, but this is related,I want to keep logs for a long period
(even more than 1 year), just to have some trace in case someone tries
to compromise my machine or do something else bad. Vulnerabilities can
be found months/years after they have been introduced. Two weeks is
not just enough, and the change in Debian has been silently enforced
(I saw it just because I diff some config files).

On 2014-09-22 12:40:20 -0400, Antoine Beaupré wrote:
> After a little more research, here's an overview of the national data
> retention policies in Europe:
> 
> http://wiki.vorratsdatenspeicherung.de/Overview_of_national_data_retention_policies
> 
> Data retention seems to have been ruled out as inconstitutional in
> Germany, to show an example of how important it could be to keep minimal
> logs.

This is different because it seems to apply to private communitation
from A to B, and information kept by a 3rd party (i.e. which is neither
A nor B).

> So please provide more references to back up your laim that "most
> countries need data retention" if you want to make a proper point here.

So, perhaps you should forget about what countries explicitly say,
but more focus on the responsibility of the end user who has a
webserver and needs trace if someone tries to do illegal things.

-- 
Vincent Lefèvre <vincent@vinc17.net> - Web: <https://www.vinc17.net/>
100% accessible validated (X)HTML - Blog: <https://www.vinc17.net/blog/>
Work: CR INRIA - computer arithmetic / AriC project (LIP, ENS-Lyon)

Message #82 received at 759382@bugs.debian.org (full text, mbox, reply):

From: ilf <ilf@zeromail.org>

To: 759382@bugs.debian.org

Subject: Re: Bug#759382: do not keep so much logs

Date: Tue, 23 Sep 2014 11:25:53 +0200

[Message part 1 (text/plain, inline)]

Vincent Lefevre:
> On http://forum.ovh.com/archive/index.php/t-47594.html someone said

A link to a five year old thread on some random webforum is not exactly 
a convincing argument. If you want to have a discussion on law, please 
link to the legal text of the law in its official and in effect form.

-- 
ilf

Über 80 Millionen Deutsche benutzen keine Konsole. Klick dich nicht weg!
		-- Eine Initiative des Bundesamtes für Tastaturbenutzung

[signature.asc (application/pgp-signature, inline)]

Message #87 received at 759382@bugs.debian.org (full text, mbox, reply):

From: Vincent Lefevre <vincent@vinc17.net>

To: ilf <ilf@zeromail.org>, 759382@bugs.debian.org

Subject: Re: Bug#759382: do not keep so much logs

Date: Tue, 23 Sep 2014 13:50:47 +0200

On 2014-09-23 11:25:53 +0200, ilf wrote:
> Vincent Lefevre:
> >On http://forum.ovh.com/archive/index.php/t-47594.html someone said
> 
> A link to a five year old thread on some random webforum is not exactly a
> convincing argument. If you want to have a discussion on law, please link to
> the legal text of the law in its official and in effect form.

The link above contains some other links. IIRC, some of them are
off-topic (such as anything related to private communications).
But there's this one:

  http://www.legifrance.gouv.fr/affichTexte.do?cidTexte=JORFTEXT000000801164&dateTexte=

about "prestataires techniques", i.e. someone who maintains a service
but doesn't want to be responsible for what is done with it. The law
provides him some protection, but on the other hand, he has some
obligations. The French law does *not* enforce these obligations,
and someone who maintains a web service is free to ignore them, but
in such a case, he no longer has such a protection.

One of these obligations is:

  Les personnes mentionnées aux 1 et 2 du I détiennent et conservent
  les données de nature à permettre l'identification de quiconque a
  contribué à la création du contenu ou de l'un des contenus des
  services dont elles sont prestataires.

Basically, it says that one needs to keep some logs that would allow
authorities to identify contributors. It doesn't say exactly what,
but I suppose that in absence of other data, the IP address and the
date/time must be kept (this will be used as a part of the
investigation). The duration isn't given in the law but in a "décret"
(and I don't have it here -- anyway it's obviously much more than two
weeks).

Note also that this law is about the creation of public contents.
Most of end-user web servers do not offer that. The important point
is that there may be very similar issues. For instance, an attacker
may compromise a web server and provide his own, illegal contents.
Using the logs may allow one to identify the attacker. Without any
log, the end user would be taken as responsible (or have a part of
responsibility).

IMHO, the default should protect end users who have the least
knowledge or do the most basic things. Many users don't have a public
web server, but they may have a web server installed and running
(sometimes automatically due to a dependency) because of various
services. For instance, I have ones to test my web site (and access it
locally), also used for sensord graphs. They are not publicized. Since
they don't contain private data and I want to have access to them from
various places, I haven't added specific restrictions (at least at the
web server level). I can check in the logs if some people try to do
anything bad with them... If in any case, due to some vulnerability,
someone compromises the server or uses it as a gateway to do illegal
things somewhere else, logs can really be useful.

Admins of more important web servers can take some more time to adjust
config files (such as log rotation), depending on there needs. But
really, for a private web server (or public with minimalist contents),
there should be good defaults.

Moreover logs can also be useful for tools like fail2ban, and it is
not clear whether such a change may affect such tools, at least in
every configuration.

-- 
Vincent Lefèvre <vincent@vinc17.net> - Web: <https://www.vinc17.net/>
100% accessible validated (X)HTML - Blog: <https://www.vinc17.net/blog/>
Work: CR INRIA - computer arithmetic / AriC project (LIP, ENS-Lyon)

Send a report that this bug log contains spam.