Debian Bug report logs - #570623
reprepro: please add multiple version management

Reply or subscribe to this bug.

Toggle useless messages

Message #5 received at submit@bugs.debian.org (full text, mbox, reply):

From: Jean-Christophe Dubacq <jcdubacq1@free.fr>

To: Debian Bug Tracking System <submit@bugs.debian.org>

Subject: reprepro: please add multiple version management

Date: Sat, 20 Feb 2010 11:10:00 +0100

Package: reprepro
Version: 4.0.2-1
Severity: wishlist

Currently, indices built by reprepro only show the latest version of
a package. I really would like to use reprepro to have an archive
with multiple versions of a package accessible (eg for rollback).

currently I do something like that after each addition to the
repository:
cd debian
apt-ftparchive generate conf/storage.conf
apt-ftparchive -c conf/releasestorage.conf release dists/storage > dists/storage/Release
if [ -f dists/storage/Release.gpg ]; then
  rm dists/storage/Release.gpg
fi
gpg --output dists/storage/Release.gpg -ba dists/storage/Release
cd ..

But it does not take only what is tracked in the 'storage' dist,
it takes everything in the pool.

It would work much better at reprepro level.

-- System Information:
Debian Release: squeeze/sid
  APT prefers unstable
  APT policy: (800, 'unstable'), (700, 'testing'), (500, 'stable'), (50, 'experimental')
Architecture: amd64 (x86_64)

Kernel: Linux 2.6.32-trunk-amd64 (SMP w/2 CPU cores)
Locale: LANG=fr_FR.UTF-8, LC_CTYPE=fr_FR.UTF-8 (charmap=UTF-8)
Shell: /bin/sh linked to /bin/bash

Versions of packages reprepro depends on:
ii  libarchive1             2.6.2-1          Single library to read/write tar, 
ii  libbz2-1.0              1.0.5-4          high-quality block-sorting file co
ii  libc6                   2.10.2-6         Embedded GNU C Library: Shared lib
ii  libdb4.8                4.8.26-1         Berkeley v4.8 Database Libraries [
ii  libgpg-error0           1.6-1            library for common error values an
ii  libgpgme11              1.2.0-1.2        GPGME - GnuPG Made Easy
ii  zlib1g                  1:1.2.3.4.dfsg-3 compression library - runtime

Versions of packages reprepro recommends:
ii  apt                           0.7.25.3   Advanced front-end for dpkg

Versions of packages reprepro suggests:
ii  gnupg-agent       2.0.14-1               GNU privacy guard - password agent
pn  inoticoming       <none>                 (no description available)
ii  lzma              4.43-14                Compression method of 7z format in
ii  xz-utils          4.999.9beta+20100212-1 XZ-format compression utilities

-- no debconf information

Message #10 received at 570623@bugs.debian.org (full text, mbox, reply):

From: "Bernhard R. Link" <brlink@debian.org>

To: Jean-Christophe Dubacq <jcdubacq1@free.fr>

Cc: 570623@bugs.debian.org

Subject: Re: Bug#570623: reprepro: please add multiple version management

Date: Sat, 20 Feb 2010 16:11:52 +0100

* Jean-Christophe Dubacq <jcdubacq1@free.fr> [100220 11:14]:
> Currently, indices built by reprepro only show the latest version of
> a package. I really would like to use reprepro to have an archive
> with multiple versions of a package accessible (eg for rollback).

Sorry, that is not coming soon. Having one version of a package per
distribution/architecture/component/type quadruple is heavily built into
the database layout and all the internal logic.

If you want different versions of the same package in the same
architecture with reprepro, you need to either store them in different
distributions, or you can create snapshots of distributions.
(You can also enable tracking with the options to not remove old
packages, but that will only keep the files in the pool and not in any
apt-getable index).

Hochachtungsvoll,
	Bernhard R. Link

Message #15 received at 570623@bugs.debian.org (full text, mbox, reply):

From: Benjamin Drung <benjamin.drung@profitbricks.com>

To: 570623@bugs.debian.org

Cc: Jean-Christophe Dubacq <jcdubacq1@free.fr>

Subject: reprepro: please add multiple version management

Date: Wed, 08 Jan 2014 15:47:56 +0100

Hi,

I am in the process of making reprepro support multiple packages per
distribution/architecture/component/type quadruple. This requires bigger
changes in the database layout and all internal logic. I have a working
prototype (for including binary packages). I plan to extent this patch
to get it in an upstream acceptable state. I like to have guidance how
to implement some things before implementing them to avoid a result that
will be rejected.

The first step is to agree on the database layout change. I came up with
two alternatives:

1) Allow duplicate entries in packages.db and sort duplicate entries by
their Debian version. They can be sorted a) upwards or b) downwards.
Depending on the request, we will either search for all versions of a
package, one specific version of the package, or for the latest version
of a package.

2) Rename the key of packages.db to also contain the version of the
package, e.g. "sl|3.03-17" or "hello_2.8-4" (which delimiter should we
use?). This would allow us to check directly for a specific version of a
package. We need to add a secondary table that allows us to access the
database as described in 1) through the secondary table. This secondary
table will allow duplicate entries and the values of the secondary table
point to the key in packages.db. Depending on the task, we either query
the first or secondary table. The secondary table will be kept in sync
by BerkeleyDB.

In the first case, we need to add a function to iterate over the
duplicate packages to find a specific version. In the second case, we
need to create the secondary table and transform the database.

Which layout do you prefer?

Another issue is the sorting of the packages in the database. We need
one function to sort all entries in the table. So we need one function
to sort binary packages and source packages, but we have
binaries_getversion() and source_getversion(). Here's the example code
(without the error handling) of the sorting function:

static int debianversioncompare(UNUSED(DB *db), const DBT *a, const DBT *b) {
    char *a_version, *b_version;
    int versioncmp;

    binaries_getversion(a->data, &a_version);
    binaries_getversion(b->data, &b_version);
    dpkgversions_cmp(a_version, b_version, &versioncmp);
    return versioncmp;
}

Do you have a suggestion how to improve this function?

Cheers,
Benjamin

Message #20 received at 570623@bugs.debian.org (full text, mbox, reply):

From: "Bernhard R. Link" <brlink@debian.org>

To: Benjamin Drung <benjamin.drung@profitbricks.com>

Cc: 570623@bugs.debian.org

Subject: Re: Bug#570623: reprepro: please add multiple version management

Date: Sun, 2 Feb 2014 14:06:50 +0100

* Benjamin Drung <benjamin.drung@profitbricks.com> [140108 15:51]:

First sorry for my late reply. I must have totally missed your mail.

> The first step is to agree on the database layout change. I came up with
> two alternatives:
>
> 1) Allow duplicate entries in packages.db and sort duplicate entries by
> their Debian version. They can be sorted a) upwards or b) downwards.
> Depending on the request, we will either search for all versions of a
> package, one specific version of the package, or for the latest version
> of a package.
>
> 2) Rename the key of packages.db to also contain the version of the
> package, e.g. "sl|3.03-17" or "hello_2.8-4" (which delimiter should we
> use?). This would allow us to check directly for a specific version of a
> package. We need to add a secondary table that allows us to access the
> database as described in 1) through the secondary table. This secondary
> table will allow duplicate entries and the values of the secondary table
> point to the key in packages.db. Depending on the task, we either query
> the first or secondary table. The secondary table will be kept in sync
> by BerkeleyDB.
>
> In the first case, we need to add a function to iterate over the
> duplicate packages to find a specific version. In the second case, we
> need to create the secondary table and transform the database.
>
> Which layout do you prefer?

I think that layout is better that better fits the code. Not yet having
looked at the code, I cannot say. I guess 1 might be simpler. In the
case of 2 I think "|" is fine, as it is already used elsewhere (though
I guess one should make sure reprepro does not allow | in package
names).

> Another issue is the sorting of the packages in the database. We need
> one function to sort all entries in the table. So we need one function
> to sort binary packages and source packages, but we have
> binaries_getversion() and source_getversion(). Here's the example code
> (without the error handling) of the sorting function:
> 
> static int debianversioncompare(UNUSED(DB *db), const DBT *a, const DBT *b) {
>     char *a_version, *b_version;
>     int versioncmp;
> 
>     binaries_getversion(a->data, &a_version);
>     binaries_getversion(b->data, &b_version);
>     dpkgversions_cmp(a_version, b_version, &versioncmp);
>     return versioncmp;
> }
> 
> Do you have a suggestion how to improve this function?

It sounds quite slow either way. Perhaps the way to go is instead
changing the data format, like having the version first (perhaps even in
preparsed format to speed things up).


	Bernhard R. Link
-- 
F8AC 04D5 0B9B 064B 3383  C3DA AFFC 96D1 151D FFDC

Message #25 received at 570623@bugs.debian.org (full text, mbox, reply):

From: Benjamin Drung <benjamin.drung@profitbricks.com>

To: 570623@bugs.debian.org

Subject: Re: Bug#570623: reprepro: please add multiple version management

Date: Mon, 03 Feb 2014 13:10:30 +0100

[Message part 1 (text/plain, inline)]

On So, 2014-02-02 at 14:06 +0100, Bernhard R. Link wrote:
> * Benjamin Drung <benjamin.drung@profitbricks.com> [140108 15:51]:
> 
> First sorry for my late reply. I must have totally missed your mail.

No problem. I have been kept busy with other projects. ;)

> > The first step is to agree on the database layout change. I came up with
> > two alternatives:
> >
> > 1) Allow duplicate entries in packages.db and sort duplicate entries by
> > their Debian version. They can be sorted a) upwards or b) downwards.
> > Depending on the request, we will either search for all versions of a
> > package, one specific version of the package, or for the latest version
> > of a package.
> >
> > 2) Rename the key of packages.db to also contain the version of the
> > package, e.g. "sl|3.03-17" or "hello_2.8-4" (which delimiter should we
> > use?). This would allow us to check directly for a specific version of a
> > package. We need to add a secondary table that allows us to access the
> > database as described in 1) through the secondary table. This secondary
> > table will allow duplicate entries and the values of the secondary table
> > point to the key in packages.db. Depending on the task, we either query
> > the first or secondary table. The secondary table will be kept in sync
> > by BerkeleyDB.
> >
> > In the first case, we need to add a function to iterate over the
> > duplicate packages to find a specific version. In the second case, we
> > need to create the secondary table and transform the database.
> >
> > Which layout do you prefer?
> 
> I think that layout is better that better fits the code. Not yet having
> looked at the code, I cannot say. I guess 1 might be simpler. In the
> case of 2 I think "|" is fine, as it is already used elsewhere (though
> I guess one should make sure reprepro does not allow | in package
> names).

Okay. Attached the patch for my prototype. Be aware: It's just a
prototype that is just able to run the commands that I wanted to test,
but isn't near to be ready for mainlining. The prototype implements case
2 just because that was my initial idea, but now I tend to think that
case 1 might be easier/cleaner.

> > Another issue is the sorting of the packages in the database. We need
> > one function to sort all entries in the table. So we need one function
> > to sort binary packages and source packages, but we have
> > binaries_getversion() and source_getversion(). Here's the example code
> > (without the error handling) of the sorting function:
> > 
> > static int debianversioncompare(UNUSED(DB *db), const DBT *a, const DBT *b) {
> >     char *a_version, *b_version;
> >     int versioncmp;
> > 
> >     binaries_getversion(a->data, &a_version);
> >     binaries_getversion(b->data, &b_version);
> >     dpkgversions_cmp(a_version, b_version, &versioncmp);
> >     return versioncmp;
> > }
> > 
> > Do you have a suggestion how to improve this function?
> 
> It sounds quite slow either way. Perhaps the way to go is instead
> changing the data format, like having the version first (perhaps even in
> preparsed format to speed things up).

Good idea, but is this function really time critical? It should be only
called when comparing duplicate keys (which shouldn't happen that often,
does it?). How do you want to preparse the version?

How would the data format change? Currently the database value contains
just the control junk. We could put the pair (version, control) as value
into the database. How should the pair separated? Maybe with a null
character? Then we could just use the pointer to the value as version
string (the null character from the pair separation would also be used
to terminate the string).

-- 
Benjamin Drung
System Developer

ProfitBricks GmbH - The IaaS-Company
Greifswalder Str. 207
D - 10405 Berlin

Mail: benjamin.drung@profitbricks.com
Fax:  +49 30 577 008 598
URL:  http://www.profitbricks.com

Sitz der Gesellschaft: Berlin.
Registergericht: Amtsgericht Charlottenburg, HRB 125506 B.
Geschäftsführer: Andreas Gauger, Achim Weiss.

[prototype.patch (text/x-patch, attachment)]

Message #30 received at 570623@bugs.debian.org (full text, mbox, reply):

From: "Bernhard R. Link" <brlink@debian.org>

To: Benjamin Drung <benjamin.drung@profitbricks.com>

Cc: 570623@bugs.debian.org

Subject: Re: Bug#570623: reprepro: please add multiple version management

Date: Tue, 4 Feb 2014 23:23:08 +0100

* Benjamin Drung <benjamin.drung@profitbricks.com> [140203 13:15]:
> Okay. Attached the patch for my prototype. Be aware: It's just a
> prototype that is just able to run the commands that I wanted to test,
> but isn't near to be ready for mainlining. The prototype implements case
> 2 just because that was my initial idea, but now I tend to think that
> case 1 might be easier/cleaner.

Thanks. I'll take a look this weekend.

> > It sounds quite slow either way. Perhaps the way to go is instead
> > changing the data format, like having the version first (perhaps even in
> > preparsed format to speed things up).
>
> Good idea, but is this function really time critical? It should be only
> called when comparing duplicate keys (which shouldn't happen that often,
> does it?).

It might also happen when updating some value otherwise. (And if the
version is in some meta-data first one also does not have to
differentiate between binaries and sources that much). One could also
take the opportunity of a format change to allow for other possible
future meta data (like the first added timestamp).

> How do you want to preparse the version?

if versions are compared they are split into epoch version and revision
and version and revision are gain split into sequences of numbers and
not-numbers. Dpkg for example first parsed all the functions and later
only compares the already split part. if easily possible it could make
sense to store it in a format like that (but then parsing a on-disk
format of the split data might be just as time-consuming as just looking
at the real data).

> How would the data format change? Currently the database value contains
> just the control junk. We could put the pair (version, control) as value
> into the database. How should the pair separated? Maybe with a null
> character?

something like that.

> Then we could just use the pointer to the value as version
> string (the null character from the pair separation would also be used
> to terminate the string).

Yes. That would be the "store verbatim" and non-preparsed variant.
Alternatively one could first store a length of the string, so one can
even faster jump to the control part.


	Bernhard R. Link
-- 
F8AC 04D5 0B9B 064B 3383  C3DA AFFC 96D1 151D FFDC

Message #35 received at 570623@bugs.debian.org (full text, mbox, reply):

From: Allard Hoeve <allardhoeve@gmail.com>

To: 570623@bugs.debian.org

Date: Wed, 23 Apr 2014 11:34:40 +0200

[Message part 1 (text/plain, inline)]

Hey all,

This is exactly what I need :)

Has there been any activity on the multiple-version-per-package front?

Best,

Allard

[Message part 2 (text/html, inline)]

Message #40 received at 570623@bugs.debian.org (full text, mbox, reply):

From: Benjamin Drung <benjamin.drung@profitbricks.com>

To: 570623@bugs.debian.org

Cc: Allard Hoeve <allardhoeve@gmail.com>

Subject: Re: Bug#570623: reprepro: please add multiple version management

Date: Wed, 21 May 2014 16:44:21 +0200

[Message part 1 (text/plain, inline)]

Hi,

I got distracted by different tasks, but now I have time to work on
reprepro again.

Am Dienstag, den 04.02.2014, 23:23 +0100 schrieb Bernhard R. Link:
> * Benjamin Drung <benjamin.drung@profitbricks.com> [140203 13:15]:
> > Okay. Attached the patch for my prototype. Be aware: It's just a
> > prototype that is just able to run the commands that I wanted to test,
> > but isn't near to be ready for mainlining. The prototype implements case
> > 2 just because that was my initial idea, but now I tend to think that
> > case 1 might be easier/cleaner.
> 
> Thanks. I'll take a look this weekend.

Any feedback so far?

> > > It sounds quite slow either way. Perhaps the way to go is instead
> > > changing the data format, like having the version first (perhaps even in
> > > preparsed format to speed things up).
> >
> > Good idea, but is this function really time critical? It should be only
> > called when comparing duplicate keys (which shouldn't happen that often,
> > does it?).
> 
> It might also happen when updating some value otherwise. (And if the
> version is in some meta-data first one also does not have to
> differentiate between binaries and sources that much). One could also
> take the opportunity of a format change to allow for other possible
> future meta data (like the first added timestamp).

How flexible should the new data structure be? What meta data besides
the timestamp could be relevant?

> > How do you want to preparse the version?
> 
> if versions are compared they are split into epoch version and revision
> and version and revision are gain split into sequences of numbers and
> not-numbers. Dpkg for example first parsed all the functions and later
> only compares the already split part. if easily possible it could make
> sense to store it in a format like that (but then parsing a on-disk
> format of the split data might be just as time-consuming as just looking
> at the real data).

The version and revision can have a nearly unlimited amount of
concatenated numbers and not-numbers. You could store the parts as list
with type information. I doubt that a different on-disk format could
increase the speed. We could split the full version into epoch, version,
and revision and store them separately, but parsing these parts will be
more time consuming. My feeling is that we should stick with the full
version as string.

> > How would the data format change? Currently the database value contains
> > just the control junk. We could put the pair (version, control) as value
> > into the database. How should the pair separated? Maybe with a null
> > character?
> 
> something like that.
> 
> > Then we could just use the pointer to the value as version
> > string (the null character from the pair separation would also be used
> > to terminate the string).
> 
> Yes. That would be the "store verbatim" and non-preparsed variant.
> Alternatively one could first store a length of the string, so one can
> even faster jump to the control part.

What do you prefer? My current implementation just concatenates the
version string (including it's null character) and the control chunk. I
could expand the tuple to a tripple and add the timestamp (in which
format?) as third parameter.

While working on reprepro, I found a typo. A patch for that is attached.

-- 
Benjamin Drung
System Developer

ProfitBricks GmbH - The IaaS-Company
Greifswalder Str. 207
D - 10405 Berlin

Mail: benjamin.drung@profitbricks.com
Fax:  +49 30 577 008 598
URL:  http://www.profitbricks.com

Sitz der Gesellschaft: Berlin.
Registergericht: Amtsgericht Charlottenburg, HRB 125506 B.
Geschäftsführer: Andreas Gauger, Achim Weiss.

[Fix-typo-connot-cannot.patch (text/x-patch, attachment)]

Message #45 received at 570623@bugs.debian.org (full text, mbox, reply):

From: "Bernhard R. Link" <brlink@debian.org>

To: Benjamin Drung <benjamin.drung@profitbricks.com>

Cc: 570623@bugs.debian.org

Subject: Re: Bug#570623: reprepro: please add multiple version management

Date: Fri, 23 May 2014 23:39:14 +0200

* Benjamin Drung <benjamin.drung@profitbricks.com> [140521 16:48]:
> I got distracted by different tasks, but now I have time to work on
> reprepro again.
> 
> Am Dienstag, den 04.02.2014, 23:23 +0100 schrieb Bernhard R. Link:
> > * Benjamin Drung <benjamin.drung@profitbricks.com> [140203 13:15]:
> > > Okay. Attached the patch for my prototype. Be aware: It's just a
> > > prototype that is just able to run the commands that I wanted to test,
> > > but isn't near to be ready for mainlining. The prototype implements case
> > > 2 just because that was my initial idea, but now I tend to think that
> > > case 1 might be easier/cleaner.
> >
> > Thanks. I'll take a look this weekend.
>
> Any feedback so far?

It's too long ago for me to remember. AFAIR it mostly was that libdb
trick that is quite hard to understand.

> > > > It sounds quite slow either way. Perhaps the way to go is instead
> > > > changing the data format, like having the version first (perhaps even in
> > > > preparsed format to speed things up).
> > >
> > > Good idea, but is this function really time critical? It should be only
> > > called when comparing duplicate keys (which shouldn't happen that often,
> > > does it?).
> > 
> > It might also happen when updating some value otherwise. (And if the
> > version is in some meta-data first one also does not have to
> > differentiate between binaries and sources that much). One could also
> > take the opportunity of a format change to allow for other possible
> > future meta data (like the first added timestamp).
> 
> How flexible should the new data structure be? What meta data besides
> the timestamp could be relevant?

It would be nice if it allowed other fields to be added later without
breaking the format again. Like some (field-length-content)* format.

> > > How do you want to preparse the version?
> > 
> > if versions are compared they are split into epoch version and revision
> > and version and revision are gain split into sequences of numbers and
> > not-numbers. Dpkg for example first parsed all the functions and later
> > only compares the already split part. if easily possible it could make
> > sense to store it in a format like that (but then parsing a on-disk
> > format of the split data might be just as time-consuming as just looking
> > at the real data).
> 
> The version and revision can have a nearly unlimited amount of
> concatenated numbers and not-numbers. You could store the parts as list
> with type information. I doubt that a different on-disk format could
> increase the speed. We could split the full version into epoch, version,
> and revision and store them separately, but parsing these parts will be
> more time consuming. My feeling is that we should stick with the full
> version as string.

Yes, might not make much sense if one has to store the preparsed format.

> While working on reprepro, I found a typo. A patch for that is attached.

applied.

	Bernhard R. Link
-- 
F8AC 04D5 0B9B 064B 3383  C3DA AFFC 96D1 151D FFDC

Message #50 received at 570623@bugs.debian.org (full text, mbox, reply):

From: Benjamin Drung <benjamin.drung@profitbricks.com>

To: 570623@bugs.debian.org

Subject: Re: Bug#570623: reprepro: please add multiple version management

Date: Mon, 26 May 2014 17:37:56 +0200

Am Freitag, den 23.05.2014, 23:39 +0200 schrieb Bernhard R. Link:
> * Benjamin Drung <benjamin.drung@profitbricks.com> [140521 16:48]:
> > I got distracted by different tasks, but now I have time to work on
> > reprepro again.
> > 
> > Am Dienstag, den 04.02.2014, 23:23 +0100 schrieb Bernhard R. Link:
> > > * Benjamin Drung <benjamin.drung@profitbricks.com> [140203 13:15]:
> > > > Okay. Attached the patch for my prototype. Be aware: It's just a
> > > > prototype that is just able to run the commands that I wanted to test,
> > > > but isn't near to be ready for mainlining. The prototype implements case
> > > > 2 just because that was my initial idea, but now I tend to think that
> > > > case 1 might be easier/cleaner.
> > >
> > > Thanks. I'll take a look this weekend.
> >
> > Any feedback so far?
> 
> It's too long ago for me to remember. AFAIR it mostly was that libdb
> trick that is quite hard to understand.

My current approach is to use the secondary DB again (case 2). The
primary DB stores packagename+version (e.g. "hello|2.5-1") and does not
allow duplicates. The secondary DB stores just the packagename (e.g.
"hello") and does allow duplicates. The duplicates are sorted by their
version.

The function get_package_name is used to derives the secondary key from
the primary key by extracting the packagename (e.g. "hello|2.5-1" ->
"hello").

This approach has the advantage that these operations are easy to
implement/cost efficient:

1) get the latest package version (first entry in the secondary DB for a
given package name)
2) get a specific package version (entry in the primary DB for a given
packagename+version combination)

For example adding a new package would do this:
Get the latest package version for the given package name (1). If the
result is NULL or the returned package version is older than the new
package, add the new package. If the version is the same, reject the new
package. If the version is newer than the new package, check if the
package version is already present in the archive (2) and reject it in
this case. Otherwise add the new package if downgrades are allowed.

> > > > > It sounds quite slow either way. Perhaps the way to go is instead
> > > > > changing the data format, like having the version first (perhaps even in
> > > > > preparsed format to speed things up).
> > > >
> > > > Good idea, but is this function really time critical? It should be only
> > > > called when comparing duplicate keys (which shouldn't happen that often,
> > > > does it?).
> > > 
> > > It might also happen when updating some value otherwise. (And if the
> > > version is in some meta-data first one also does not have to
> > > differentiate between binaries and sources that much). One could also
> > > take the opportunity of a format change to allow for other possible
> > > future meta data (like the first added timestamp).
> > 
> > How flexible should the new data structure be? What meta data besides
> > the timestamp could be relevant?
> 
> It would be nice if it allowed other fields to be added later without
> breaking the format again. Like some (field-length-content)* format.

Fields to store (for the beginning):
1) package version string
2) control chunk string
3) added timestamp

Here are different, but similar on-disk format proposals:

First variant
=============

Physical order of the data
--------------------------

field 1
field 2
...
field n
sizeof(field n) [stored as 32-bit unsigned int]
...
sizeof(field 2) [stored as 32-bit unsigned int]
sizeof(field 1) [stored as 32-bit unsigned int]
number of field [stored as 8-bit (or 16-bit?) unsigned int]

In our case this would be
-------------------------

[variable] package version string + '\0'
[variable] control chunk string + '\0'
[8 bytes] timestamp (which format? 64-bit Unix timestamp?)
[4 bytes] sizeof(timestamp) = 8 (in case of 64-bit timestamp)
[4 bytes] strlen(control chunk) + 1
[4 bytes] strlen(package version) + 1
[1 byte] 3

Access to the individual entries for a given void *data and size_t len:
size_t end = (size_t)data + len;

char *version = (char*)data;
size_t version_len = *(uint32_t)(end - sizeof(uint8_t) - sizeof(uint32_t));
fail if *(uint8_t)(end - sizeof(uint8_t)) < 1

char chunk = (char*)((size_t)data + version_len);
size_t chunk_len = *(uint32_t)(end - sizeof(uint8_t) - 2 * sizeof(uint32_t));
fail if *(uint8_t)(end - sizeof(uint8_t)) < 2

time_t timestamp = *(int64_t)((size_t)data + version_len + chunk_len);
fail if *(uint8_t)(end - sizeof(uint8_t)) < 3

Additional checks:
size_t timestamp_len = *(uint32_t)(end - sizeof(uint8_t) - 3 * sizeof(uint32_t));
timestamp_len == sizeof(int64_t)
len == version_len + chunk_len + timestamp_len + sizeof(uint8_t) + 3 * sizeof(uint32_t)

Second variant
==============

The second variant is similar to the first, but only the size of fields
are stored that have a variable length. In our case, we would store the
length of the version and control chunk, but not the length of the
timestamp.

third variant
=============

Instead of storing the length of the field, the offset is stored:

Physical order of the data
--------------------------

field 1
field 2
...
field n
offset field n [stored as 32-bit unsigned int]
...
offset field 3 [stored as 32-bit unsigned int]
offset field 2 [stored as 32-bit unsigned int]
number of field [stored as 8-bit (or 16-bit?) unsigned int]

In our case this would be
-------------------------

[variable] package version string + '\0'
[variable] control chunk string + '\0'
[8 bytes] timestamp (which format? 64-bit Unix timestamp?)
[4 bytes] strlen(package version) + strlen(control chunk) + 2
[4 bytes] strlen(package version) + 1
[1 byte] 3

Access to the individual entries for a given void *data and size_t len:
size_t end = (size_t)data + len;

char *version = (char*)data;
size_t chunk_offset = *(uint32_t)(end - sizeof(uint8_t) - sizeof(uint32_t));
size_t version_len = chunk_offset;
fail if *(uint8_t)(end - sizeof(uint8_t)) < 1

char chunk = (char*)((size_t)data + chunk_offset);
size_t timestamp_offset = *(uint32_t)(end - sizeof(uint8_t) - 2 * sizeof(uint32_t));
size_t chunk_len = timestamp_offset - chunk_offset;
fail if *(uint8_t)(end - sizeof(uint8_t)) < 2

time_t timestamp = *(int64_t)((size_t)data + timestamp_offset);
fail if *(uint8_t)(end - sizeof(uint8_t)) < 3

Additional checks:
chunk_offset < timestamp_offset
timestamp_offset < len
len == version_len + chunk_len + timestamp_len + sizeof(uint8_t) + 3 * sizeof(uint32_t)

Modified first/second/third variant
===================================

This variant can be a modification to the first, second, or third
variant and therefore become variant four till the sixth. Instead of
storing the number of field at the end of the data block, the number of
fields are stored at the beginning of the data block:

number of field [stored as 8-bit (or 16-bit?) unsigned int]
field 1
field 2
...
field n
sizeof/offset m
...
sizeof/offset 2
sizeof/offset 1

Question
========

Which on-disk format do you prefer?

-- 
Benjamin Drung
System Developer

ProfitBricks GmbH - The IaaS-Company
Greifswalder Str. 207
D - 10405 Berlin

Mail: benjamin.drung@profitbricks.com
Fax:  +49 30 577 008 598
URL:  http://www.profitbricks.com

Sitz der Gesellschaft: Berlin.
Registergericht: Amtsgericht Charlottenburg, HRB 125506 B.
Geschäftsführer: Andreas Gauger, Achim Weiss.

Message #55 received at 570623@bugs.debian.org (full text, mbox, reply):

From: Benjamin Drung <benjamin.drung@profitbricks.com>

To: 570623@bugs.debian.org

Cc: Allard Hoeve <allardhoeve@gmail.com>

Subject: Re: Bug#570623: reprepro: please add multiple version management

Date: Fri, 30 May 2014 23:04:57 +0200

[Message part 1 (text/plain, inline)]

Hi,

Am Montag, den 26.05.2014, 17:37 +0200 schrieb Benjamin Drung:
> Here are different, but similar on-disk format proposals:
> 
> First variant
> =============
> 
> Physical order of the data
> --------------------------
> 
> field 1
> field 2
> ...
> field n
> sizeof(field n) [stored as 32-bit unsigned int]
> ...
> sizeof(field 2) [stored as 32-bit unsigned int]
> sizeof(field 1) [stored as 32-bit unsigned int]
> number of field [stored as 8-bit (or 16-bit?) unsigned int]
> 
> In our case this would be
> -------------------------
> 
> [variable] package version string + '\0'
> [variable] control chunk string + '\0'
> [8 bytes] timestamp (which format? 64-bit Unix timestamp?)
> [4 bytes] sizeof(timestamp) = 8 (in case of 64-bit timestamp)
> [4 bytes] strlen(control chunk) + 1
> [4 bytes] strlen(package version) + 1
> [1 byte] 3
> 
> Access to the individual entries for a given void *data and size_t len:
> size_t end = (size_t)data + len;
> 
> char *version = (char*)data;
> size_t version_len = *(uint32_t)(end - sizeof(uint8_t) - sizeof(uint32_t));
> fail if *(uint8_t)(end - sizeof(uint8_t)) < 1
> 
> char chunk = (char*)((size_t)data + version_len);
> size_t chunk_len = *(uint32_t)(end - sizeof(uint8_t) - 2 * sizeof(uint32_t));
> fail if *(uint8_t)(end - sizeof(uint8_t)) < 2
> 
> time_t timestamp = *(int64_t)((size_t)data + version_len + chunk_len);
> fail if *(uint8_t)(end - sizeof(uint8_t)) < 3
> 
> Additional checks:
> size_t timestamp_len = *(uint32_t)(end - sizeof(uint8_t) - 3 * sizeof(uint32_t));
> timestamp_len == sizeof(int64_t)
> len == version_len + chunk_len + timestamp_len + sizeof(uint8_t) + 3 * sizeof(uint32_t)
> 
> Second variant
> ==============
> 
> The second variant is similar to the first, but only the size of fields
> are stored that have a variable length. In our case, we would store the
> length of the version and control chunk, but not the length of the
> timestamp.

I went ahead and implemented the second variant. The data is parsed via
helper functions in packagedata.c and can be simple replaced in case you
prefer a different on-disk format.

I found space indentations in main.c and replaced them with tabs (patch
0001 attached). My current work is attached as patch 0002. You need to
run autoreconf after applying that patch. Please have a look at it and
let me know if I should change something or if I could go ahead with the
current approach. Adding new packages with the include command work and
the on-disk format is correct. The package deletion does not work, yet.
(The DB key needs to be correctly specified.) I will continue working on
reprepro next week.

-- 
Benjamin Drung
System Developer

ProfitBricks GmbH - The IaaS-Company
Greifswalder Str. 207
D - 10405 Berlin

Mail: benjamin.drung@profitbricks.com
Fax:  +49 30 577 008 598
URL:  http://www.profitbricks.com

Sitz der Gesellschaft: Berlin.
Registergericht: Amtsgericht Charlottenburg, HRB 125506 B.
Geschäftsführer: Andreas Gauger, Achim Weiss.

[0001-Fix-indentation-spaces-to-tabs.patch (text/x-patch, attachment)]

[0002-Add-multiple-version-management.patch (text/x-patch, attachment)]

Message #60 received at 570623@bugs.debian.org (full text, mbox, reply):

From: Benjamin Drung <benjamin.drung@profitbricks.com>

To: 570623@bugs.debian.org

Cc: Allard Hoeve <allardhoeve@gmail.com>

Subject: Re: Bug#570623: reprepro: please add multiple version management

Date: Thu, 12 Jun 2014 17:26:47 +0200

[Message part 1 (text/plain, inline)]

Hi,

the logging and tracking was not working in all cases when adding
packages with the include command. The fix is in the attached 0003
patch. The include and processincoming commands do work now (except when
these commands want to remove a package, which is still on my todo
list).

The attached patch 0004 make the copy command work and allows one to
specify versions to it (e.g. reprepro copy dst src hello=2.8-1).

If you prefer one big patch for the new feature instead of multiple
smaller ones, apply the attached Add-multiple-version-management.patch
instead of the patches 0002 - 0004.

While working on the feature, I simplified the Debian packaging. Three
patches (debian-000*.patch) are attached, too. They are hopefully
self-explaining.

Next: Make the package deletion work. (The DB key needs to be correctly
specified.)

-- 
Benjamin Drung
System Developer

ProfitBricks GmbH - The IaaS-Company
Greifswalder Str. 207
D - 10405 Berlin

Mail: benjamin.drung@profitbricks.com
Fax:  +49 30 577 008 598
URL:  http://www.profitbricks.com

Sitz der Gesellschaft: Berlin.
Registergericht: Amtsgericht Charlottenburg, HRB 125506 B.
Geschäftsführer: Andreas Gauger, Achim Weiss.

[0003-Fix-logging-and-tracking-when-adding-new-packages.patch (text/x-patch, attachment)]

[0004-Support-specifying-versions-to-the-copy-command.patch (text/x-patch, attachment)]

[Add-multiple-version-management.patch (text/x-patch, attachment)]

[debian-0001-Add-build-dependency-on-liblzma-dev.patch (text/x-patch, attachment)]

[debian-0002-Wrap-long-lines-with-wrap-and-sort.patch (text/x-patch, attachment)]

[debian-0003-Switch-to-dh-9.patch (text/x-patch, attachment)]

Message #65 received at 570623@bugs.debian.org (full text, mbox, reply):

From: "Bernhard R. Link" <brlink@debian.org>

To: Benjamin Drung <benjamin.drung@profitbricks.com>

Cc: 570623@bugs.debian.org

Subject: Re: Bug#570623: reprepro: please add multiple version management

Date: Mon, 23 Jun 2014 20:05:46 +0200

* Benjamin Drung <benjamin.drung@profitbricks.com> [140612 17:30]:
> the logging and tracking was not working in all cases when adding
> packages with the include command. The fix is in the attached 0003
> patch. The include and processincoming commands do work now (except when
> these commands want to remove a package, which is still on my todo
> list).

What version of reprepro is that patch relative to? I'm a bit at loss
at packagedata.c.


	Bernhard R. Link
-- 
F8AC 04D5 0B9B 064B 3383  C3DA AFFC 96D1 151D FFDC

Message #70 received at 570623@bugs.debian.org (full text, mbox, reply):

From: Benjamin Drung <benjamin.drung@profitbricks.com>

To: 570623@bugs.debian.org

Subject: Re: Bug#570623: reprepro: please add multiple version management

Date: Tue, 24 Jun 2014 19:44:54 +0200

[Message part 1 (text/plain, inline)]

Am Montag, den 23.06.2014, 20:05 +0200 schrieb Bernhard R. Link:
> * Benjamin Drung <benjamin.drung@profitbricks.com> [140612 17:30]:
> > the logging and tracking was not working in all cases when adding
> > packages with the include command. The fix is in the attached 0003
> > patch. The include and processincoming commands do work now (except when
> > these commands want to remove a package, which is still on my todo
> > list).
> 
> What version of reprepro is that patch relative to? I'm a bit at loss
> at packagedata.c.

That 0003 patch was a follow-up patch for the previously posted 0002
patch.

Here is a complete status update with all previously and newly developed
patches: There is one prerequisite:
0001-Fix-indentation-spaces-to-tabs.patch. Then you can decide to apply
patch 0002 to 0014 or to apply Add-multiple-version-management.patch
(which combines patch 0002 to 0014 in one big patch). The patches were
created against the current git master branch.

The patches should be ready to use. I use them in a new setup for one or
two weeks now. With the latest patches, everything should work (tested
copy, include, remove, ls, list, listfilter without any issue).

PS: While working on this feature, I found a bug related to the new xz
support. A patch for that is attached to bug #748658.

-- 
Benjamin Drung
System Developer

ProfitBricks GmbH - The IaaS-Company
Greifswalder Str. 207
D - 10405 Berlin

Mail: benjamin.drung@profitbricks.com
Fax:  +49 30 577 008 598
URL:  http://www.profitbricks.com

Sitz der Gesellschaft: Berlin.
Registergericht: Amtsgericht Charlottenburg, HRB 125506 B.
Geschäftsführer: Andreas Gauger, Achim Weiss.

[0001-Fix-indentation-spaces-to-tabs.patch (text/x-patch, attachment)]

[0002-Add-multiple-version-management.patch (text/x-patch, attachment)]

[0003-Fix-logging-and-tracking-when-adding-new-packages.patch (text/x-patch, attachment)]

[0004-Support-specifying-versions-to-the-copy-command.patch (text/x-patch, attachment)]

[0005-Fix-ls-command-to-list-multiple-versions-of-one-pack.patch (text/x-patch, attachment)]

[0006-Fix-list-command-to-list-multiple-versions-of-one-pa.patch (text/x-patch, attachment)]

[0007-Fix-package-removal-and-support-specifying-versions-.patch (text/x-patch, attachment)]

[0008-Rename-variable-todo-to-remaining.patch (text/x-patch, attachment)]

[0009-Pass-error-code-back-to-BerkeleyDB-and-print-nicer-e.patch (text/x-patch, attachment)]

[0010-Drop-unused-table_sec_newglobalcursor-function.patch (text/x-patch, attachment)]

[0011-Merge-table_adduniqsizedrecord2-into-table_adduniqsi.patch (text/x-patch, attachment)]

[0012-Use-table_getcomplexrecord-for-table_getrecord.patch (text/x-patch, attachment)]

[0013-Drop-unused-function-table_replacerecord.patch (text/x-patch, attachment)]

[0014-Merge-cursor_nexttempdata2-into-cursor_nexttempdata-.patch (text/x-patch, attachment)]

[Add-multiple-version-management.patch (text/x-patch, attachment)]

Message #75 received at 570623@bugs.debian.org (full text, mbox, reply):

From: Michael Stucki <michael.stucki@typo3.org>

To: 570623@bugs.debian.org

Subject: Re: Bug#570623: reprepro: please add multiple version management

Date: Fri, 14 Nov 2014 10:04:07 +0100

[Message part 1 (text/plain, inline)]

What is the status of this project? It looks complete so what is needed to
have this merged?
Let me know how I can help!

[Message part 2 (text/html, inline)]

Message #80 received at 570623@bugs.debian.org (full text, mbox, reply):

From: Johannes Amorosa | Celluloid VFX <johannesa@celluloid-vfx.com>

To: 570623@bugs.debian.org

Subject: Re: Bug#570623: reprepro: please add multiple version management

Date: Mon, 15 Dec 2014 16:56:43 +0100

Exactly what I need.
Looking forward to have this patch in the package repository.
Can I help to implement this?

Joe

Message #85 received at 570623@bugs.debian.org (full text, mbox, reply):

From: Walter Doekes <wjdoekes@osso.nl>

To: 570623@bugs.debian.org

Cc: wjdoekes@osso.nl

Subject: Re: Bug#570623: reprepro: please add multiple version management

Date: Tue, 07 Apr 2015 12:56:08 +0200

[Message part 1 (text/plain, inline)]

Cool. So, I tested this by installing:

  - https://alioth.debian.org/anonscm/git/mirrorer/mirrorer.git
    (git repo)

  - branched off the (default) debian branch [2]

  - patched with Add-multiple-version-management.patch

https://bugs.debian.org/cgi-bin/bugreport.cgi?msg=70;filename=Add-multiple-version-management.patch;att=15;bug=570623

  - fixed 3 conflicts in main.c, not sure what was conflicting, probably
    leftover whitespace or something.

  - autoreconf -vi (see INSTALL for dependencies)
  - ./configure --prefix=/usr/local; make; make install
  - renamed the resultant binaries with "-mver" suffix so they don't
    conflict with the rest of my setup


Initial observations:

  - I started a new multi-version repository and loaded up several
    versions of a package: it works.
  - Downgrading/upgrading a single package from that repository
    works.

  - Warnings when including a new package:

> debian-mver# reprepro-mver -C acos include wheezy /srv/junk/asterisk*13.3.*changes
> Error while parsing 'asterisk-config|1:13.3.0-acos1+o0' as version: epoch in version is not number
> Parse errors processing versions.
> Error while parsing 'asterisk-dbg|1:13.3.0-acos1+o0' as version: epoch in version is not number
> Parse errors processing versions.
...

    It complained about the epoch in version not being a number. I
    assume someone takes "asterisk-config|1" to be an epoch, instead
    of just "1". (See below/attachment for a fix.)

  - When including two packages with the same minor/micro version,
    things were less okay:

> debian-mver# reprepro-mver -C acos include wheezy /srv/junk/old/asterisk_13.2.0*acos2*changes
> Error while parsing 'asterisk-config|1:13.2.0-acos2+o0+2up' as version: epoch in version is not number
> Parse errors processing versions.
...
> Internal error of the underlying BerkeleyDB database:
> Within references.db subtable references at put: DB_KEYEXIST: Key/data pair already exists
> Warning: database 'wheezy|acos|amd64' was modified but no index file was exported.
> Changes will only be visible after the next 'export'!
> There have been errors!

   But a dump [1] of the references file revealed no immediate problem:

> debian-mver# diff -pu <(cat -A references.db.2) <(cat -A references.db.3)
> --- /dev/fd/63	2015-04-07 11:45:35.900044000 +0200
> +++ /dev/fd/62	2015-04-07 11:45:35.900044000 +0200
> @@ -8,38 +8,72 @@ db_pagesize=4096$
>  HEADER=END$	
>  >>>  pool/acos/a/asterisk/asterisk-config_13.2.0-acos1+o0+2up_all.deb^@$
>  >>>  wheezy|acos|amd64^@$
> +>>>  pool/acos/a/asterisk/asterisk-config_13.2.0-acos2+o0+2up_all.deb^@$
> +>>>  wheezy|acos|amd64^@$
...

  I added a few debug prints, and I got this:

...
> Skipping inclusion of 'asterisk-modules' '1:13.2.0-acos2+o0+2up' in 'wheezy|acos|amd64', as this version already exists.
> Skipping inclusion of 'asterisk' '1:13.2.0-acos2+o0+2up' in 'wheezy|acos|amd64', as this version already exists.

> db: 'pool/acos/a/asterisk/asterisk_13.2.0-acos2+o0+2up.dsc' adding to references.db(references).
> Internal error of the underlying BerkeleyDB database:
> Within references.db subtable references at put: DB_KEYEXIST: Key/data pair already exists
> db: 'pool/acos/a/asterisk/asterisk_13.2.0.orig.tar.gz' adding to references.db(references).
> Internal error of the underlying BerkeleyDB database:
> Within references.db subtable references at put: DB_KEYEXIST: Key/data pair already exists
> db: 'pool/acos/a/asterisk/asterisk_13.2.0-acos2+o0+2up.debian.tar.gz' adding to references.db(references).
> Internal error of the underlying BerkeleyDB database:
> Within references.db subtable references at put: DB_KEYEXIST: Key/data pair already exists
> There have been errors!

  Those errors only popped up when uploading the 2nd (acos2) version.
  For the first (acos1), I just get the "Skipping" notices like
  expected (when calling it a second/third time).

  - Removing a package worked, unless we have multiple versions:

> debian-mver# reprepro-mver -C acos remove wheezy libpjproject-dev=2.3.0.0.ast20150218-acos1

  OK

  - Removing a package did not work when we have multiple versions:

> debian-mver# reprepro-mver  -C acos list wheezy | head -n2
> wheezy|acos|amd64: asterisk 1:13.3.0-acos1+o0
> wheezy|acos|amd64: asterisk 1:13.2.0-acos2+o0+2up
...
> debian-mver# reprepro-mver -C acos remove wheezy asterisk=1:13.2.0-acos2+o0+2up
> removing 'asterisk=1:13.2.0-acos2+o0+2up' from 'wheezy|acos|amd64'...
> Error while parsing 'asterisk|1:13.2.0-acos2+o0+2up' as version: epoch in version is not number
> Parse errors processing versions.
> Error while parsing 'asterisk|1:13.2.0-acos2+o0+2up' as version: epoch in version is not number
> Parse errors processing versions.
> /srv/reprepro/debian-mver/db/packages.db/wheezy|acos|amd64: DB_SECONDARY_BAD: Secondary index inconsistent with primary
> Internal error of the underlying BerkeleyDB database:
> Within packages.db subtable wheezy|acos|amd64 at del: DB_SECONDARY_BAD: Secondary index inconsistent with primary
> There have been errors!

  - However, when adding bug570623_version_compare.patch (attached),
    removal started to work as expected.


In summary:

- patching the version compare helps, but
- multiple closely related versions still cause grief


Thanks for the patches!


Cheers,
Walter Doekes
OSSO B.V.


[1] references.db was dumped like this:
    $ db5.1_dump db/references.db |
      perl -pe '/^ /&&s/^/>>> /&&s/[0-9a-f]{2}/chr hex$&/ige'

[2] changing from the `debian` branch to the `master` branch didn't
    seem to change anything

[bug570623_version_compare.patch (text/x-diff, attachment)]

Message #90 received at 570623@bugs.debian.org (full text, mbox, reply):

From: David Kleuker <post@davidak.de>

To: 570623@bugs.debian.org

Subject: Re: Bug#570623: reprepro: please add multiple version management

Date: Mon, 09 Nov 2015 14:14:28 +0100

we would really like to have this. any updates?

Message #97 received at 570623@bugs.debian.org (full text, mbox, reply):

From: Devel Chip <devel.chip@gmail.com>

To: 570623@bugs.debian.org

Subject: Re: Bug#570623: reprepro: please add multiple version management

Date: Mon, 18 Apr 2016 08:38:35 +0200

[Message part 1 (text/plain, inline)]

> Error while parsing 'asterisk-config|1:13.3.0-acos1+o0' as version: epoch in version is not number

I don't think that '|' character is allowed in the names by design.

But I've encountered another unrelated problem when syncing.

The conf/updates file (package name is in the mirror.packages):

Name: upstream-update
Method: http://localhost:20080/
Components: main
FilterList: purge ../mirror.packages

The receiving repository has multiple versions of a package.

The following error occurs:

Calculating packages to get...
Package database is not sorted!!!
reprepro: upgradelist.c:134: save_package_version: Assertion `0' failed.

[Message part 2 (text/html, inline)]

Message #102 received at 570623@bugs.debian.org (full text, mbox, reply):

From: Devel Chip <devel.chip@gmail.com>

To: 570623@bugs.debian.org

Subject: Re: Bug#570623: Info received (Bug#570623: reprepro: please add multiple version management)

Date: Mon, 18 Apr 2016 08:53:29 +0200

[Message part 1 (text/plain, inline)]

ATTENTION: I have no idea of what I'm doing

fixing update/sync:

diff --git a/upgradelist.c b/upgradelist.c
index 272b46c..8ee0de2 100644
--- a/upgradelist.c
+++ b/upgradelist.c
@@ -126,7 +126,7 @@ static retvalue save_package_version(struct upgradelist
*upgrade, const char *pa
                if (strcmp(packagename, upgrade->last->name) > 0) {
                        upgrade->last->next = package;
                        upgrade->last = package;
-               } else {
+               } else if (strcmp(packagename, upgrade->last->name) != 0) {
                        /* this should only happen if the underlying
                         * database-method get changed, so just throwing
                         * out here */


On 18 April 2016 at 08:42, Debian Bug Tracking System <owner@bugs.debian.org
> wrote:

> Thank you for the additional information you have supplied regarding
> this Bug report.
>
> This is an automatically generated reply to let you know your message
> has been received.
>
> Your message is being forwarded to the package maintainers and other
> interested parties for their attention; they will reply in due course.
>
> Your message has been sent to the package maintainer(s):
>  Bernhard R. Link <brlink@debian.org>
>
> If you wish to submit further information on this problem, please
> send it to 570623@bugs.debian.org.
>
> Please do not send mail to owner@bugs.debian.org unless you wish
> to report a problem with the Bug-tracking system.
>
> --
> 570623: http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=570623
> Debian Bug Tracking System
> Contact owner@bugs.debian.org with problems
>

[Message part 2 (text/html, inline)]

Message #107 received at 570623@bugs.debian.org (full text, mbox, reply):

From: Benjamin Drung <benjamin.drung@profitbricks.com>

To: 570623@bugs.debian.org

Cc: Jean-Christophe Dubacq <jcdubacq1@free.fr>, Allard Hoeve <allardhoeve@gmail.com>, Michael Stucki <michael.stucki@typo3.org>, Walter Doekes <wjdoekes@osso.nl>, David Kleuker <post@davidak.de>, Devel Chip <devel.chip@gmail.com>, Waldemar Brodkorb <mail@waldemar-brodkorb.de>

Subject: Re: Bug#570623: reprepro: please add multiple version management

Date: Fri, 10 Feb 2017 17:51:46 +0100

[Message part 1 (text/plain, inline)]

Hi everyone,

the last days I worked on rebasing my multiple-version management
patches on reprepro 5.1.1. Thanks to the upstream refactoring, I had to
re-do the patch nearly from scratch. In that process I split the big
patch into small, reviewable and maintainable patches.

Attached are the all patches. You can also pull them from the
5.1.1-mutiple-versions branch from
https://github.com/profitbricks/reprepro
Feel free to report bugs on GitHub.

Bernhard, please review my patches and include the patches that you
like. Some patches are unrelated to the multiple versions feature (e.g.
patches 8 and 9). Some patches could already be included even when you
do not apply the final patch. Let me know if you want further
refactoring.

Release Notes
=============

1) The database layout of the multiple-versions branch changed between
the 4.17.1 based patch set and the 5.1.1 patch set. There is currently
no migration script (I will provide one after my vacation).

2) The database layout changes from the upstream 5.1.1 release and the
multiple versions patch set. The difference is as following:

5.1.1
-----

* packages.db maps "package name" to "control file" without duplicates
* no packagenames.db

5.1.1 + multiple versions
-------------------------

* packages.db maps "package name|version" to "control file" without
duplicates
* packagenames.db maps "package name" to "package name|version"
allowing duplicates and duplicates sorted by dpkg --compare-versions
descending

I am working on a migration from 5.1.1 to 5.1.1 + multiple versions and
 will probably finish it when I am back from vacation.

So you have to start with an empty database for now.

-- 
Benjamin Drung
System Developer
Debian & Ubuntu Developer

ProfitBricks GmbH
Greifswalder Str. 207
D - 10405 Berlin

Email: benjamin.drung@profitbricks.com
URL:  http://www.profitbricks.com

Sitz der Gesellschaft: Berlin.
Registergericht: Amtsgericht Charlottenburg, HRB 125506B.
Geschäftsführer: Andreas Gauger, Achim Weiss.

[0001-testsuite-Use-dpkg-source-format-1.0.patch (text/x-patch, attachment)]

[0001-warn-if-old-legacy-files.db-is-still-used-and-add-ne.patch (text/x-patch, attachment)]

[0002-testsuite-Use-the-host-architecture-by-default.patch (text/x-patch, attachment)]

[0003-testsuite-Add-basic-bash_unit-based-tests.patch (text/x-patch, attachment)]

[0004-Update-auto-generated-files.patch (text/x-patch, attachment)]

[0005-Add-ignores-for-tests-to-.gitignore.patch (text/x-patch, attachment)]

[0006-debian-Switch-to-dh-9.patch (text/x-patch, attachment)]

[0007-debian-Update-changelog.patch (text/x-patch, attachment)]

[0008-format-Fix-indentation-spaces-to-tabs.patch (text/x-patch, attachment)]

[0009-Evaluate-return-value-of-write-command.patch (text/x-patch, attachment)]

[0010-Print-version-when-removing-a-package.patch (text/x-patch, attachment)]

[0011-refactor-Introduce-static-newcursor-function.patch (text/x-patch, attachment)]

[0012-table_getrecord-Add-newline-to-error-message.patch (text/x-patch, attachment)]

[0013-refactor-Merge-cursor_nexttemp-into-cursor_nexttempd.patch (text/x-patch, attachment)]

[0014-refactor-Introduce-parse_data.patch (text/x-patch, attachment)]

[0015-table_printerror-Improve-database-error-message.patch (text/x-patch, attachment)]

[0016-refactor-database-Move-cursor-struct-upwards.patch (text/x-patch, attachment)]

[0017-refactor-rename-table_newduplicatecursor.patch (text/x-patch, attachment)]

[0018-refactor-Introduce-cursor_next.patch (text/x-patch, attachment)]

[0019-refactor-rename-todo-to-remaining.patch (text/x-patch, attachment)]

[0020-refactor-target_removepackage-Support-specifying-pac.patch (text/x-patch, attachment)]

[0021-refactor-Introduce-splitnameandversion.patch (text/x-patch, attachment)]

[0022-refactor-table_close-Set-default-return-value.patch (text/x-patch, attachment)]

[0023-Add-trace-debugging-output.patch (text/x-patch, attachment)]

[0024-Add-helper-function-debianversioncompare.patch (text/x-patch, attachment)]

[0025-Add-helper-function-get_package_name.patch (text/x-patch, attachment)]

[0026-Add-helper-function-package_primarykey.patch (text/x-patch, attachment)]

[0027-Add-helper-function-cascade_strcmp.patch (text/x-patch, attachment)]

[0028-Add-helper-function-strcmp2.patch (text/x-patch, attachment)]

[0029-Add-helper-function-table_newduplicatecursor.patch (text/x-patch, attachment)]

[0030-Add-helper-function-package_openduplicateiterator.patch (text/x-patch, attachment)]

[0031-Support-listing-multiple-versions-in-ls-command.patch (text/x-patch, attachment)]

[0032-Support-listing-multiple-versions-in-list-command.patch (text/x-patch, attachment)]

[0033-refactor-Introduce-nameandversion-struct-for-copy-co.patch (text/x-patch, attachment)]

[0034-refactor-Use-nameandversion-struct-for-remove-comman.patch (text/x-patch, attachment)]

[0035-refactor-target_addpackage-Add-replace-variable.patch (text/x-patch, attachment)]

[0036-Add-multiple-version-management.patch (text/x-patch, attachment)]

[0037-testsuite-Add-multiversion-test-cases.patch (text/x-patch, attachment)]

[0038-Update-auto-generated-files.patch (text/x-patch, attachment)]

Message #112 received at 570623@bugs.debian.org (full text, mbox, reply):

From: Benjamin Drung <benjamin.drung@profitbricks.com>

To: 570623@bugs.debian.org

Cc: Jean-Christophe Dubacq <jcdubacq1@free.fr>, Allard Hoeve <allardhoeve@gmail.com>, Michael Stucki <michael.stucki@typo3.org>, Walter Doekes <wjdoekes@osso.nl>, David Kleuker <post@davidak.de>, Devel Chip <devel.chip@gmail.com>, sebastian@fds-team.de

Subject: Re: Bug#570623: reprepro: please add multiple version management

Date: Wed, 29 Mar 2017 19:31:51 +0200

[Message part 1 (text/plain, inline)]

Hi everyone,

I am happy to announce the feature complete version of reprepro 5.1.1
with multiple version management. Attached are all 58 patches. You can
also pull the 5.1.1-mutiple-versions branch from
https://github.com/profitbricks/reprepro
Feel free to report bugs on GitHub.

Bernhard, please review my patches and include the patches that you
like. Some patches are unrelated to the multiple versions feature (e.g.
patches 11 and 12). Some patches could already be included even when
you do not apply the final patch. Let me know if you want further
refactoring.

Release Notes
=============

The multiple-versions patch set adds following features:

* Add shunit2 based tests (Closes: #857302)
* Support multiple versions. (Closes: #570623)
* Add listdistros command (Closes: #857303)
* Add the commands move, movesrc, movematched, movefilter
* Add Limit and Archive option

Behavior changes
----------------

The multiple-versions reprepro keeps all package versions in the
archive. Set "Limit: 1" to keep only one version per package in the
archive to restore the previous behavior.

Database layout changes
-----------------------

The database layout changes from the upstream 5.1.1 release to the
multiple versions patch set. The difference is as following:

5.1.1:

* packages.db maps "package name" to "control file" without duplicates
* no packagenames.db

5.1.1 + multiple versions:

* packages.db maps "package name|version" to "control file" without
duplicates
* packagenames.db maps "package name" to "package name|version"
allowing duplicates and duplicates sorted by dpkg --compare-versions
descending

The multiple-versions reprepro supports migrating from 5.1.1 to
5.1.1 + multiple versions. Warning: There is no way back (but could be
done with a simple Python script)!

Notes for users of 4.16.1 + multiple versions
---------------------------------------------

The database layout of the multiple-versions branch changed between
the 4.17.1 based patch set and the 5.1.1 patch set. Attached is the
small Python script 'migrate-db-from-4.17-to-5.1' to migrate the
database to the new layout. You have to install python3-bsddb3 and run
the script in basedir. It should be idempontent.

-- 
Benjamin Drung
System Developer
Debian & Ubuntu Developer

ProfitBricks GmbH
Greifswalder Str. 207
D - 10405 Berlin

Email: benjamin.drung@profitbricks.com
URL:  http://www.profitbricks.com

Sitz der Gesellschaft: Berlin.
Registergericht: Amtsgericht Charlottenburg, HRB 125506B.
Geschäftsführer: Andreas Gauger, Achim Weiss.

[0001-testsuite-Use-dpkg-source-format-1.0.patch (text/x-patch, attachment)]

[0002-testsuite-Use-the-host-architecture-by-default.patch (text/x-patch, attachment)]

[0003-testsuite-Use-existing-priority.patch (text/x-patch, attachment)]

[0004-testsuite-genpackage.sh-Silence-output.patch (text/x-patch, attachment)]

[0005-testsuite-Add-basic-shunit2-based-tests.patch (text/x-patch, attachment)]

[0006-Update-auto-generated-files.patch (text/x-patch, attachment)]

[0007-Add-ignores-for-tests-to-.gitignore.patch (text/x-patch, attachment)]

[0008-debian-Switch-to-dh-9.patch (text/x-patch, attachment)]

[0009-Run-shunit2-tests-on-build-time.patch (text/x-patch, attachment)]

[0010-debian-Update-changelog.patch (text/x-patch, attachment)]

[0011-format-Fix-indentation-spaces-to-tabs.patch (text/x-patch, attachment)]

[0012-Evaluate-return-value-of-write-command.patch (text/x-patch, attachment)]

[0013-Print-version-when-removing-a-package.patch (text/x-patch, attachment)]

[0014-refactor-Introduce-static-newcursor-function.patch (text/x-patch, attachment)]

[0015-table_getrecord-Add-newline-to-error-message.patch (text/x-patch, attachment)]

[0016-refactor-Merge-cursor_nexttemp-into-cursor_nexttempd.patch (text/x-patch, attachment)]

[0017-refactor-Introduce-parse_data.patch (text/x-patch, attachment)]

[0018-table_printerror-Improve-database-error-message.patch (text/x-patch, attachment)]

[0019-refactor-database-Move-cursor-struct-upwards.patch (text/x-patch, attachment)]

[0020-refactor-rename-table_newduplicatecursor.patch (text/x-patch, attachment)]

[0021-refactor-Introduce-cursor_next.patch (text/x-patch, attachment)]

[0022-refactor-rename-todo-to-remaining.patch (text/x-patch, attachment)]

[0023-refactor-target_removepackage-Support-specifying-pac.patch (text/x-patch, attachment)]

[0024-refactor-Introduce-splitnameandversion.patch (text/x-patch, attachment)]

[0025-refactor-table_close-Set-default-return-value.patch (text/x-patch, attachment)]

[0026-Add-trace-debugging-output.patch (text/x-patch, attachment)]

[0027-Add-helper-function-debianversioncompare.patch (text/x-patch, attachment)]

[0028-Add-helper-function-get_package_name.patch (text/x-patch, attachment)]

[0029-Add-helper-function-package_primarykey.patch (text/x-patch, attachment)]

[0030-Add-helper-function-cascade_strcmp.patch (text/x-patch, attachment)]

[0031-Add-helper-function-strcmp2.patch (text/x-patch, attachment)]

[0032-Add-helper-function-table_newduplicatecursor.patch (text/x-patch, attachment)]

[0033-package_openiterator-Support-opened-databases.patch (text/x-patch, attachment)]

[0034-Add-helper-function-package_openduplicateiterator.patch (text/x-patch, attachment)]

[0035-Support-listing-multiple-versions-in-ls-command.patch (text/x-patch, attachment)]

[0036-Support-listing-multiple-versions-in-list-command.patch (text/x-patch, attachment)]

[0037-refactor-Introduce-nameandversion-struct-for-copy-co.patch (text/x-patch, attachment)]

[0038-refactor-Use-nameandversion-struct-for-remove-comman.patch (text/x-patch, attachment)]

[0039-Add-multiple-version-management.patch (text/x-patch, attachment)]

[0040-Add-Limit-option.patch (text/x-patch, attachment)]

[0041-testsuite-Add-multiversion-test-cases.patch (text/x-patch, attachment)]

[0042-Update-auto-generated-files.patch (text/x-patch, attachment)]

[0043-fix-tests.patch (text/x-patch, attachment)]

[0044-table_addrecord-Support-non-duplicate-tables.patch (text/x-patch, attachment)]

[0045-Convert-old-database-format-into-new-format.patch (text/x-patch, attachment)]

[0046-Keep-track-of-all-opened-database-tables.patch (text/x-patch, attachment)]

[0047-Add-print_opened_tables-for-debugging-purposes.patch (text/x-patch, attachment)]

[0048-Remove-tracking-of-opened-individual-databases.patch (text/x-patch, attachment)]

[0049-Enhance-copy-functions-parameters-to-support-moving.patch (text/x-patch, attachment)]

[0050-Add-fromtarget-to-struct-target_package_list.patch (text/x-patch, attachment)]

[0051-Add-more-trace-debugging-output.patch (text/x-patch, attachment)]

[0052-Implement-remove-source-support.patch (text/x-patch, attachment)]

[0053-Add-listdistros-command.patch (text/x-patch, attachment)]

[0054-testsuite-Add-test-case-for-listdistros-command.patch (text/x-patch, attachment)]

[0055-Add-move-commands.patch (text/x-patch, attachment)]

[0056-testsuite-Add-test-cases-for-move-command.patch (text/x-patch, attachment)]

[0057-Add-more-debugging-output.patch (text/x-patch, attachment)]

[0058-Add-Archive-option.patch (text/x-patch, attachment)]

[migrate-db-from-4.17-to-5.1 (text/x-python, attachment)]

Message #117 received at 570623@bugs.debian.org (full text, mbox, reply):

From: Johannes Amorosa <johannesa@celluloid-vfx.com>

To: 570623@bugs.debian.org

Subject: Re: Bug#570623: reprepro: please add multiple version management

Date: Thu, 30 Mar 2017 09:43:47 +0200

On 29.03.2017 19:31, Benjamin Drung wrote:
> Hi everyone,
>
> I am happy to announce the feature complete version of reprepro 5.1.1
> with multiple version management. Attached are all 58 patches. You can
> also pull the 5.1.1-mutiple-versions branch from
> https://github.com/profitbricks/reprepro
> Feel free to report bugs on GitHub.
>
> Bernhard, please review my patches and include the patches that you
> like. Some patches are unrelated to the multiple versions feature (e.g.
> patches 11 and 12). Some patches could already be included even when
> you do not apply the final patch. Let me know if you want further
> refactoring.
>
> Release Notes
> =============
>
> The multiple-versions patch set adds following features:
>
> * Add shunit2 based tests (Closes: #857302)
> * Support multiple versions. (Closes: #570623)
> * Add listdistros command (Closes: #857303)
> * Add the commands move, movesrc, movematched, movefilter
> * Add Limit and Archive option
>
> Behavior changes
> ----------------
>
> The multiple-versions reprepro keeps all package versions in the
> archive. Set "Limit: 1" to keep only one version per package in the
> archive to restore the previous behavior.
>
> Database layout changes
> -----------------------
>
> The database layout changes from the upstream 5.1.1 release to the
> multiple versions patch set. The difference is as following:
>
> 5.1.1:
>
> * packages.db maps "package name" to "control file" without duplicates
> * no packagenames.db
>
> 5.1.1 + multiple versions:
>
> * packages.db maps "package name|version" to "control file" without
> duplicates
> * packagenames.db maps "package name" to "package name|version"
> allowing duplicates and duplicates sorted by dpkg --compare-versions
> descending
>
> The multiple-versions reprepro supports migrating from 5.1.1 to
> 5.1.1 + multiple versions. Warning: There is no way back (but could be
> done with a simple Python script)!
>
> Notes for users of 4.16.1 + multiple versions
> ---------------------------------------------
>
> The database layout of the multiple-versions branch changed between
> the 4.17.1 based patch set and the 5.1.1 patch set. Attached is the
> small Python script 'migrate-db-from-4.17-to-5.1' to migrate the
> database to the new layout. You have to install python3-bsddb3 and run
> the script in basedir. It should be idempontent.
>

@Benjamin Great thanks for doing this work. Looking forward to a release 
- it's a feature we're in need.

Greeting from Berlin to Berlin
Johannes

Message #122 received at 570623@bugs.debian.org (full text, mbox, reply):

From: Benjamin Drung <benjamin.drung@profitbricks.com>

To: 570623@bugs.debian.org

Subject: Re: Bug#570623: reprepro: please add multiple version management

Date: Thu, 13 Apr 2017 16:40:40 +0200

[Message part 1 (text/plain, inline)]

Hi everyone,

I found some issues with my patch set from 2017-03-29 which I have
fixed with the new patch set as of yesterday. Attached are all 65
patches as tarball. You can also pull the 5.1.1-mutiple-versions branch
from https://github.com/profitbricks/reprepro
Feel free to report bugs on GitHub.

Release Notes
=============

Changes between 2017-03-29 and 2017-04-12:

* Fix typo "could not found" -> "could not find" 
* Fix crash when Limit and Tracking is set
* Fix logging when using Archive option
* Fix tracking data when using move command
* Fix tracking data with Archive option
* Do not archive newly added packages (for downgrades)
* Use database environment to avoid database corruption
* Fix BDB0073 DB_NOTFOUND at del (when copying existing packages)

The test cases were enhanced to also check the tracking information and
some new tests were added.

In case you experience following error:
BDB0088 DB_SECONDARY_BAD: Secondary index inconsistent with primary
you can use the attached reconstruct_secondarydb Python script to
reconstruct the secondary database. (I had the issues that the sorting
of the secondary database was wrong and this inconsistency was only
revealed by the new Archive and Limit option)

In addition, dump_packages is attached to dump the content of the
packages.db and packagenames.db in a human readable format (useful to
compare the result of reconstruct_secondarydb).

-- 
Benjamin Drung
System Developer
Debian & Ubuntu Developer

ProfitBricks GmbH
Greifswalder Str. 207
D - 10405 Berlin

Email: benjamin.drung@profitbricks.com
URL:  http://www.profitbricks.com

Sitz der Gesellschaft: Berlin.
Registergericht: Amtsgericht Charlottenburg, HRB 125506B.
Geschäftsführer: Andreas Gauger, Achim Weiss.

[dump_packages (text/x-python, attachment)]

[reconstruct_secondarydb (text/x-python, attachment)]

[multiple-versions-patches.tar.xz (application/x-xz-compressed-tar, attachment)]

Message #127 received at 570623@bugs.debian.org (full text, mbox, reply):

From: Benjamin Drung <benjamin.drung@profitbricks.com>

To: 570623@bugs.debian.org

Subject: Re: Bug#570623: reprepro: please add multiple version management

Date: Mon, 03 Sep 2018 13:16:16 +0200

[Message part 1 (text/plain, inline)]

Hi everyone,

I rebased the patch set on top of the 5.2.0-1 release. Attached are all
71 patches as tarball. You can also pull the multiple-versions branch
from https://github.com/profitbricks/reprepro
Feel free to report bugs on GitHub.

Release Notes
=============

Changes between 2017-04-12 and 2018-09-03:

* Rebased on 5.2.0
* Fix missing quotation mark in component list
* Fixed "Package database is not sorted!!!" by update command
  https://github.com/profitbricks/reprepro/issues/6
* Fixed screwing up database names by the database migration
  https://github.com/profitbricks/reprepro/issues/8
* Fix adding the same upstream tarball twice
  https://github.com/profitbricks/reprepro/issues/9
* Fix removesrc when multiple version are available

-- 
Benjamin Drung
System Developer
Debian & Ubuntu Developer

ProfitBricks GmbH
Greifswalder Str. 207
10405 Berlin

Email: benjamin.drung@profitbricks.com
URL: https://www.profitbricks.de

Sitz der Gesellschaft: Berlin
Registergericht: Amtsgericht Charlottenburg, HRB 125506 B
Geschäftsführer: Achim Weiss, Matthias Steinberg, Christoph Steffens

[multiple-versions-patches-5.2.0.tar.xz (application/x-xz-compressed-tar, attachment)]

Message #132 received at 570623@bugs.debian.org (full text, mbox, reply):

From: John Goerzen <jgoerzen@complete.org>

To: 570623@bugs.debian.org, benjamin.drung@profitbricks.com

Subject: Prognosis for integrating multiple version support into reprepro?

Date: Fri, 09 Nov 2018 18:26:35 -0600

Hi folks,

First of all, thanks to all of you for your work on reprepro.  I am
looking at a use case for which reprepro plus the profitbricks multiple
version support looks ideal.  However, as I want to set this up for
long-term success without a lot of fiddling, I'm wondering what the
prognosis is for getting this patch set integrated into reprepro
itself.  I don't see any comments in the bug log after either the 2017
or the 2018 patches, and am just wondering where things are?  Is there
something I could do to help move it along/

Thanks,

John

Message #137 received at 570623@bugs.debian.org (full text, mbox, reply):

From: John Goerzen <jgoerzen@complete.org>

To: 570623@bugs.debian.org, benjamin.drung@profitbricks.com

Subject: Submitted MR on Salsa

Date: Tue, 13 Nov 2018 12:48:48 -0600

Hi folks,

In case it makes things easier, I submitted
https://salsa.debian.org/brlink/reprepro/merge_requests/1 which
represents Benjamin's 5.2.0 patchset, unmodified, with his original
commits from github.  I'm just resubmitting it there in case it makes it
easier to apply.

Thanks,

John

Message #142 received at 570623@bugs.debian.org (full text, mbox, reply):

From: John Goerzen <jgoerzen@complete.org>

To: 570623@bugs.debian.org, benjamin.drung@profitbricks.com

Subject: Re: Submitted MR on Salsa

Date: Tue, 13 Nov 2018 15:41:21 -0600

One additional comment - it would probably be good to default Limit: 1
to mimic prior behavior, wouldn't it?

- John

Message #147 received at 570623@bugs.debian.org (full text, mbox, reply):

From: Benjamin Drung <benjamin.drung@cloud.ionos.com>

To: John Goerzen <jgoerzen@complete.org>, 570623@bugs.debian.org, benjamin.drung@profitbricks.com

Subject: Re: Submitted MR on Salsa

Date: Wed, 14 Nov 2018 16:47:05 +0100

Hi John,

thanks for creating this pull request.

Am Dienstag, den 13.11.2018, 15:41 -0600 schrieb John Goerzen:
> One additional comment - it would probably be good to default
> Limit: 1 to mimic prior behavior, wouldn't it?

I changed the default to "Limit: 1", rebased the patch set on master,
and created a pull request for it:
https://salsa.debian.org/brlink/reprepro/merge_requests/2

-- 
Benjamin Drung
System Developer
Debian & Ubuntu Developer

1&1 IONOS Cloud GmbH | Greifswalder Str. 207 | 10405 Berlin | Germany
E-mail: benjamin.drung@cloud.ionos.com | Web: www.ionos.de

Head Office: Berlin, Germany
District Court Berlin Charlottenburg, Registration number: HRB 125506 B
Executive Management: Christoph Steffens, Matthias Steinberg, Achim
Weiss

Member of United Internet

Message #152 received at 570623@bugs.debian.org (full text, mbox, reply):

From: Vincent Bernat <bernat@debian.org>

To: Debian Bug Tracking System <570623@bugs.debian.org>

Subject: Re: reprepro: please add multiple version management

Date: Sun, 31 Mar 2019 15:12:32 +0200

Package: reprepro
Version: 5.3.0-1
Followup-For: Bug #570623

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256

Hey!

What is the status of this feature? The merge request is not available
anymore and the GitHub repository of the fork is not up-to-date with
5.3.0.

- -- System Information:
Debian Release: buster/sid
  APT prefers unstable-debug
  APT policy: (500, 'unstable-debug'), (500, 'unstable'), (101, 'experimental-debug'), (101, 'experimental')
Architecture: amd64 (x86_64)

Kernel: Linux 4.19.0-4-amd64 (SMP w/4 CPU cores)
Kernel taint flags: TAINT_USER
Locale: LANG=fr_FR.utf8, LC_CTYPE=fr_FR.utf8 (charmap=UTF-8), LANGUAGE=en_US:en (charmap=UTF-8)
Shell: /bin/sh linked to /usr/bin/dash
Init: systemd (via /run/systemd/system)
LSM: AppArmor: enabled

Versions of packages reprepro depends on:
ii  libarchive13   3.3.3-4
ii  libbz2-1.0     1.0.6-9
ii  libc6          2.28-8
ii  libdb5.3       5.3.28+dfsg1-0.6
ii  libgpg-error0  1.35-1
ii  libgpgme11     1.12.0-6
ii  liblzma5       5.2.4-1
ii  zlib1g         1:1.2.11.dfsg-1

Versions of packages reprepro recommends:
ii  apt  1.8.0

Versions of packages reprepro suggests:
ii  gnupg-agent              2.2.13-1
ii  gpg-agent [gnupg-agent]  2.2.13-1
pn  inoticoming              <none>
pn  lzip                     <none>
ii  pinentry-curses          1.1.0-1+b1

- -- no debconf information

-----BEGIN PGP SIGNATURE-----

iQJGBAEBCAAwFiEErvI0h2bzccaJpzYAlaQv6DU1JfkFAlygvLwSHGJlcm5hdEBk
ZWJpYW4ub3JnAAoJEJWkL+g1NSX5WUIP/3DY63Dao5HqvaXyiVG/Q1jLsn2YC7re
6miL77z5sDLHgtcbn8U2ObIdUvbnHvA7+4hvy6wl5lInxsHV0N5RlaeemiQ3LbYI
WVAYCgGbZNEy+80+sQ/1TorIn+Q/uTAeKMNrdnUT2hzuPTR7974mCdTBRI4a5ZXa
w1my8vBWMMHEgcX7hvcZ8gE/GujQF9HXL8SpA8n2yS4K1+9RoCUdWiPulbpYXc43
hSc0dgZ3mpIej7vJt5AtthtHhZssmMKDCW+43ciAO7ZlUMrG99Oji9q3GR1LTrkY
oIOSY6FVDm7pD7gbR+BRTfi1g1GP7uNCnWWVcntzgVzOZYJvbeEy6Wq3UY+MJhc0
DwX2aDYH8ehn/TyWWfNiKwUZcrOPDzK+Ki1iFQXJIYrnNqmdHVql9fsN8zh3ODPx
i418tDz9kXHs3LISJxA3f43I+53CpLL4OJQJKzu7VEfvz0F3RyQA27Sd761Xtjaq
twEsdYj9eXZbpmE41yIZl8cMK3vMGMFbDe7G7iOaAQPV/M7TL5LLa8S2fybGwNjS
St70JvMZUO7sQsh4nE9KW/nzeA2f+il5/jA6gGJ6fHEU6pIjklMCM2YcSdXkG4sk
NgNTkdL4pe+xbL+Wjjkc9fGhwpLvKFT+NROwrj7UHAfwY8xBR54x6tobRkhJPymH
/sllHO6/dL4l
=nCAm
-----END PGP SIGNATURE-----

Message #157 received at 570623@bugs.debian.org (full text, mbox, reply):

From: Benjamin Drung <benjamin.drung@cloud.ionos.com>

To: Vincent Bernat <bernat@debian.org>, 570623@bugs.debian.org

Subject: Re: Bug#570623: reprepro: please add multiple version management

Date: Wed, 03 Apr 2019 12:11:55 +0200

Am Sonntag, den 31.03.2019, 15:12 +0200 schrieb Vincent Bernat:
> Package: reprepro
> Version: 5.3.0-1
> Followup-For: Bug #570623
> 
> What is the status of this feature? The merge request is not
> available anymore

It seems that the support for pull requests were disabled.

>  and the GitHub repository of the fork is not up-to-date with
> 5.3.0.

I refreshed the patches on top of version 5.3.0 and pushed it to 
https://salsa.debian.org/bdrung/reprepro/ and 
https://github.com/profitbricks/reprepro

-- 
Benjamin Drung
System Developer
Debian & Ubuntu Developer

1&1 IONOS Cloud GmbH | Greifswalder Str. 207 | 10405 Berlin | Germany
E-mail: benjamin.drung@cloud.ionos.com | Web: www.ionos.de

Head Office: Berlin, Germany
District Court Berlin Charlottenburg, Registration number: HRB 125506 B
Executive Management: Christoph Steffens, Matthias Steinberg, Achim
Weiss

Member of United Internet

Message #162 received at 570623@bugs.debian.org (full text, mbox, reply):

From: Michael Prokop <mika@debian.org>

To: Benjamin Drung <benjamin.drung@cloud.ionos.com>, 570623@bugs.debian.org

Cc: "Bernhard R. Link" <brlink@debian.org>

Subject: Re: Bug#570623: reprepro: please add multiple version management

Date: Wed, 3 Jul 2019 11:59:23 +0200

[Message part 1 (text/plain, inline)]

Hi,

* Benjamin Drung [Wed Apr 03, 2019 at 12:11:55PM +0200]:

> I refreshed the patches on top of version 5.3.0 and pushed it to 
> https://salsa.debian.org/bdrung/reprepro/ and 
> https://github.com/profitbricks/reprepro

Thanks for your efforts, Benjamin, highly appreciated.

Bernhard, is there any specific reason why you didn't respond here
any longer? Is there anything which would help to increase chances
to get this feature merged in the near future?

regards
-mika-

[signature.asc (application/pgp-signature, inline)]

Message #167 received at 570623@bugs.debian.org (full text, mbox, reply):

From: Michael Prokop <mika@debian.org>

To: "Bernhard R. Link" <brlink@debian.org>, 570623@bugs.debian.org

Cc: Benjamin Drung <benjamin.drung@cloud.ionos.com>

Subject: Re: Bug#570623: reprepro: please add multiple version management

Date: Tue, 24 Mar 2020 15:36:39 +0100

[Message part 1 (text/plain, inline)]

Hi,

* Michael Prokop [Wed Jul 03, 2019 at 11:59:23AM +0200]:
> * Benjamin Drung [Wed Apr 03, 2019 at 12:11:55PM +0200]:

> > I refreshed the patches on top of version 5.3.0 and pushed it to 
> > https://salsa.debian.org/bdrung/reprepro/ and 
> > https://github.com/profitbricks/reprepro

> Thanks for your efforts, Benjamin, highly appreciated.

> Bernhard, is there any specific reason why you didn't respond here
> any longer? Is there anything which would help to increase chances
> to get this feature merged in the near future?

Ping. :) Any news here?

regards
-mika-

[signature.asc (application/pgp-signature, inline)]

Message #172 received at 570623@bugs.debian.org (full text, mbox, reply):

From: Marek Marczykowski-Górecki <marmarek@invisiblethingslab.com>

To: 570623@bugs.debian.org

Subject: Re: Bug#570623: reprepro: please add multiple version management

Date: Fri, 8 Jan 2021 21:13:48 +0100

[Message part 1 (text/plain, inline)]

On Tue, 24 Mar 2020 15:36:39 +0100 Michael Prokop <mika@debian.org> wrote:
> Hi,
> 
> * Michael Prokop [Wed Jul 03, 2019 at 11:59:23AM +0200]:
> > * Benjamin Drung [Wed Apr 03, 2019 at 12:11:55PM +0200]:
> 
> > > I refreshed the patches on top of version 5.3.0 and pushed it to 
> > > https://salsa.debian.org/bdrung/reprepro/ and 
> > > https://github.com/profitbricks/reprepro
> 
> > Thanks for your efforts, Benjamin, highly appreciated.

I have tested the above branch (mostly 'include' and 'export' commands)
and generally it works fine. I've found one weird corner case related to
updating from older reprepro (5.1.1, without the patches), possibly db
format:
 - packages that were in the db before update, are no longer included in
   generated metadata ('export' command), unless added again with
   'include' command
 - 'dumpreferences' _does_ include those missing packages

It may be also that I'm missing some explicit db conversion step (I
haven't done anything, just updated reprepro package).

-- 
Best Regards,
Marek Marczykowski-Górecki
Invisible Things Lab
A: Because it messes up the order in which people normally read text.
Q: Why is top-posting such a bad thing?

[signature.asc (application/pgp-signature, inline)]

Message #177 received at 570623@bugs.debian.org (full text, mbox, reply):

From: Martina Francis <martinafrancis531@gmail.com>

To: undisclosed-recipients:;

Subject: Dobry dzień moja droga

Date: Thu, 21 Oct 2021 09:32:20 -0700

-- 
Dobry dzień moja droga
Jak się masz i twoja rodzina.
Jestem pani Martina Francis, chora wdowa pisząca ze szpitalnego łóżka
bez dziecka. Kontaktuję się z Państwem, abyście dowiedzieli się o moim
pragnieniu przekazania sumy (2 700 000,00 USD MILIONÓW USD), którą
odziedziczyłam po moim zmarłym mężu na cele charytatywne, obecnie
fundusz jest nadal w banku. Niedawno mój lekarz powiedział mi, że mam
poważną chorobę nowotworową i moje życie nie jest już gwarantowane,
dlatego podejmuję tę decyzję..

Chcę, abyście skorzystali z tego funduszu dla ludzi ubogich,
maltretowanych dzieci, mniej uprzywilejowanych, kościołów, sierocińców
i cierpiących wdów w społeczeństwie.

Proszę, wróć do mnie natychmiast po przeczytaniu tej wiadomości, aby
uzyskać więcej szczegółów dotyczących tej agendy humanitarnej. A także
wyślij mi następujące informacje, jak poniżej.

Twoje pełne imię ..........
Adres ...........
Zdjęcie ...............

Niech Bóg cię błogosławi, kiedy czekam na twoją odpowiedź.

Twoja siostra.
Pani Martina Francis.

Message #182 received at 570623@bugs.debian.org (full text, mbox, reply):

From: Kodiak Firesmith <kfiresmith@whoi.edu>

To: "570623@bugs.debian.org" <570623@bugs.debian.org>

Subject: Re: Bug#570623: reprepro: please add multiple version management

Date: Mon, 30 May 2022 21:19:33 +0000

[Message part 1 (text/plain, inline)]

I’d love to see this merged.  It looks like Benjamin Drung put an absolute
mountain of work into getting this ready to merge, and that a lot of folks
have wanted this feature over the years.  

After more than 170 messages it’d be a shame to just see this feature die on
the vine.

- Kodiak

On Fri, 8 Jan 2021 21:13:48 +0100 Marek
=?utf-8?Q?Marczykowski-G=C3=B3recki?= <marmarek@invisiblethingslab.com>
wrote: 
> On Tue, 24 Mar 2020 15:36:39 +0100 Michael Prokop <mika@debian.org> wrote:

> > Hi, 
> > 
> > * Michael Prokop [Wed Jul 03, 2019 at 11:59:23AM +0200]: 
> > > * Benjamin Drung [Wed Apr 03, 2019 at 12:11:55PM +0200]: 
> > 
> > > > I refreshed the patches on top of version 5.3.0 and pushed it to 
> > > > https://salsa.debian.org/bdrung/reprepro/ and 
> > > > https://github.com/profitbricks/reprepro 
> > 
> > > Thanks for your efforts, Benjamin, highly appreciated. 
> 
> I have tested the above branch (mostly 'include' and 'export' commands) 
> and generally it works fine. I've found one weird corner case related to 
> updating from older reprepro (5.1.1, without the patches), possibly db 
> format: 
>  - packages that were in the db before update, are no longer included in 
>    generated metadata ('export' command), unless added again with 
>    'include' command 
>  - 'dumpreferences' _does_ include those missing packages 
> 
> It may be also that I'm missing some explicit db conversion step (I 
> haven't done anything, just updated reprepro package). 
> 
> -- 
> Best Regards, 
> Marek Marczykowski-Górecki 
> Invisible Things Lab 
> A: Because it messes up the order in which people normally read text. 
> Q: Why is top-posting such a bad thing? 

[Message part 2 (text/html, inline)]

[smime.p7s (application/pkcs7-signature, attachment)]

Message #187 received at 570623@bugs.debian.org (full text, mbox, reply):

From: Bastian Germann <bage@debian.org>

To: 570623@bugs.debian.org

Subject: Re: Bug#570623: reprepro: please add multiple version management

Date: Wed, 27 Jul 2022 12:50:51 +0200

I am about to upload an experimental version with Benjamin's changes.
Please open new bugs for the issues that you may have with the new version even if they are already mentioned here.

Message #192 received at 570623-close@bugs.debian.org (full text, mbox, reply):

From: Debian FTP Masters <ftpmaster@ftp-master.debian.org>

To: 570623-close@bugs.debian.org

Subject: Bug#570623: fixed in reprepro 5.4.0-1

Date: Wed, 27 Jul 2022 11:20:58 +0000

Source: reprepro
Source-Version: 5.4.0-1
Done: Bastian Germann <bage@debian.org>

We believe that the bug you reported is fixed in the latest version of
reprepro, which is due to be installed in the Debian FTP archive.

A summary of the changes between this version and the previous one is
attached.

Thank you for reporting the bug, which will now be closed.  If you
have further comments please address them to 570623@bugs.debian.org,
and the maintainer will reopen the bug report if appropriate.

Debian distribution maintenance software
pp.
Bastian Germann <bage@debian.org> (supplier of updated reprepro package)

(This message was generated automatically at their request; if you
believe that there is a problem with it please contact the archive
administrators by mailing ftpmaster@ftp-master.debian.org)


-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA512

Format: 1.8
Date: Wed, 27 Jul 2022 12:43:10 +0200
Source: reprepro
Architecture: source
Version: 5.4.0-1
Distribution: experimental
Urgency: medium
Maintainer: Bastian Germann <bage@debian.org>
Changed-By: Bastian Germann <bage@debian.org>
Closes: 570623 730572 857302
Changes:
 reprepro (5.4.0-1) experimental; urgency=medium
 .
   * Import new upstream version 5.4.0
     (Closes: #570623, #857302, #730572, LP: #799889)
   * Add ISC license for new test files
   * Run shunit2 tests on build time
Checksums-Sha1:
 825523930c3bde6fe32f43aa6e3321bfb1a41181 1757 reprepro_5.4.0-1.dsc
 982a0b97e1f2d88667e17d2991b1873dc01ade55 458700 reprepro_5.4.0.orig.tar.xz
 bf4e09c5d5897a7ae33bf5a922a52a7a2b1810bf 14608 reprepro_5.4.0-1.debian.tar.xz
 4ed2d1144f04a4a6c697e32fa158e6e139148f92 10211 reprepro_5.4.0-1_source.buildinfo
Checksums-Sha256:
 a9c3ea5c7244063ccf4d243a7fefb166a4d4980960c78cb5f24564556f27b939 1757 reprepro_5.4.0-1.dsc
 4e32f728b0053d28f832d1b51d07577f85b364a2bca44bdd573fb9024081da28 458700 reprepro_5.4.0.orig.tar.xz
 101d1ed858c717a9632b9cd41d1a1990cd5c736426f64f2e9bc4543d853cfba4 14608 reprepro_5.4.0-1.debian.tar.xz
 a71aa2fbc102d6aa7d8184b92c46d2fbe56ec998108fd53bd31d9eca34d1325c 10211 reprepro_5.4.0-1_source.buildinfo
Files:
 25eba5c2a04c2cfac5401400f1ed76ba 1757 utils optional reprepro_5.4.0-1.dsc
 eba99a3ed4c37477f47c4724608c2f9b 458700 utils optional reprepro_5.4.0.orig.tar.xz
 2e1e377dd10cd4342d052d6527cbec4f 14608 utils optional reprepro_5.4.0-1.debian.tar.xz
 71676cf7c8cbd6d2382e9c5d864248c3 10211 utils optional reprepro_5.4.0-1_source.buildinfo

-----BEGIN PGP SIGNATURE-----

iQHEBAEBCgAuFiEEQGIgyLhVKAI3jM5BH1x6i0VWQxQFAmLhGRgQHGJhZ2VAZGVi
aWFuLm9yZwAKCRAfXHqLRVZDFCx8DACFvm3YEwNELSZb4Y7ndDEScuNUbKKf/hYQ
RrxYZz+3oWpF/NFmRp/Y9jclYZVzn3LzXvaL6Hp3gb5wIwLKfuoapM6g+m3zeqvE
PwkMHEyB5R4megkqcIdGD3q+HgAEC5HO92xfkukgdf3UYG3WZ9DZ7t98RtkcvN1z
x+uNPiBda3A1akxku/3YhrUJ1YK4buYwa11XkruuYcjJHh6NspLojiDKGpCmJQjL
O3Xoonfo8Uyfwtg3IQvRTFpwDF9c2GNReyf4JS1MxAEoC30P4axMcammNRI7k9Of
hj3V55RgptDbowLYWUXxuX2UfddjzodmoZiYF1pxq+2Hz1anl8Ta5+a51uMxr3Vh
kLLLCZA4uQCQ3msSIUeVAhNr5J6+3BDJ52XRAg4/SpZK6rX9P5OwA4lIEwSwhImW
2jToHuu6EMhdkLwa7E6EXvMQwdRHmZolEalL3bakxBlBohNv560Ff4Feo8Pj1beA
7C/jhcjwXU3tltiTU/w2a6MCYYp/ZFY=
=80ap
-----END PGP SIGNATURE-----

Send a report that this bug log contains spam.