Debian Bug report logs -
#671503
document APT repository format
Toggle useless messages
Report forwarded
to debian-bugs-dist@lists.debian.org, hramrach@gmail.com, debian-devel@lists.debian.org:
Bug#671503; Package general.
(Fri, 04 May 2012 16:51:04 GMT) (full text, mbox, link).
Acknowledgement sent
to Michal Suchanek <michal.suchanek@ruk.cuni.cz>:
New Bug report received and forwarded. Copy sent to hramrach@gmail.com, debian-devel@lists.debian.org.
(Fri, 04 May 2012 16:51:04 GMT) (full text, mbox, link).
Message #5 received at submit@bugs.debian.org (full text, mbox, reply):
Package: general
Severity: important
Hello,
I wanted to create a repository of my own packages so that I can use the
standard Debian tools to install these packages and resolve any
dependencies automatically.
However, there is no documentation of the format of these repositories.
There are multiple tools dealing with these.
apt (apt-get) downloads packages, and indirectly through apt libraries
aptitude and other tools do also.
dpkg-scanpackages and apt-ftparchive are supposed aids in creating such
repositories apt can download from.
The man pages of neither apt-get nor dpkg-scanpackages nor
apt-ftparchive document the repository format.
By reverse-engineering a Debian mirror site I managed to write a script
using dpkg-scanpackages that created a repository that apt could use.
Unfortunately, the undocumented format of apt archives has changed and I
had to reverse-engineer a Debian mirror again to figure out what has
changed.
After reporting this as a bug it was recommended that I use
apt-ftparchive instead so that I don't have to deal with the details of
this undocumented format myself.
Alas, what I feared was true.
dpkg-scanpackages is a simplistic tool so it is possible to use its
output even in the face of lack of documentation.
This, however, does not apply the apt-ftparchive. It is supposed to
create the required files fully automatically. With the provided
documentation I was able to make it do exactly nothing, fully
automatically.
I would appreciate if Debian fully decumented its core tools like
package manager.
Thanks
Michal
-- System Information:
Debian Release: wheezy/sid
APT prefers stable
APT policy: (900, 'stable'), (500, 'testing'), (410, 'unstable'), (200, 'experimental'), (111, 'oldstable')
Architecture: amd64 (x86_64)
Kernel: Linux 3.2.0-2-amd64 (SMP w/4 CPU cores)
Locale: LANG=en_US.UTF-8, LC_CTYPE=en_US.UTF-8 (charmap=UTF-8) (ignored: LC_ALL set to en_US.UTF-8)
Shell: /bin/sh linked to /bin/bash
Information forwarded
to debian-bugs-dist@lists.debian.org, debian-devel@lists.debian.org:
Bug#671503; Package general.
(Fri, 04 May 2012 17:18:07 GMT) (full text, mbox, link).
Acknowledgement sent
to Neil Williams <codehelp@debian.org>:
Extra info received and forwarded to list. Copy sent to debian-devel@lists.debian.org.
(Fri, 04 May 2012 17:18:07 GMT) (full text, mbox, link).
Message #10 received at 671503@bugs.debian.org (full text, mbox, reply):
[Message part 1 (text/plain, inline)]
On Fri, 04 May 2012 18:49:34 +0200
Michal Suchanek <michal.suchanek@ruk.cuni.cz> wrote:
> Package: general
> Severity: important
>
> I wanted to create a repository of my own packages so that I can use the
> standard Debian tools to install these packages and resolve any
> dependencies automatically.
Use better tools - reprepro which has very good documentation for it's
usage but there is no real reason for documentation of the output
format itself.
> By reverse-engineering a Debian mirror site I managed to write a script
> using dpkg-scanpackages that created a repository that apt could use.
Please don't re-invent wheels like that again - it's not necessary.
> Unfortunately, the undocumented format of apt archives has changed and I
> had to reverse-engineer a Debian mirror again to figure out what has
> changed.
reprepro repositories keep working. It's NOT necessary to exactly mimic
an existing mirror in order to have a working repository.
> After reporting this as a bug it was recommended that I use
> apt-ftparchive instead so that I don't have to deal with the details of
> this undocumented format myself.
Exactly, use reprepro.
> This, however, does not apply the apt-ftparchive. It is supposed to
> create the required files fully automatically. With the provided
> documentation I was able to make it do exactly nothing, fully
> automatically.
Example conf to get a simple reprepro repository (save it as
conf/distributions):
Origin: Debian
Suite: local
Codename: test
Architectures: i386
Components: main
$ reprepro export
Job done.
--
Neil Williams
=============
http://www.linux.codehelp.co.uk/
[Message part 2 (application/pgp-signature, inline)]
Information forwarded
to debian-bugs-dist@lists.debian.org, debian-devel@lists.debian.org:
Bug#671503; Package general.
(Fri, 04 May 2012 21:06:54 GMT) (full text, mbox, link).
Acknowledgement sent
to David Kalnischkies <kalnischkies@gmail.com>:
Extra info received and forwarded to list. Copy sent to debian-devel@lists.debian.org.
(Fri, 04 May 2012 21:06:58 GMT) (full text, mbox, link).
Message #15 received at 671503@bugs.debian.org (full text, mbox, reply):
On Fri, May 4, 2012 at 6:49 PM, Michal Suchanek
<michal.suchanek@ruk.cuni.cz> wrote:
> This, however, does not apply the apt-ftparchive. It is supposed to
> create the required files fully automatically. With the provided
> documentation I was able to make it do exactly nothing, fully
> automatically.
For the record: This was reported in a "very friendly" way in #671496.
With the correct configuration (which wouldn't be that much harder
to write than one for reprepro) it would work as described in manpage
and example files, but i agree with Neil that better tools exist -
for this usecase at least.
Completely ignoring the mail itself and just referring to the title
(beside ignoring even the first word in that):
"repository format is not documented" is a valid bug - and it should be
documented for the benefit of people who write the various tools used to
generate repository data.
The added benefit would be that we would have a central point where
changes/additions can be discussed to avoid problems like we had with
the introduction of InRelease or Translation-en which either aren't
supported or implemented differently in different tools because nobody
knows who the authority is.
Currently many (not all) of the discussions with this topic end up on
various mailinglists/bugtrackers associated to APT as the lowest common
denominator, but this usually means that a lot of people who should be
aren't in the loop ending in the previous mentioned problems.
I would personal tend toward ftp-master to be the authority with reference
implementation being dak, but they have no public mailinglist and dak isn't
used by all derivatives…
Best regards
David Kalnischkies
Information forwarded
to debian-bugs-dist@lists.debian.org, debian-devel@lists.debian.org:
Bug#671503; Package general.
(Fri, 04 May 2012 21:15:19 GMT) (full text, mbox, link).
Acknowledgement sent
to Russ Allbery <rra@debian.org>:
Extra info received and forwarded to list. Copy sent to debian-devel@lists.debian.org.
(Fri, 04 May 2012 21:15:20 GMT) (full text, mbox, link).
Message #20 received at 671503@bugs.debian.org (full text, mbox, reply):
David Kalnischkies <kalnischkies@gmail.com> writes:
> Completely ignoring the mail itself and just referring to the title
> (beside ignoring even the first word in that): "repository format is
> not documented" is a valid bug - and it should be documented for the
> benefit of people who write the various tools used to generate
> repository data.
> The added benefit would be that we would have a central point where
> changes/additions can be discussed to avoid problems like we had with
> the introduction of InRelease or Translation-en which either aren't
> supported or implemented differently in different tools because nobody
> knows who the authority is.
> Currently many (not all) of the discussions with this topic end up on
> various mailinglists/bugtrackers associated to APT as the lowest common
> denominator, but this usually means that a lot of people who should be
> aren't in the loop ending in the previous mentioned problems.
> I would personal tend toward ftp-master to be the authority with
> reference implementation being dak, but they have no public mailinglist
> and dak isn't used by all derivatives…
I think debian-policy is the right repository for this. I think it would
make the most sense to maintain this via a looser update method than the
normal Policy process and to instead just apply any update that ftp-master
says is in place, since the decision-making is already handled through
ftp-master and other discussions and there's not much to be gained by an
additional decision process.
I would be happy to maintain this as a sub-Policy as part of the
debian-policy package and (subject to my always insufficient time
available) be happy to handle the mechanics of formatting and wordsmithing
updates, but don't personally have time to write the original text or to
investigate and document changes.
A prerequisite for this working properly would be for ftp-master to feel
like they could send updates to somewhere and have them be absorbed and
handled by someone. I don't think it's going to be viable to add having
to write the documentation as yet another ftp-master task, nor do I think
it would be viable to expect someone outside the group making changes to
reverse-engineer and guess at the changes. It would need to be a
cooperation between notification of changes from the folks working on them
and someone doing the wordsmithing and turning that into a document.
--
Russ Allbery (rra@debian.org) <http://www.eyrie.org/~eagle/>
Information forwarded
to debian-bugs-dist@lists.debian.org, debian-devel@lists.debian.org:
Bug#671503; Package general.
(Sat, 05 May 2012 00:57:03 GMT) (full text, mbox, link).
Acknowledgement sent
to Lisandro Damián Nicanor Pérez Meyer <perezmeyer@gmail.com>:
Extra info received and forwarded to list. Copy sent to debian-devel@lists.debian.org.
(Sat, 05 May 2012 00:57:03 GMT) (full text, mbox, link).
Message #25 received at 671503@bugs.debian.org (full text, mbox, reply):
[Message part 1 (text/plain, inline)]
On Vie 04 May 2012 18:13:01 Russ Allbery escribió:
[big snip]
I think Russ' proposal is quite a nice solution.
Kinds regards, Lisandro.
--
The volume of a pizza of thickness a and radius z can be described by
the following formula:
pi zz a
Lisandro Damián Nicanor Pérez Meyer
http://perezmeyer.com.ar/
http://perezmeyer.blogspot.com/
[signature.asc (application/pgp-signature, inline)]
Bug reassigned from package 'general' to 'debian-policy'.
Request was from Holger Levsen <holger@layer-acht.org>
to control@bugs.debian.org.
(Sat, 05 May 2012 18:18:03 GMT) (full text, mbox, link).
Information forwarded
to debian-bugs-dist@lists.debian.org, Debian Policy List <debian-policy@lists.debian.org>:
Bug#671503; Package debian-policy.
(Sat, 05 May 2012 19:18:06 GMT) (full text, mbox, link).
Acknowledgement sent
to Joerg Jaspert <joerg@debian.org>:
Extra info received and forwarded to list. Copy sent to Debian Policy List <debian-policy@lists.debian.org>.
(Sat, 05 May 2012 19:18:06 GMT) (full text, mbox, link).
Message #32 received at 671503@bugs.debian.org (full text, mbox, reply):
On 12836 March 1977, David Kalnischkies wrote:
> I would personal tend toward ftp-master to be the authority with reference
> implementation being dak, but they have no public mailinglist and dak isn't
> used by all derivatives…
debian-dak@lists.d.o
On 12836 March 1977, Russ Allbery wrote:
> I would be happy to maintain this as a sub-Policy as part of the
> debian-policy package and (subject to my always insufficient time
> available) be happy to handle the mechanics of formatting and wordsmithing
> updates, but don't personally have time to write the original text or to
> investigate and document changes.
> A prerequisite for this working properly would be for ftp-master to feel
> like they could send updates to somewhere and have them be absorbed and
> handled by someone. I don't think it's going to be viable to add having
> to write the documentation as yet another ftp-master task, nor do I think
> it would be viable to expect someone outside the group making changes to
> reverse-engineer and guess at the changes. It would need to be a
> cooperation between notification of changes from the folks working on them
> and someone doing the wordsmithing and turning that into a document.
We could deal with that, yes.
--
bye, Joerg
1. 0 2. 1 3. 2 4. 3 5. 4 6. 5 7. 6 8. 7
|-) What sort of FTP proxy firewall do you have?
-- libnet-perl 1.16-1
Information forwarded
to debian-bugs-dist@lists.debian.org, Debian Policy List <debian-policy@lists.debian.org>:
Bug#671503; Package debian-policy.
(Sun, 06 May 2012 03:21:03 GMT) (full text, mbox, link).
Acknowledgement sent
to Paul Wise <pabs@debian.org>:
Extra info received and forwarded to list. Copy sent to Debian Policy List <debian-policy@lists.debian.org>.
(Sun, 06 May 2012 03:21:05 GMT) (full text, mbox, link).
Message #37 received at 671503@bugs.debian.org (full text, mbox, reply):
On Sat, May 5, 2012 at 5:13 AM, Russ Allbery wrote:
> I think debian-policy is the right repository for this. I think it would
> make the most sense to maintain this via a looser update method than the
> normal Policy process and to instead just apply any update that ftp-master
> says is in place, since the decision-making is already handled through
> ftp-master and other discussions and there's not much to be gained by an
> additional decision process.
As someone who has been working on validating the apt repository
metadata from our derivatives, I would greatly appreciate it if
debian-policy documented that metadata, especially the Release files.
--
bye,
pabs
http://wiki.debian.org/PaulWise
Message sent on
to Michal Suchanek <michal.suchanek@ruk.cuni.cz>:
Bug#671503.
(Wed, 16 May 2012 16:48:08 GMT) (full text, mbox, link).
Message #40 received at 671503-submitter@bugs.debian.org (full text, mbox, reply):
Could you clarify how this differs from #481129?
Message sent on
to Michal Suchanek <michal.suchanek@ruk.cuni.cz>:
Bug#671503.
(Thu, 17 May 2012 11:52:42 GMT) (full text, mbox, link).
Message #43 received at 671503-submitter@bugs.debian.org (full text, mbox, reply):
Excerpts from Filipus Klutiero's message of Wed May 16 18:44:21 +0200 2012:
> Could you clarify how this differs from #481129?
It's 4 years later.
Sorry, forgot that I filed the bug already. It's quite some time.
Given there is no feedback in 4 years I guess it is futile reporting
this.
Admittedly there is no text in social contract about using
Debian-proprietary formats. And a format only defined by "apt can read
that" is definitely Debian-proprietary there is no better term for that.
I'd say it's slightly discriminatory against software not part of Debian
that cannot rely on getting notified when "apt can read that" silently
changes, there is no document defining what apt should be able to read
that software authors can rely on to interoperate with apt, one of the
core Debian tools. Apt in turn relies on open standards like HTTP and
FTP to interoperate with the rest of the world.
Thanks
Michal
Message sent on
to Michal Suchanek <michal.suchanek@ruk.cuni.cz>:
Bug#671503.
(Thu, 17 May 2012 12:58:00 GMT) (full text, mbox, link).
Message #46 received at 671503-submitter@bugs.debian.org (full text, mbox, reply):
Michal Suchanek writes ("Re: Bug#671503: general: APT repository format is not documented"):
> Excerpts from Filipus Klutiero's message of Wed May 16 18:44:21 +0200 2012:
> > Could you clarify how this differs from #481129?
>
> It's 4 years later.
>
> Sorry, forgot that I filed the bug already. It's quite some time.
>
> Given there is no feedback in 4 years I guess it is futile reporting
> this.
Well, it's useful to bring it up again.
> Admittedly there is no text in social contract about using
> Debian-proprietary formats. And a format only defined by "apt can read
> that" is definitely Debian-proprietary there is no better term for that.
Everyone agrees that it would be better if this were documented.
(I have struggled on occasion myself due to the lack of
documentation.)
But I think the use of the word "proprietary" is going too far. It's
certainly a special Debian format, but that wouldn't be changed if it
were documented. But it's not secret and we publish at least two
writer implementations and one reader implementation AFAIK, with
proper Free licences.
> I'd say it's slightly discriminatory against software not part of Debian
> that cannot rely on getting notified when "apt can read that" silently
> changes, there is no document defining what apt should be able to read
> that software authors can rely on to interoperate with apt, one of the
> core Debian tools. Apt in turn relies on open standards like HTTP and
> FTP to interoperate with the rest of the world.
I think this is not an appropriate use of the social contract or its
concepts.
Rather than complaining that this documentation doesn't exist, how
about writing the document yourself ? It's not a trivial job but it
should be feasible by looking at the apt source code.
Once such a document exists, even if it's a bit sketchy or perhaps not
entirely accurate, it will be much easier to insist that future
changes are likewise documented.
Thanks,
Ian.
Message sent on
to Michal Suchanek <michal.suchanek@ruk.cuni.cz>:
Bug#671503.
(Thu, 17 May 2012 13:21:19 GMT) (full text, mbox, link).
Message #49 received at 671503-submitter@bugs.debian.org (full text, mbox, reply):
Excerpts from Ian Jackson's message of Thu May 17 14:53:30 +0200 2012:
> Michal Suchanek writes ("Re: Bug#671503: general: APT repository format is not documented"):
> > Excerpts from Filipus Klutiero's message of Wed May 16 18:44:21 +0200 2012:
> > > Could you clarify how this differs from #481129?
> >
> > It's 4 years later.
> >
> > Sorry, forgot that I filed the bug already. It's quite some time.
> >
> > Given there is no feedback in 4 years I guess it is futile reporting
> > this.
>
> Well, it's useful to bring it up again.
>
> > Admittedly there is no text in social contract about using
> > Debian-proprietary formats. And a format only defined by "apt can read
> > that" is definitely Debian-proprietary there is no better term for that.
>
> Everyone agrees that it would be better if this were documented.
> (I have struggled on occasion myself due to the lack of
> documentation.)
>
> But I think the use of the word "proprietary" is going too far. It's
> certainly a special Debian format, but that wouldn't be changed if it
> were documented. But it's not secret and we publish at least two
> writer implementations and one reader implementation AFAIK, with
> proper Free licences.
However, it's easier to reverse-engineer an existing repository than
the source code so for all practical purposes it's the same as if it
were closed source.
>
> > I'd say it's slightly discriminatory against software not part of Debian
> > that cannot rely on getting notified when "apt can read that" silently
> > changes, there is no document defining what apt should be able to read
> > that software authors can rely on to interoperate with apt, one of the
> > core Debian tools. Apt in turn relies on open standards like HTTP and
> > FTP to interoperate with the rest of the world.
>
> I think this is not an appropriate use of the social contract or its
> concepts.
>
> Rather than complaining that this documentation doesn't exist, how
> about writing the document yourself ? It's not a trivial job but it
> should be feasible by looking at the apt source code.
For me it is not feasible at all.
I can, of course, describe what current repositories look like or what
the current apt code accepts.
However, that has silently changed in the past and is considered apt
feature, not a bug.
>
> Once such a document exists, even if it's a bit sketchy or perhaps not
> entirely accurate, it will be much easier to insist that future
> changes are likewise documented.
I am not so sure about that.
So long as the document merely describes what apt happens to do at the
moment rather than apt implementing what the document says there is no
saying this document has any value.
The status was 'documented' by existing repositories which stopped
working.
Thanks
Michal
Message sent on
to Michal Suchanek <michal.suchanek@ruk.cuni.cz>:
Bug#671503.
(Thu, 17 May 2012 13:24:13 GMT) (full text, mbox, link).
Message #52 received at 671503-submitter@bugs.debian.org (full text, mbox, reply):
Hello,
On 2012-05-17 13:48, Michal Suchanek wrote:
> Admittedly there is no text in social contract about using
> Debian-proprietary formats. And a format only defined by "apt can read
> that" is definitely Debian-proprietary there is no better term for that.
>
> I'd say it's slightly discriminatory against software not part of Debian
> that cannot rely on getting notified when "apt can read that" silently
> changes, there is no document defining what apt should be able to read
> that software authors can rely on to interoperate with apt, one of the
> core Debian tools. Apt in turn relies on open standards like HTTP and
> FTP to interoperate with the rest of the world.
As someone who had to reverse-engineer APT repository format I fully
agree with the above. With one minor addition that some software which
is (non-core) part of Debian suffer from the same problem.
--
Eugene V. Lyubimkin aka JackYF, JID: jackyf.devel(maildog)gmail.com
C++/Perl developer, Debian Developer
Message sent on
to Michal Suchanek <michal.suchanek@ruk.cuni.cz>:
Bug#671503.
(Thu, 17 May 2012 16:51:20 GMT) (full text, mbox, link).
Message #55 received at 671503-submitter@bugs.debian.org (full text, mbox, reply):
reassign 481129 debian-policy
merge 481129 671503
thanks
On 2012-05-17 07:48, Michal Suchanek wrote:
> Excerpts from Filipus Klutiero's message of Wed May 16 18:44:21 +0200 2012:
>> Could you clarify how this differs from #481129?
> It's 4 years later.
>
> Sorry, forgot that I filed the bug already. It's quite some time.
>
>
Merging the reports then.
Message sent on
to Michal Suchanek <michal.suchanek@ruk.cuni.cz>:
Bug#671503.
(Fri, 18 May 2012 11:24:01 GMT) (full text, mbox, link).
Message #58 received at 671503-submitter@bugs.debian.org (full text, mbox, reply):
Michal Suchanek <michal.suchanek@ruk.cuni.cz> writes:
> Excerpts from Ian Jackson's message of Thu May 17 14:53:30 +0200 2012:
>> Michal Suchanek writes ("Re: Bug#671503: general: APT repository format is not documented"):
>> > Excerpts from Filipus Klutiero's message of Wed May 16 18:44:21 +0200 2012:
>> > > Could you clarify how this differs from #481129?
>> >
>> > It's 4 years later.
>> >
>> > Sorry, forgot that I filed the bug already. It's quite some time.
>> >
>> > Given there is no feedback in 4 years I guess it is futile reporting
>> > this.
>>
>> Well, it's useful to bring it up again.
>>
>> > Admittedly there is no text in social contract about using
>> > Debian-proprietary formats. And a format only defined by "apt can read
>> > that" is definitely Debian-proprietary there is no better term for that.
>>
>> Everyone agrees that it would be better if this were documented.
>> (I have struggled on occasion myself due to the lack of
>> documentation.)
>>
>> But I think the use of the word "proprietary" is going too far. It's
>> certainly a special Debian format, but that wouldn't be changed if it
>> were documented. But it's not secret and we publish at least two
>> writer implementations and one reader implementation AFAIK, with
>> proper Free licences.
>
> However, it's easier to reverse-engineer an existing repository than
> the source code so for all practical purposes it's the same as if it
> were closed source.
>
>>
>> > I'd say it's slightly discriminatory against software not part of Debian
>> > that cannot rely on getting notified when "apt can read that" silently
>> > changes, there is no document defining what apt should be able to read
>> > that software authors can rely on to interoperate with apt, one of the
>> > core Debian tools. Apt in turn relies on open standards like HTTP and
>> > FTP to interoperate with the rest of the world.
>>
>> I think this is not an appropriate use of the social contract or its
>> concepts.
>>
>> Rather than complaining that this documentation doesn't exist, how
>> about writing the document yourself ? It's not a trivial job but it
>> should be feasible by looking at the apt source code.
>
> For me it is not feasible at all.
>
> I can, of course, describe what current repositories look like or what
> the current apt code accepts.
>
> However, that has silently changed in the past and is considered apt
> feature, not a bug.
>
>>
>> Once such a document exists, even if it's a bit sketchy or perhaps not
>> entirely accurate, it will be much easier to insist that future
>> changes are likewise documented.
>
> I am not so sure about that.
>
> So long as the document merely describes what apt happens to do at the
> moment rather than apt implementing what the document says there is no
> saying this document has any value.
>
> The status was 'documented' by existing repositories which stopped
> working.
>
> Thanks
>
> Michal
I would suggest you look at existing repositories, whatever scraps of
information is in the manuals and maybe a bit at the source and start to
write a documentation. Once you have that offer it for review and other
people can pitch in their bits of knowledge. Getting the current format
documented right shouldn't be that hard if someone just starts.
And once such a document exists it is much easier to get people do
document changes or hit them over the head if they don't.
Remember that you don't have to be 100% right in what you write. You
only need to write a draft to start the process. Getting people to
comment and correct any mistakes you simply don't know about is much
much easier than getting someone else to write the whole thing.
MfG
Goswin
Message sent on
to Michal Suchanek <michal.suchanek@ruk.cuni.cz>:
Bug#671503.
(Fri, 18 May 2012 11:24:07 GMT) (full text, mbox, link).
Message #61 received at 671503-submitter@bugs.debian.org (full text, mbox, reply):
CC'ing the apt list deity@lists.debian.org.
Goswin von Brederlow writes ("Re: Bug#481129: Bug#671503: general: APT repository format is not documented"):
> Michal Suchanek <michal.suchanek@ruk.cuni.cz> writes:
> > [ discussions regarding documenting the apt repository format ]
>
> I would suggest you look at existing repositories, whatever scraps of
> information is in the manuals and maybe a bit at the source and start to
> write a documentation. Once you have that offer it for review and other
> people can pitch in their bits of knowledge. Getting the current format
> documented right shouldn't be that hard if someone just starts.
Right.
> And once such a document exists it is much easier to get people do
> document changes or hit them over the head if they don't.
Can the apt maintainers confirm that once such a document exists, they
will insist that future contributions to apt which change the
repository format update the document ?
What form do the apt maintainers think the document should take ?
Should it eventually be in the apt source package or somewhere else ?
> Remember that you don't have to be 100% right in what you write. You
> only need to write a draft to start the process. Getting people to
> comment and correct any mistakes you simply don't know about is much
> much easier than getting someone else to write the whole thing.
Indeed so.
Thanks,
Ian.
Message sent on
to Michal Suchanek <michal.suchanek@ruk.cuni.cz>:
Bug#671503.
(Fri, 18 May 2012 11:42:29 GMT) (full text, mbox, link).
Message #64 received at 671503-submitter@bugs.debian.org (full text, mbox, reply):
[Message part 1 (text/plain, inline)]
On Fri, May 18, 2012 at 12:02:47PM +0100, Ian Jackson wrote:
> CC'ing the apt list deity@lists.debian.org.
>
> Goswin von Brederlow writes ("Re: Bug#481129: Bug#671503: general: APT repository format is not documented"):
> > Michal Suchanek <michal.suchanek@ruk.cuni.cz> writes:
> > > [ discussions regarding documenting the apt repository format ]
> >
> > I would suggest you look at existing repositories, whatever scraps of
> > information is in the manuals and maybe a bit at the source and start to
> > write a documentation. Once you have that offer it for review and other
> > people can pitch in their bits of knowledge. Getting the current format
> > documented right shouldn't be that hard if someone just starts.
>
> Right.
>
> > And once such a document exists it is much easier to get people do
> > document changes or hit them over the head if they don't.
>
> Can the apt maintainers confirm that once such a document exists, they
> will insist that future contributions to apt which change the
> repository format update the document ?
>
> What form do the apt maintainers think the document should take ?
> Should it eventually be in the apt source package or somewhere else ?
I do not think that APT is responsible for the repository format. The
repository format is defined by ftpmaster, not by APT. APT has to my
knowledge not defined anything new, but only implemented changes to
the repository format after they were introduced by ftpmaster (see
InRelease files).
We currently have three independent implementations of the repository
format in the archive: APT, cupt, smartpm. Furthermore, tools like
debian-cd probably also have some knowledge about the repository
format.
The repository format should thus be part of Policy, not part of
APT. APT is one of the users of that format, not the one defining
it (it might just get stricter in behavior from time to time, just
like compilers). Changes to the format should require approval of
ftpmaster, as they have to implement them on the server-side.
--
Julian Andres Klode - Debian Developer, Ubuntu Member
See http://wiki.debian.org/JulianAndresKlode and http://jak-linux.org/.
[Message part 2 (application/pgp-signature, inline)]
Message sent on
to Michal Suchanek <michal.suchanek@ruk.cuni.cz>:
Bug#671503.
(Fri, 18 May 2012 12:45:35 GMT) (full text, mbox, link).
Message #67 received at 671503-submitter@bugs.debian.org (full text, mbox, reply):
[Message part 1 (text/plain, inline)]
On Fri, May 18, 2012 at 01:38:40PM +0200, Julian Andres Klode wrote:
> On Fri, May 18, 2012 at 12:02:47PM +0100, Ian Jackson wrote:
> > CC'ing the apt list deity@lists.debian.org.
> >
> > Goswin von Brederlow writes ("Re: Bug#481129: Bug#671503: general: APT repository format is not documented"):
> > > Michal Suchanek <michal.suchanek@ruk.cuni.cz> writes:
> > > > [ discussions regarding documenting the apt repository format ]
> > >
> > > I would suggest you look at existing repositories, whatever scraps of
> > > information is in the manuals and maybe a bit at the source and start to
> > > write a documentation. Once you have that offer it for review and other
> > > people can pitch in their bits of knowledge. Getting the current format
> > > documented right shouldn't be that hard if someone just starts.
> >
> > Right.
> >
> > > And once such a document exists it is much easier to get people do
> > > document changes or hit them over the head if they don't.
> >
> > Can the apt maintainers confirm that once such a document exists, they
> > will insist that future contributions to apt which change the
> > repository format update the document ?
> >
> > What form do the apt maintainers think the document should take ?
> > Should it eventually be in the apt source package or somewhere else ?
>
> I do not think that APT is responsible for the repository format. The
> repository format is defined by ftpmaster, not by APT. APT has to my
> knowledge not defined anything new, but only implemented changes to
> the repository format after they were introduced by ftpmaster (see
> InRelease files).
>
> We currently have three independent implementations of the repository
> format in the archive: APT, cupt, smartpm. Furthermore, tools like
> debian-cd probably also have some knowledge about the repository
> format.
>
> The repository format should thus be part of Policy, not part of
> APT. APT is one of the users of that format, not the one defining
> it (it might just get stricter in behavior from time to time, just
> like compilers). Changes to the format should require approval of
> ftpmaster, as they have to implement them on the server-side.
>
A working draft could be something like the following. It mostly
describes the current format for Release, Packages, and Sources
files. It's thus missing Contents and Translations, pdiffs, and
stuff, but it's a beginning.
It specifies different requirements for servers and clients,
in order to have clients be backwards compatible with more
repositories, and forcing servers to be stricter. Don't know
how good that is, though.
============================
= Debian Repository Format =
============================
This documents a subset of the Debian repository format. This is a work
in progress.
"Release" files
===============
The file "dists/$DIST/Release" shall contain meta-information about the
distribution and checksums for the indices. The file "dists/$DIST/Release.gpg"
shall be an GPG signature of the "Release" file, compatible with the format
used by the GPG options "-a -b -s". The file "dists/$DIST/InRelease" shall be
the "Release" file with a GPG clearsign signature compatible with the format
used by the GPG options "-a -s --clearsign".
The following fields might be available:
- Origin
- Label
- Suite
- Codename
- Date
- Valid-Until
- Architectures
- Components
- Description
- MD5sum, SHA1, SHA256
- NotAutomatic and ButAutomaticUpgrades
Servers shall provide the Release file, and its signed counterparts with at
least the following keys:
- SHA256
- Origin
- Suite and/or Codename
- Architectures
Clients shall accept missing Release files, and Release files without the
fields required for servers. They might reject Release files that do not
contain at least one of the fields defined herein.
Architectures
-------------
Whitespace separated unique single words identifying Debian machine architectures
as described in Architecture specification strings, Section 11.1.
Origin
------
Shall indicate the origin of the repository.
Label
-----
Optional field including some kind of label.
Suite
-----
The Suite field shall describe the suite. In Debian, this shall be one of
oldstable, stable, testing, unstable, or experimental; with optional suffixes
separated by "-" (such as "stable-updates").
Codename
--------
The Suite field shall describe the codename of the release. This is mostly
a free-form string used to give a name to a release.
Date, Valid-Until
-----------------
The Date field shall specify the time at which the Release file was created. The
Valid-Until field shall specify at which time the Release file should be
considered expired by the client. Client behaviour on expired Release files
is unspecified.
Components
-----------
A whitespace separated list of areas.
Example:
Components: main contrib non-free
MD5sum, SHA1, SHA256
--------------------
Those fields shall be multi-line fields containing multiple lines of whitespace
separated data. Each line shall contain
(1) The checksum of the file in the format corresponding to the field
(2) The size of the file (integer >= 0)
(3) The filename relative to the directory of the Release file
Each datum may be seperated by one or more whitespace characters.
Server requirements:
The field shall contain data about all uncompressed files, and should also
contain information about all compressed files. The checksum and sizes shall
match the actual existing files.
Client behaviour:
Any file should be checked at least once, either in compressed or
uncompressed form, depending on which data is available. If a file
has no associated data, the client shall inform the user about this
under possibly dangerous situations (such as installing a package
from that repository). If a file does not match the data specified
in the release file, the client shall not use any information from
that file, inform the user, and might use old information (such as
the previous locally kept information) instead.
NotAutomatic and ButAutomaticUpgrades
-------------------------------------
The NotAutomatic and ButAutomaticUpgrades fields are boolean fields
instructing the package manager. They may contain the values "yes"
and "no".
If "NotAutomatic: yes" is specified, the client should prevent installation
of packages from this repository unless explicitely requested (APT will assign
priority 1 to that repository).
If "ButAutomaticUpgrades: yes" is specified in addition to "NotAutomatic: yes",
the client should cause upgrades to packages from that repository to be
installed automatically (APT will assign priority 100 to that repository).
If both are either missing or set to "No", the repository should behave like
any other repository (APT will assign either priority 500 or 990 by default,
depending on whether the release is it's target release).
Other combinations are undefined.
"Packages" Indices
==================
The files dists/$DIST/$COMP/binary-$ARCH/Packages are called Binary Packages
Indices. They consist of multiple paragraphs, where each paragraph has the
format defined in Policy 5.3 (Binary package control files -- DEBIAN/control),
and the additional fields defined in this section.
Filename
--------
The Filename field shall list the path of the package archive relative to the
base directory of the repository.
Example:
Filename: pool/main/a/apt/apt_0.9.3_amd64.deb
Required: yes
Size
----
The size field shall give the size of the package file, in bytes.
Example:
Size: 1158196
Required: yes
MD5sum, SHA1, SHA256, SHA512
----------------------------
Checksums for the package. They shall be represented in hexadecimal
notation. The SHA512 field is not in active use prior to this
specification, the MD5sum and SHA1 fields should be considered
deprecated, but should still be provided.
Examples:
MD5sum: 2519c8c1afd27e70cf4ac10a5fa46e32
SHA1: 646eda5b6d51190181c15f5537428161f6f04c1d
SHA256: 3183eff291d1e9d905e78a6b467bbfb90b20fc2808d50b5e91bf55158b4c18be
Server requirements: SHA256 shall be available
Client requirements: Shall accept files without any such fields, should warn
if those fields are missing and a package is used.
Description-md5
----------------
An md5sum of the english description. This will be used to lookup the
translations in the translation indices. If this field is not defined,
the md5sum shall be calculated from the Description field.
Server requirements:
Either Description or Description-md5 shall be specified.
Client requirements:
If neither Description, nor Description-MD5 is defined, the result shall
be the same as if an empty description was specified for all languages.
If Description-md5 is defined, the long description shall be looked up
via translation indices if requested.
Example:
Description-md5: 9fb97a88cb7383934ef963352b53b4a7
Description
-----------
The Description field shall contain the complete package description, if
Description-md5 is not defined; or only the short description of the package,
if Description-md5 is defined.
"Sources" Indices
=================
The files dists/$DIST/$COMP/source/Sources are called Sources indices. They
consist of multiple paragraphs, where each paragraph has the format defined
in Policy 5.5 (5.4 Debian source control files -- .dsc), with the following
changes and additional fields. The changes are:
- The "Source" field is renamed to "Package"
- A new mandatory field "Package-List"
- A new mandatory field "Directory"
- A new optional field "Priority"
- A new optional field "Section"
Package-List
------------
The Package-List field shall contain multiple lines of package information,
where each line begins with a whitespace and has the following format:
$PKGNAME $TYPE $SECTION $PRIORITY
$PKGNAME is the name of the package, $TYPE is "deb" or "udeb", $SECTION
is the section of the package, and $PRIORITY is the priority of the package.
Example:
Package-List:
apt deb admin important
apt-doc deb doc optional
apt-transport-https deb admin optional
apt-utils deb admin important
libapt-inst1.5 deb admin important
libapt-pkg-dev deb libdevel optional
libapt-pkg-doc deb doc optional
libapt-pkg4.12 deb admin important
Directory
---------
The directory field shall list the location of the source package in the
repository, relative to the base directory of the repository.
Example:
Directory: pool/main/a/apt
Priority
--------
Shall contain the value "source".
Example:
Priority: source
Section
-------
Shall contain the section specified for the source package??
Example:
Section: admin
>
--
Julian Andres Klode - Debian Developer, Ubuntu Member
See http://wiki.debian.org/JulianAndresKlode and http://jak-linux.org/.
[Message part 2 (application/pgp-signature, inline)]
Message sent on
to Michal Suchanek <michal.suchanek@ruk.cuni.cz>:
Bug#671503.
(Fri, 18 May 2012 12:55:09 GMT) (full text, mbox, link).
Message #70 received at 671503-submitter@bugs.debian.org (full text, mbox, reply):
On Fri, May 18, 2012 at 01:38:40PM +0200, Julian Andres Klode wrote:
> I do not think that APT is responsible for the repository format. The
> repository format is defined by ftpmaster, not by APT. APT has to my
> knowledge not defined anything new, but only implemented changes to
> the repository format after they were introduced by ftpmaster (see
> InRelease files).
OK, we actually do have some more extensions not used by Debian or
Ubuntu. One of those is the "Important: yes" field, which is like
Essential, but does not force installation of the package like
Essential would do (and does not force immediate configuration
nowadays, so that we can use it for custom meta packages [so
that users cannot accidentally remove the meta package that
configures the complete system]).
I don't know of any other extensions, though. In any case,
they should probably not be part of an official specification,
but rather documented in APT.
--
Julian Andres Klode - Debian Developer, Ubuntu Member
See http://wiki.debian.org/JulianAndresKlode and http://jak-linux.org/.
Message sent on
to Michal Suchanek <michal.suchanek@ruk.cuni.cz>:
Bug#671503.
(Fri, 18 May 2012 14:09:21 GMT) (full text, mbox, link).
Message #73 received at 671503-submitter@bugs.debian.org (full text, mbox, reply):
FWIW
posted on the wiki: http://wiki.debian.org/RepositoryFormat
Thanks
Michal
Message sent on
to Michal Suchanek <michal.suchanek@ruk.cuni.cz>:
Bug#671503.
(Fri, 18 May 2012 16:51:12 GMT) (full text, mbox, link).
Message #76 received at 671503-submitter@bugs.debian.org (full text, mbox, reply):
[Message part 1 (text/plain, inline)]
On Fri, May 18, 2012 at 04:06:23PM +0200, Michal Suchanek wrote:
> FWIW
>
> posted on the wiki: http://wiki.debian.org/RepositoryFormat
>
> Thanks
>
> Michal
I have now documented the Contents indices and the diffs
as well, mostly (sans the exact format we use for the
patches), and Translation indices. Now we're basically
only missing details, it is fairly complete otherwise
(i.e. we should have mentioned every file and field
currently in use, but may not have explained all of
them completely).
We now have documented
dists/$DIST/Release (and InRelease, Release.gpg)
dists/$DIST/$COMP/binary-$ARCH/Packages
dists/$DIST/$COMP/source/Sources
dists/$DIST/$COMP/Contents-$ARCH.gz
dists/$DIST/$COMP/i18n/{Index,Translation-*.bz2}
*.diff/Index *.diff/%Y-%m-%d-%H%M.%S.gz
The other Release files have been omitted, as they are not
used anywhere. We are only missing udeb content files and
packages files now, which are just small subsentences.
In a few months, I'd like to rework this in DocBook form,
and submit it to debian-policy for inclusion into official
Policy, as a sub-policy like copyright-format.
--
Julian Andres Klode - Debian Developer, Ubuntu Member
See http://wiki.debian.org/JulianAndresKlode and http://jak-linux.org/.
[Message part 2 (application/pgp-signature, inline)]
Message sent on
to Michal Suchanek <michal.suchanek@ruk.cuni.cz>:
Bug#671503.
(Fri, 18 May 2012 17:00:09 GMT) (full text, mbox, link).
Message #79 received at 671503-submitter@bugs.debian.org (full text, mbox, reply):
On Fri, May 18, 2012 at 04:06:23PM +0200, Michal Suchanek wrote:
> FWIW
>
> posted on the wiki: http://wiki.debian.org/RepositoryFormat
What's the opinion about the flat repository format, where you
just have one directory with Release, Packages, Sources, and
friends and no sub-directories?
Should they be documented as well then? We would then have two
kind of documented repository formats:
1. Debian-style, with a pool (or similar) and a dists directory
2. Flat-style, with just one directory
This should cover everything we currently support. Although I don't
know much about how much stuff we support in flat directories WRT
Translation, Contents, and diffs.
--
Julian Andres Klode - Debian Developer, Ubuntu Member
See http://wiki.debian.org/JulianAndresKlode and http://jak-linux.org/.
Message sent on
to Michal Suchanek <michal.suchanek@ruk.cuni.cz>:
Bug#671503.
(Fri, 18 May 2012 17:48:10 GMT) (full text, mbox, link).
Message #82 received at 671503-submitter@bugs.debian.org (full text, mbox, reply):
+++ Julian Andres Klode [2012-05-18 13:38 +0200]:
> We currently have three independent implementations of the repository
> format in the archive: APT, cupt, smartpm.
I think reprepro is another?
/usr/share/doc/reprepro/manual.html contains a 'repository basics'
section which includes useful layout/format information.
Wookey
Message sent on
to Michal Suchanek <michal.suchanek@ruk.cuni.cz>:
Bug#671503.
(Fri, 18 May 2012 17:54:07 GMT) (full text, mbox, link).
Message #85 received at 671503-submitter@bugs.debian.org (full text, mbox, reply):
[Message part 1 (text/plain, inline)]
On Fri, May 18, 2012 at 06:45:00PM +0100, Wookey wrote:
> +++ Julian Andres Klode [2012-05-18 13:38 +0200]:
>
> > We currently have three independent implementations of the repository
> > format in the archive: APT, cupt, smartpm.
>
> I think reprepro is another?
Of course, I was just only talking about clients. When it comes to
creating we probably have much more than 3 programs needing some
knowledge of the repository format. We have dak, apt-ftparchive,
reprepro, debian-cd, everyone's small script, mini-dinstall (the
latter using flat repositories, if I am not mistaken).
--
Julian Andres Klode - Debian Developer, Ubuntu Member
See http://wiki.debian.org/JulianAndresKlode and http://jak-linux.org/.
[Message part 2 (application/pgp-signature, inline)]
Message sent on
to Michal Suchanek <michal.suchanek@ruk.cuni.cz>:
Bug#671503.
(Fri, 18 May 2012 18:15:14 GMT) (full text, mbox, link).
Message #88 received at 671503-submitter@bugs.debian.org (full text, mbox, reply):
Excerpts from Julian Andres Klode's message of Fri May 18 18:49:10 +0200 2012:
> On Fri, May 18, 2012 at 04:06:23PM +0200, Michal Suchanek wrote:
> > FWIW
> >
> > posted on the wiki: http://wiki.debian.org/RepositoryFormat
> >
> > Thanks
> >
> > Michal
>
> I have now documented the Contents indices and the diffs
> as well, mostly (sans the exact format we use for the
> patches), and Translation indices. Now we're basically
> only missing details, it is fairly complete otherwise
> (i.e. we should have mentioned every file and field
> currently in use, but may not have explained all of
> them completely).
>
> We now have documented
>
> dists/$DIST/Release (and InRelease, Release.gpg)
> dists/$DIST/$COMP/binary-$ARCH/Packages
> dists/$DIST/$COMP/source/Sources
> dists/$DIST/$COMP/Contents-$ARCH.gz
> dists/$DIST/$COMP/i18n/{Index,Translation-*.bz2}
> *.diff/Index *.diff/%Y-%m-%d-%H%M.%S.gz
>
> The other Release files have been omitted, as they are not
> used anywhere. We are only missing udeb content files and
> packages files now, which are just small subsentences.
>
> In a few months, I'd like to rework this in DocBook form,
> and submit it to debian-policy for inclusion into official
> Policy, as a sub-policy like copyright-format.
>
Yes, looks fairly complete.
The formatting is not consistent but that will have to be changed for
docbook anyway.
Also would need some proof-reading. If nothing else somebody should look
in a few weeks from now if it still makes sense ;-)
I put a link on the RepositoryHowto page for more exposure.
I am not so sure documenting Debian installer files is tremendously
useful. I don't think anyone outside Debian Installer team makes Debian
Installer repositories and there are other aspects of Debian Installer
that would need to be documented in order for it to be usable for
'outside' people in non-default configurations.
Thanks
Michal
Message sent on
to Michal Suchanek <michal.suchanek@ruk.cuni.cz>:
Bug#671503.
(Fri, 18 May 2012 18:30:09 GMT) (full text, mbox, link).
Message #91 received at 671503-submitter@bugs.debian.org (full text, mbox, reply):
On Fri, May 18, 2012 at 08:12:16PM +0200, Michal Suchanek wrote:
> The formatting is not consistent but that will have to be changed for
> docbook anyway.
Yes, and it will also be more readable then, than the current wiki
version.
>
> Also would need some proof-reading. If nothing else somebody should look
> in a few weeks from now if it still makes sense ;-)
I'd also like to hear bits from the launchpad team about their
implementation and see whether they agree with everything then.
>
> I put a link on the RepositoryHowto page for more exposure.
>
> I am not so sure documenting Debian installer files is tremendously
> useful. I don't think anyone outside Debian Installer team makes Debian
> Installer repositories and there are other aspects of Debian Installer
> that would need to be documented in order for it to be usable for
> 'outside' people in non-default configurations.
We still need to at least document the udeb stuff, the images and
other stuff is not relevant to the core format and probably defined
by the d-i team anyway (and installed by-hand).
The udeb stuff is relatively easy to document, as it just adds one
new directory and one new filename for Contents files, so can be
done in about two sentences (and udebs are uploaded by standard
.changes with the rest of the package, and are thus standard).
--
Julian Andres Klode - Debian Developer, Ubuntu Member
See http://wiki.debian.org/JulianAndresKlode and http://jak-linux.org/.
Message sent on
to Michal Suchanek <michal.suchanek@ruk.cuni.cz>:
Bug#671503.
(Fri, 18 May 2012 23:42:06 GMT) (full text, mbox, link).
Message #94 received at 671503-submitter@bugs.debian.org (full text, mbox, reply):
On Sat, May 19, 2012 at 12:58 AM, Julian Andres Klode wrote:
> What's the opinion about the flat repository format, where you
> just have one directory with Release, Packages, Sources, and
> friends and no sub-directories?
>
> Should they be documented as well then? We would then have two
> kind of documented repository formats:
>
> 1. Debian-style, with a pool (or similar) and a dists directory
> 2. Flat-style, with just one directory
>
> This should cover everything we currently support. Although I don't
> know much about how much stuff we support in flat directories WRT
> Translation, Contents, and diffs.
I would like to see the flat-style repository documented too, since
some of the derivatives in the Debian derivatives census use it and I
would like to lint their apt repositories.
--
bye,
pabs
http://wiki.debian.org/PaulWise
Message sent on
to Michal Suchanek <michal.suchanek@ruk.cuni.cz>:
Bug#671503.
(Sat, 19 May 2012 07:21:06 GMT) (full text, mbox, link).
Message #97 received at 671503-submitter@bugs.debian.org (full text, mbox, reply):
Julian Andres Klode <jak@debian.org> writes:
> On Fri, May 18, 2012 at 04:06:23PM +0200, Michal Suchanek wrote:
>> FWIW
>>
>> posted on the wiki: http://wiki.debian.org/RepositoryFormat
>>
>> Thanks
>>
>> Michal
>
> I have now documented the Contents indices and the diffs
> as well, mostly (sans the exact format we use for the
> patches), and Translation indices. Now we're basically
> only missing details, it is fairly complete otherwise
> (i.e. we should have mentioned every file and field
> currently in use, but may not have explained all of
> them completely).
>
> We now have documented
>
> dists/$DIST/Release (and InRelease, Release.gpg)
> dists/$DIST/$COMP/binary-$ARCH/Packages
> dists/$DIST/$COMP/source/Sources
> dists/$DIST/$COMP/Contents-$ARCH.gz
> dists/$DIST/$COMP/i18n/{Index,Translation-*.bz2}
> *.diff/Index *.diff/%Y-%m-%d-%H%M.%S.gz
>
> The other Release files have been omitted, as they are not
> used anywhere. We are only missing udeb content files and
> packages files now, which are just small subsentences.
>
> In a few months, I'd like to rework this in DocBook form,
> and submit it to debian-policy for inclusion into official
> Policy, as a sub-policy like copyright-format.
This describes repositories of the form
deb uri suite component [...]
There should be a mention of flat repositories of the form
deb url path/
This changes nothing for the contents of files but it does change their
location and I think it's worth mentioning how that sources.list entry
maps to a repository.
MfG
Goswin
Message sent on
to Michal Suchanek <michal.suchanek@ruk.cuni.cz>:
Bug#671503.
(Sat, 19 May 2012 12:36:10 GMT) (full text, mbox, link).
Message #100 received at 671503-submitter@bugs.debian.org (full text, mbox, reply):
On Sat, May 19, 2012 at 07:38:59AM +0800, Paul Wise wrote:
> On Sat, May 19, 2012 at 12:58 AM, Julian Andres Klode wrote:
>
> > What's the opinion about the flat repository format, where you
> > just have one directory with Release, Packages, Sources, and
> > friends and no sub-directories?
> >
> > Should they be documented as well then? We would then have two
> > kind of documented repository formats:
> >
> > Â Â Â Â 1. Debian-style, with a pool (or similar) and a dists directory
> > Â Â Â Â 2. Flat-style, with just one directory
> >
> > This should cover everything we currently support. Although I don't
> > know much about how much stuff we support in flat directories WRT
> > Translation, Contents, and diffs.
>
> I would like to see the flat-style repository documented too, since
> some of the derivatives in the Debian derivatives census use it and I
> would like to lint their apt repositories.
I added (and others edited formatting a bit)
= Flat Repository Format =
A flat repository does not use the {{{dists}}} hierarchy of directories,
and instead places meta index and indices directly into the archive root
(or some part below it) In sources.list syntax, a flat repository is specified
like this:
{{{
deb uri directory/
}}}
Where {{{uri}}} specifies the archive root, and {{{directory}}} specifies the
position of the meta index and the indices relative to the archive root. In
Flat repositories, the following indices are supported:
* Packages (under the location {{{directory/Packages}}})
* Sources (under the location {{{directory/Sources}}})
!InRelease, Release, Release.gpg meta-information are supported as well. Diffs,
Translations, and Contents indices are not defined for that repository format.
Indices may be compressed just like in the standard Debian repository format.
--
Julian Andres Klode - Debian Developer, Ubuntu Member
See http://wiki.debian.org/JulianAndresKlode and http://jak-linux.org/.
Message sent on
to Michal Suchanek <michal.suchanek@ruk.cuni.cz>:
Bug#671503.
(Sun, 20 May 2012 11:33:55 GMT) (full text, mbox, link).
Message #103 received at 671503-submitter@bugs.debian.org (full text, mbox, reply):
On Sat, May 19, 2012 at 2:00 PM, Julian Andres Klode <jak@debian.org> wrote:
> = Flat Repository Format =
>
> A flat repository does not use the {{{dists}}} hierarchy of directories,
> and instead places meta index and indices directly into the archive root
> (or some part below it) In sources.list syntax, a flat repository is specified
> like this:
>
> {{{
> deb uri directory/
> }}}
I don't think defining sources.list syntax in a client-agnostic document
is a good move. APT has the 'sources.list' manpage for it and other clients
might or might not have different ways to specify repositories.
(beside, that it would be deb-src, too)
> !InRelease, Release, Release.gpg meta-information are supported as well. Diffs,
> Translations, and Contents indices are not defined for that repository format.
> Indices may be compressed just like in the standard Debian repository format.
Translations are supported, although with a different name: directory/en
(and co) instead of Translation-en. For Contents i am not sure, but i think
apt-file downloads these, too. (not sure if this should be a reason to include
it in a specification through or just keep it as some legacy cruft around)
Diffs are supported by apt, but it will not be used if not in Release.
(if no Release file is present, diffs will not be tried).
It's the same for the non-flat repository and true for other files as well
- and should be a reasonable thing to allow clients to do.
In that train of thought, I think it would be a good idea to require a
repository to have a Release (or InRelease) file including all files
[in their current state] composing this repository.
They are easy to create and this way a client could stop guessing if
they like to, avoiding possibly a lot of 404's.
Best combined with a strong recommendation on signing them.
Best regards
David Kalnischkies
P.S.: Could we please stop talking to three bugs and two mailinglists?
Especially as [0] suggests it is the wrong list…
[0] http://lists.debian.org/debian-devel/2012/05/msg00222.html
Information stored
:
Bug#671503; Package debian-policy.
(Mon, 21 May 2012 08:24:05 GMT) (full text, mbox, link).
Acknowledgement sent
to Michal Suchanek <michal.suchanek@ruk.cuni.cz>:
Extra info received and filed, but not forwarded.
(Mon, 21 May 2012 08:24:06 GMT) (full text, mbox, link).
Message #108 received at 671503-quiet@bugs.debian.org (full text, mbox, reply):
Excerpts from David Kalnischkies's message of Sun May 20 13:31:33 +0200 2012:
> On Sat, May 19, 2012 at 2:00 PM, Julian Andres Klode <jak@debian.org> wrote:
> > = Flat Repository Format =
> >
> > A flat repository does not use the {{{dists}}} hierarchy of directories,
> > and instead places meta index and indices directly into the archive root
> > (or some part below it) In sources.list syntax, a flat repository is specified
> > like this:
> >
> > {{{
> > Â deb uri directory/
> > }}}
>
> I don't think defining sources.list syntax in a client-agnostic document
> is a good move. APT has the 'sources.list' manpage for it and other clients
> might or might not have different ways to specify repositories.
> (beside, that it would be deb-src, too)
The sources.list is *the* file that specifies where to look for
repositories, even for non-apt clients. Maybe it could be made clearer
that this is an example, not a specification of the file.
Thanks
Michal
Message sent on
to Michal Suchanek <michal.suchanek@ruk.cuni.cz>:
Bug#671503.
(Mon, 21 May 2012 08:24:08 GMT) (full text, mbox, link).
Information forwarded
to debian-bugs-dist@lists.debian.org, Debian Policy List <debian-policy@lists.debian.org>:
Bug#671503; Package debian-policy.
(Mon, 21 May 2012 08:54:03 GMT) (full text, mbox, link).
Acknowledgement sent
to Jonathan Nieder <jrnieder@gmail.com>:
Extra info received and forwarded to list. Copy sent to Debian Policy List <debian-policy@lists.debian.org>.
(Mon, 21 May 2012 08:54:03 GMT) (full text, mbox, link).
Message #116 received at 671503@bugs.debian.org (full text, mbox, reply):
Michal Suchanek wrote:
> The sources.list is *the* file that specifies where to look for
> repositories, even for non-apt clients.
dselect uses a different format for /var/lib/dpkg/cmethopt and
/var/lib/dpkg/<method>/*. Most of the other non-apt package managers
I know are apt clones. If I remember correctly, reprepro can pull
from one APT repository to populate another and does not use
sources.list format, either.
Using the sources.list format to illustrate terminology doesn't seem
like a bad idea, though.
Hope that helps,
Jonathan
Severity set to 'wishlist' from 'important'
Request was from Russ Allbery <rra@debian.org>
to control@bugs.debian.org.
(Sun, 12 Aug 2012 18:30:07 GMT) (full text, mbox, link).
Changed Bug title to 'document APT repository format' from 'general: APT repository format is not documented'
Request was from Russ Allbery <rra@debian.org>
to control@bugs.debian.org.
(Sun, 12 Aug 2012 18:30:07 GMT) (full text, mbox, link).
Message sent on
to Michal Suchanek <michal.suchanek@ruk.cuni.cz>:
Bug#671503.
(Thu, 13 Mar 2014 12:18:21 GMT) (full text, mbox, link).
Message #123 received at 671503-submitter@bugs.debian.org (full text, mbox, reply):
I have proposal for you.
Reply sent
to Sean Whitton <spwhitton@spwhitton.name>:
You have taken responsibility.
(Fri, 11 Aug 2017 19:58:39 GMT) (full text, mbox, link).
Notification sent
to Michal Suchanek <michal.suchanek@ruk.cuni.cz>:
Bug acknowledged by developer.
(Fri, 11 Aug 2017 19:58:39 GMT) (full text, mbox, link).
Message #128 received at 671503-close@bugs.debian.org (full text, mbox, reply):
[Message part 1 (text/plain, inline)]
control: user debian-policy@packages.debian.org
control: usertag -1 +obsolete
control: tag -1 +wontfix
Russ Allbery and I did a round of in-person bug triage at DebConf17 and
we are closing this bug as inactive.
The reasons for closing fall into the following categories, from most
frequent to least frequent:
- issue is appropriate for Policy, there is a consensus on how to fix
the problem, but preparing the patch is very time-consuming and no-one
has volunteered to do it, and we do not judge the issue to be
important enough to keep an open bug around;
- issue is appropriate for Policy but there does not yet exist a
consensus on what should change, and no recent discussion. A fresh
discussion might allow us to reach consensus, and the messages in the
old bug are unlikely to help very much; or
- issue is not appropriate for Policy.
If you feel this bug is still relevant and want to restart the
discussion, you can re-open the bug. However, please consider instead
opening a new bug with a message that summarises and condenses the
previous discussion, updates the report for the current state of Debian,
and makes clear exactly what you think should change.
A lot of these old bugs have long side tangents and numerous messages,
and that old discussion is not necessarily helpful for figuring out what
Debian Policy should say today.
--
Sean Whitton
[signature.asc (application/pgp-signature, inline)]
Added tag(s) wontfix.
Request was from Sean Whitton <spwhitton@spwhitton.name>
to control@bugs.debian.org.
(Fri, 11 Aug 2017 20:18:21 GMT) (full text, mbox, link).
Bug archived.
Request was from Debbugs Internal Request <owner@bugs.debian.org>
to internal_control@bugs.debian.org.
(Sat, 09 Sep 2017 07:25:37 GMT) (full text, mbox, link).
Send a report that this bug log contains spam.
Debian bug tracking system administrator <owner@bugs.debian.org>.
Last modified:
Fri Jan 12 11:29:31 2018;
Machine Name:
buxtehude
Debian Bug tracking system
Debbugs is free software and licensed under the terms of the GNU
Public License version 2. The current version can be obtained
from https://bugs.debian.org/debbugs-source/.
Copyright © 1999 Darren O. Benham,
1997,2003 nCipher Corporation Ltd,
1994-97 Ian Jackson,
2005-2017 Don Armstrong, and many other contributors.