Debian Bug report logs - #505173
dput: provide SFTP support via Paramiko library

Reply or subscribe to this bug.

Toggle useless messages

Message #5 received at submit@bugs.debian.org (full text, mbox, reply):

From: "Cody A.W. Somerville" <cody.somerville@canonical.com>

To: submit@bugs.debian.org

Subject: Patch to add dput sftp transport and host argument support

Date: Mon, 10 Nov 2008 02:58:32 -0500

[Message part 1 (text/plain, inline)]

Subject: Patch to add dput sftp transport and host argument support
Package: dput
Version: 0.9.2.35ubuntu1
Severity: wishlist
Tags: patch

I've created a patch to add support for sftp transport and the ability to
pass an "argument" to a host. The latter allows you to have a single stanza
in dput.cf instead of several for almost identical host configurations. I've
modified the ppa stanza to take advantage of this feature, enabling me to
upload to any ppa without modifying dput.cf by simplying appending 'ppa'
with a colon and the launchpad id of the ppa. So, for example, to upload to
my own PPA I would type: dput ppa:cody-somerville <package.changes>

I've filed a bug to get this patch included in Ubuntu and figured it would
be
of interest to Debian as well. You can find the bug report and patch at
https://bugs.launchpad.net/ubuntu/+source/dput/+bug/295613

-- System Information:
Debian Release: lenny/sid
  APT prefers hardy-updates
  APT policy: (500, 'hardy-updates'), (500, 'hardy-security'), (500,
'hardy-backports'), (500, 'hardy')
Architecture: i386 (i686)

Kernel: Linux 2.6.24-21-generic (SMP w/1 CPU core)
Locale: LANG=en_CA.UTF-8, LC_CTYPE=en_CA.UTF-8 (charmap=UTF-8)
Shell: /bin/sh linked to /bin/dash

Versions of packages dput depends on:
ii  gnupg                     1.4.6-2ubuntu5 GNU privacy guard - a free PGP
rep
ii  python                    2.5.2-0ubuntu1 An interactive high-level
object-o

dput recommends no packages.

-- no debconf information

[Message part 2 (text/html, inline)]

Message #12 received at 505173@bugs.debian.org (full text, mbox, reply):

From: Yves-Alexis Perez <corsac@debian.org>

To: 505173@bugs.debian.org

Cc: 505173-subscribe@bugs.debian.org

Subject: news?

Date: Tue, 11 Aug 2009 14:18:37 +0200

Hey, is there some news about this? It seems ubuntu did integrate the
patch from Cody, but Debian didn't.

I'm managing a private repository, and upload using scp, but would like
to tune the sshd config to use internal-sftp and chroot only. But for
that, I would need dput to support sftp, so…

Anyway, is there some info on the integration?

Cheers and thanks,
-- 
Yves-Alexis

Message #15 received at 505173@bugs.debian.org (full text, mbox, reply):

From: Y Giridhar Appaji Nag <appaji@debian.org>

To: Yves-Alexis Perez <corsac@debian.org>, 505173@bugs.debian.org

Subject: Re: Bug#505173: news?

Date: Wed, 12 Aug 2009 17:21:28 +0530

[Message part 1 (text/plain, inline)]

On 09/08/11 14:18 +0200, Yves-Alexis Perez said ...
> Hey, is there some news about this? It seems ubuntu did integrate the
> patch from Cody, but Debian didn't.

So far the only users of sftp have been Ubuntu/PPA.

[snip...]

> Anyway, is there some info on the integration?

http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=505176#15

I am reluctant to merge the patch that would force dput users to install bzr.
I'll happily include anything that uses paramiko directly.

Giridhar

-- 
Y Giridhar Appaji Nag | http://people.debian.org/~appaji/

[signature.asc (application/pgp-signature, inline)]

Message #20 received at 505173@bugs.debian.org (full text, mbox, reply):

From: Yves-Alexis Perez <corsac@debian.org>

To: Y Giridhar Appaji Nag <appaji@debian.org>

Cc: 505173@bugs.debian.org, 505176@bugs.debian.org

Subject: Re: Bug#505173: news?

Date: Wed, 12 Aug 2009 13:56:33 +0200

On mer, 2009-08-12 at 17:21 +0530, Y Giridhar Appaji Nag wrote:
> On 09/08/11 14:18 +0200, Yves-Alexis Perez said ...
> > Hey, is there some news about this? It seems ubuntu did integrate
> the
> > patch from Cody, but Debian didn't.
> 
> So far the only users of sftp have been Ubuntu/PPA.

Yeah, I wouldn't have needed it until I read about internal-sftp stuff
in ssh, which seems nice to have but won't work with scp.
> 
> [snip...]
> 
> > Anyway, is there some info on the integration?
> 
> http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=505176#15

Yeah, sorry I read/replied on the wrong bug :/
> 
> I am reluctant to merge the patch that would force dput users to install bzr.
> I'll happily include anything that uses paramiko directly.

I can understand that. If that needs becomes really important for me, I
guess I'll kick myself and start looking at that. Until then, I don't
think I'd like bzr to be installed with dput :)

Cheers,
-- 
Yves-Alexis

Message #25 received at 505173@bugs.debian.org (full text, mbox, reply):

From: Iain Lane <laney@debian.org>

To: Y Giridhar Appaji Nag <appaji@debian.org>, 505173@bugs.debian.org

Subject: Re: Bug#505173: news?

Date: Thu, 14 Jul 2011 23:32:04 +0100

[Message part 1 (text/plain, inline)]

Hiya,

On Wed, Aug 12, 2009 at 05:21:28PM +0530, Y Giridhar Appaji Nag wrote:
> On 09/08/11 14:18 +0200, Yves-Alexis Perez said ...
> > Hey, is there some news about this? It seems ubuntu did integrate the
> > patch from Cody, but Debian didn't.
> 
> So far the only users of sftp have been Ubuntu/PPA.
> 
> [snip...]
> 
> > Anyway, is there some info on the integration?
> 
> http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=505176#15
> 
> I am reluctant to merge the patch that would force dput users to install bzr.
> I'll happily include anything that uses paramiko directly.

Here's one that does that.

The bzrlib solution is demonstrably better though; it uses the OpenSSH
configuration whereas here the best I could do without adding way too much
code for an application like dput was to default to using an SSH agent or
allow users to specify their key location/password.

I created a bzr bug asking for a separate library

  https://bugs.launchpad.net/bzr/+bug/810783

hopefully they'll do that and we can merge the patches back together.

Anyway, mine is attached, for what it's worth. It works for me to upload to
Ubuntu using SFTP.

Cheers,
Iain

[0001-Add-a-new-SFTP-method.patch (text/x-diff, attachment)]

[signature.asc (application/pgp-signature, inline)]

Message #30 received at 505173@bugs.debian.org (full text, mbox, reply):

From: Clint Adams <clint@debian.org>

To: 505173@bugs.debian.org

Subject: dput/sftp

Date: Fri, 30 Sep 2011 19:44:47 +0000

I would also like sftp support irrespective of the ppa business.

Message #35 received at 505173@bugs.debian.org (full text, mbox, reply):

From: Daniel Lintott <daniel@serverb.co.uk>

To: submit@bugs.debian.org

Cc: 505173@bugs.debian.org

Subject: dput: Patch to suport for host arguments

Date: Sun, 08 Jun 2014 12:41:09 +0100

[Message part 1 (text/plain, inline)]

Package: dput
Version: 0.9.6.4
Severity: wishlist

In bug #505173 and #505176 a patch was proposed to add SFTP and host
argument support from Ubuntu to the Debian version of dput.

There was some reluctance surrounding this as adding SFTP support
introduced a dependency on bzr.

I work with closely with an upstream author where we make extensive of
Launchpad's PPA feature, so having the ability to easily upload to
multiple PPA's from Debian would be very useful.

I have taken the current patch applied by Ubuntu in the latest release
of dput and split the SFTP support from it, leaving just the host
argument support.

I have tested this locally on my Wheezy machine and the attached patch
works as expected,

Please consider merging this into Debian.

Regards,

Daniel

-- System Information:
Debian Release: 7.5
  APT prefers stable-updates
  APT policy: (500, 'stable-updates'), (500, 'stable')
Architecture: amd64 (x86_64)
Foreign Architectures: i386

Kernel: Linux 3.2.0-4-amd64 (SMP w/2 CPU cores)
Locale: LANG=en_GB.UTF-8, LC_CTYPE=en_GB.UTF-8 (charmap=UTF-8)
Shell: /bin/sh linked to /bin/dash

Versions of packages dput depends on:
ii  gnupg   1.4.12-7+deb7u3
ii  python  2.7.3-4+deb7u1

dput recommends no packages.

Versions of packages dput suggests:
ii  lintian         2.5.22.1~bpo70+1
pn  mini-dinstall   <none>
ii  openssh-client  1:6.0p1-4+deb7u1
ii  rsync           3.0.9-4

-- no debconf information

[0001-Add-argument-passing-to-support-PPA-upload.patch (text/x-patch, attachment)]

[signature.asc (application/pgp-signature, attachment)]

Message #42 received at 505173@bugs.debian.org (full text, mbox, reply):

From: u <u@451f.org>

To: 505173@bugs.debian.org

Subject: Any update

Date: Tue, 15 Nov 2016 17:27:00 +0000

Hi!

I'd like to be able to use SFTP to upload from Debian to a Ubuntu PPA too.

Is there any progress / ETA on integrating this patch into Debian's dput?

Thanks & cheers!
u.

Message #47 received at 505173@bugs.debian.org (full text, mbox, reply):

From: Ben Finney <bignose@debian.org>

To: u <u@451f.org>, 505173@bugs.debian.org

Subject: Re: Bug#505173: Any update

Date: Wed, 16 Nov 2016 18:36:18 +1100

[Message part 1 (text/plain, inline)]

Thanks for your interest.

On 15-Nov-2016, u wrote:

> I'd like to be able to use SFTP to upload from Debian to a Ubuntu
> PPA too.

What tools already exist to do this?

> Is there any progress / ETA on integrating this patch into Debian's
> dput?

An extensive rewrite of the code base (to support Python 3, among
other changes) needs to go in before patches for features can be
considered again.

-- 
 \           “I do not believe in immortality of the individual, and I |
  `\        consider ethics to be an exclusively human concern with no |
_o__)  superhuman authority behind it.” —Albert Einstein, letter, 1953 |
Ben Finney <bignose@debian.org>

[signature.asc (application/pgp-signature, inline)]

Message #52 received at 505173@bugs.debian.org (full text, mbox, reply):

From: u <u@451f.org>

To: Ben Finney <bignose@debian.org>, 505173@bugs.debian.org

Subject: Re: Bug#505173: Any update

Date: Wed, 16 Nov 2016 10:17:00 +0000

Hi!

Ben Finney:
> Thanks for your interest.
> 
> On 15-Nov-2016, u wrote:
> 
>> I'd like to be able to use SFTP to upload from Debian to a Ubuntu
>> PPA too.
> 
> What tools already exist to do this?

Well, dput is supposed to work with SFTP simply by specifying "method =
sftp" in any dput config file such as ~/.dput.cf.

However, when specifying this method, dput throws this error: "Unknown
upload method: sftp"

There is an answer to this here:
https://bugs.launchpad.net/launchpad/+bug/251685/comments/12 claiming
"The dput package on Ubuntu installs a /usr/share/dput/sftp.py file -
maybe you can copy it to your Debian instance although I'm surprised if
it's not in Debian's dput by now?"

Hence, my question actually is: why can't Debian's dput work with SFTP now?

I understand that besides this problem there is a dependency on bzr -
which seems unnecessary.

Hence my second question: there are some patches attached to this bug -
will they be integrated and what's the ETA for that?

Currently, I'm able to use dput only using FTP, which seems slightly
insecure to me :)

Cheers!
u.

Message #57 received at 505173@bugs.debian.org (full text, mbox, reply):

From: Ben Finney <bignose@debian.org>

To: u <u@451f.org>

Cc: 505173@bugs.debian.org

Subject: Re: Bug#505173: Any update

Date: Thu, 17 Nov 2016 04:27:37 +1100

[Message part 1 (text/plain, inline)]

On 16-Nov-2016, u wrote:
> Well, dput is supposed to work with SFTP simply by specifying
> "method = sftp" in any dput config file such as ~/.dput.cf.

That has never been the case for the ‘dput’ package.

This bug is a request for a new feature that ‘dput’ doesn't yet have.

> However, when specifying this method, dput throws this error:
> "Unknown upload method: sftp"

Right, because the feature is not yet implemented :-)

> Hence, my question actually is: why can't Debian's dput work with
> SFTP now?

If you mean: why can't Dput do that now? The answer is: because the
feature is not yet implemented.

If you mean: why not just add that feature the same way Ubuntu has
done? The answer is, because this feature and others are better
addressed by a differently designed API. I'm not keen to add more code
to the legacy API for that reason.

> Currently, I'm able to use dput only using FTP, which seems slightly
> insecure to me :)

You're right, it is. There are better methods already: the ‘https’
method is more secure, and the ‘scp’ method more secure still.

I can see the desire for ‘sftp’ support, and it is on the To Do list.
Since it's not causing any problems in Debian, though, it is waiting
for more extensive changes in Dput before it can be considered.

-- 
 \             “The reward of energy, enterprise and thrift is taxes.” |
  `\                                                  —William Feather |
_o__)                                                                  |
Ben Finney <bignose@debian.org>

[signature.asc (application/pgp-signature, inline)]

Message #62 received at 505173@bugs.debian.org (full text, mbox, reply):

From: u <u@451f.org>

To: Ben Finney <bignose@debian.org>

Cc: 505173@bugs.debian.org

Subject: Re: Bug#505173: Any update

Date: Thu, 17 Nov 2016 09:52:00 +0000

Hi,

Ben Finney:
> On 16-Nov-2016, u wrote:
>> Well, dput is supposed to work with SFTP simply by specifying
>> "method = sftp" in any dput config file such as ~/.dput.cf.
> 
> That has never been the case for the ‘dput’ package. 
> This bug is a request for a new feature that ‘dput’ doesn't yet have.

I see.

>> Hence, my question actually is: why can't Debian's dput work with
>> SFTP now?
> 
> If you mean: why can't Dput do that now? The answer is: because the
> feature is not yet implemented.
> 
> If you mean: why not just add that feature the same way Ubuntu has
> done? The answer is, because this feature and others are better
> addressed by a differently designed API. I'm not keen to add more code
> to the legacy API for that reason.

Ack, thanks a lot for your answer.

Cheers!
u.

Message #67 received at 505173@bugs.debian.org (full text, mbox, reply):

From: Julian Andres Klode <julian.klode@canonical.com>

To: 505173@bugs.debian.org

Subject: Re: Bug#505173: New patch for SFTP support

Date: Thu, 11 Jan 2018 21:36:24 +0100

[Message part 1 (text/plain, inline)]

Control: user ubuntu-devel@lists.ubuntu.com
Control: usertag -1 origin-ubuntu ubuntu-patch

Hi Ben,

as I mentioned to you, I have implemented sftp support for dput
again. I was merging the latest dput 1.0.1 (tag missing, BTW) and
had to rewrite the method because bzrlib was not available on Python 3.

I attached a patch introducing the new method and manual page updates
for it. I tried to make sure to follow the coding style of the other
code as far as possible :) (I also attached a small bugfix for dput.cf's
ubuntu entries).

This implements sftp support by using paramiko's sftp support on top
of an openssh connection (invoking the ssh binary). It's a stripped down
version of what bzrlib was doing, and there is reason for doing it that
way:

I looked at both paramiko and asyncssh, as well as bindings to libssh2,
and they all suffer from an important problem: Interactively establishing
a connection. For example, paramiko just raises an exception if you have
an encrypted private key file; and it also only provides 3 strategies for
dealing with unknown hosts: Add automatically, Warn and add, or Reject
- no interactive stuff. asyncssh is similarly limited, perhaps even more,
and libssh2 bindings only provide explicit authentication methods.

Hence I decided to follow bzr and use ssh(1) to establish the connection
in a subprocess and then use paramiko's sftp implementation on top of it.
This ensures that we get all the normal connection setup and authentication
handling as we do if we just ssh somewhere.

A possible improvement would be to upload the file atomically - bzr does
that. The other methods don't, but it seems reasonably easy to implement
- I'd be happy to do so.

I also have a few other commits in my git branch you might find interesting:

   https://git.launchpad.net/~juliank/+git/dput for-debian
   https://git.launchpad.net/~juliank/+git/dput?h=for-debian (web view)

Let me know what you think!

Thanks,
Julian
-- 
debian developer - deb.li/jak | jak-linux.org - free software dev
ubuntu core developer                              i speak de, en

[0001-Implement-SFTP-support.patch (text/x-diff, attachment)]

[0002-dput.cf-Adjust-upload-paths-for-Ubuntu-and-ppas.patch (text/x-diff, attachment)]

Send a report that this bug log contains spam.