Debian Bug report logs - #714606
ruby-net-ssh: can't add a new key into hash during iteration during ssh.exec

Toggle useless messages

Message #5 received at submit@bugs.debian.org (full text, mbox, reply):

From: Martin Steigerwald <ms@teamix.de>

To: Debian Bug Tracking System <submit@bugs.debian.org>

Subject: ruby-net-ssh: can't add a new key into hash during iteration during ssh.exec

Date: Mon, 01 Jul 2013 11:35:40 +0200

Package: ruby-net-ssh
Version: 1:2.5.2-2
Severity: normal

Dear Maintainer,

I am currently packaging our own distkeys key distribution ruby script
(see #712787 RFS: distkeys/1.0 -- distribute SSH keys).

However it only works with Ruby 1.8 for now, as with Ruby 1.9 I get a error
back from ruby-net-ssh when trying to add or remove a key:

./distkeys -K somekey.pub -H somehost add

Host: somehost
Connecting to host somehost (user: ms, port: 9999)...
Opening SFTP session...
Key somekey added.
Creating a backup to .ssh/authorized_keys-2013-07-01.bak if not already done today...
Uploading keys to .ssh/authorized_keys-new...
File does exist and has correct size, moving to .ssh/authorized_keys...
/usr/lib/ruby/vendor_ruby/net/ssh/connection/session.rb:296:in `[]=': can't add a new key into hash during iteration (RuntimeError)
        from /usr/lib/ruby/vendor_ruby/net/ssh/connection/session.rb:296:in `open_channel'
        from /usr/lib/ruby/vendor_ruby/net/ssh/connection/session.rb:320:in `exec'
        from /usr/lib/ruby/vendor_ruby/net/ssh/connection/session.rb:354:in `exec!'
        from ./distkeys:206:in `block in commit'
        from /var/lib/gems/1.9.1/gems/net-sftp-2.1.2/lib/net/sftp/request.rb:87:in `call'
        from /var/lib/gems/1.9.1/gems/net-sftp-2.1.2/lib/net/sftp/request.rb:87:in `respond_to'
        from /var/lib/gems/1.9.1/gems/net-sftp-2.1.2/lib/net/sftp/session.rb:948:in `dispatch_request'
        from /var/lib/gems/1.9.1/gems/net-sftp-2.1.2/lib/net/sftp/session.rb:911:in `when_channel_polled'
        from /usr/lib/ruby/vendor_ruby/net/ssh/connection/channel.rb:311:in `call'
        from /usr/lib/ruby/vendor_ruby/net/ssh/connection/channel.rb:311:in `process'
        from /usr/lib/ruby/vendor_ruby/net/ssh/connection/session.rb:214:in `block in preprocess'
        from /usr/lib/ruby/vendor_ruby/net/ssh/connection/session.rb:214:in `each'
        from /usr/lib/ruby/vendor_ruby/net/ssh/connection/session.rb:214:in `preprocess'
        from /usr/lib/ruby/vendor_ruby/net/ssh/connection/session.rb:197:in `process'
        from /usr/lib/ruby/vendor_ruby/net/ssh/connection/session.rb:161:in `block in loop'
        from /usr/lib/ruby/vendor_ruby/net/ssh/connection/session.rb:161:in `loop'
        from /usr/lib/ruby/vendor_ruby/net/ssh/connection/session.rb:161:in `loop'
        from /var/lib/gems/1.9.1/gems/net-sftp-2.1.2/lib/net/sftp/session.rb:802:in `loop'
        from /var/lib/gems/1.9.1/gems/net-sftp-2.1.2/lib/net/sftp/request.rb:72:in `wait'
        from /var/lib/gems/1.9.1/gems/net-sftp-2.1.2/lib/net/sftp/session.rb:842:in `wait_for'
        from /var/lib/gems/1.9.1/gems/net-sftp-2.1.2/lib/net/sftp/session.rb:320:in `lstat!'
        from ./distkeys:200:in `commit'
        from ./distkeys:571:in `handle_host'
        from ./distkeys:677:in `block in handle_gwhost'
        from ./distkeys:660:in `each'
        from ./distkeys:660:in `handle_gwhost'
        from ./distkeys:692:in `loop'
        from ./distkeys:797:in `<main>'


I also tried after purging ruby-net-ssh which also removes ruby-net-sftp and
ruby-net-ssh-gateway and installing as gems:

mango:~# gem install net-sftp net-ssh net-ssh-gateway
Fetching: net-ssh-2.6.7.gem (100%)
Fetching: net-sftp-2.1.2.gem (100%)
Successfully installed net-ssh-2.6.7
Successfully installed net-sftp-2.1.2
Successfully installed net-ssh-2.6.7


This gives the following backtrace:

/var/lib/gems/1.9.1/gems/net-ssh-2.6.7/lib/net/ssh/connection/session.rb:299:in `[]=': can't add a new key into hash during iteration (RuntimeError)
        from /var/lib/gems/1.9.1/gems/net-ssh-2.6.7/lib/net/ssh/connection/session.rb:299:in `open_channel'
        from /var/lib/gems/1.9.1/gems/net-ssh-2.6.7/lib/net/ssh/connection/session.rb:323:in `exec'
        from /var/lib/gems/1.9.1/gems/net-ssh-2.6.7/lib/net/ssh/connection/session.rb:357:in `exec!'
        from ./distkeys:206:in `block in commit'
        from /var/lib/gems/1.9.1/gems/net-sftp-2.1.2/lib/net/sftp/request.rb:87:in `call'
        from /var/lib/gems/1.9.1/gems/net-sftp-2.1.2/lib/net/sftp/request.rb:87:in `respond_to'
        from /var/lib/gems/1.9.1/gems/net-sftp-2.1.2/lib/net/sftp/session.rb:948:in `dispatch_request'
        from /var/lib/gems/1.9.1/gems/net-sftp-2.1.2/lib/net/sftp/session.rb:911:in `when_channel_polled'
        from /var/lib/gems/1.9.1/gems/net-ssh-2.6.7/lib/net/ssh/connection/channel.rb:311:in `call'
        from /var/lib/gems/1.9.1/gems/net-ssh-2.6.7/lib/net/ssh/connection/channel.rb:311:in `process'
        from /var/lib/gems/1.9.1/gems/net-ssh-2.6.7/lib/net/ssh/connection/session.rb:217:in `block in preprocess'
        from /var/lib/gems/1.9.1/gems/net-ssh-2.6.7/lib/net/ssh/connection/session.rb:217:in `each'
        from /var/lib/gems/1.9.1/gems/net-ssh-2.6.7/lib/net/ssh/connection/session.rb:217:in `preprocess'
        from /var/lib/gems/1.9.1/gems/net-ssh-2.6.7/lib/net/ssh/connection/session.rb:200:in `process'
        from /var/lib/gems/1.9.1/gems/net-ssh-2.6.7/lib/net/ssh/connection/session.rb:164:in `block in loop'
        from /var/lib/gems/1.9.1/gems/net-ssh-2.6.7/lib/net/ssh/connection/session.rb:164:in `loop'
        from /var/lib/gems/1.9.1/gems/net-ssh-2.6.7/lib/net/ssh/connection/session.rb:164:in `loop'
        from /var/lib/gems/1.9.1/gems/net-sftp-2.1.2/lib/net/sftp/session.rb:802:in `loop'
        from /var/lib/gems/1.9.1/gems/net-sftp-2.1.2/lib/net/sftp/request.rb:72:in `wait'
        from /var/lib/gems/1.9.1/gems/net-sftp-2.1.2/lib/net/sftp/session.rb:842:in `wait_for'
        from /var/lib/gems/1.9.1/gems/net-sftp-2.1.2/lib/net/sftp/session.rb:320:in `lstat!'
        from ./distkeys:200:in `commit'
        from ./distkeys:571:in `handle_host'
        from ./distkeys:677:in `block in handle_gwhost'
        from ./distkeys:660:in `each'
        from ./distkeys:660:in `handle_gwhost'
        from ./distkeys:692:in `loop'
        from ./distkeys:797:in `<main>'

With Ruby 1.8 this works.


The code where this happens in distkeys as of commit
bf13f12e8ca3846998cc1cb610403ad958979377 (I will add a tag
with this bug report number) to the git repo:


		request = @sftp.lstat!(newauthkeyfile) do | response |
			if response.ok?
				# File size okay?
				if response[:attrs].size >= wantedsize
					puts "File does exist and has correct size, moving to #{@authkeyfile}..."
					# Move the new keyfile over the old one
					@ssh.exec!(  "mv  #{newauthkeyfile} #{@authkeyfile}" )

					# We saved the changes, so no unsaved changes anymore
					@changed = false
				end
			end
		end
		#@sftp.loop


URL to git repo is:

git://oss.teamix.org/distkeys.git


I am trying to work-around this issue by using sftp.rename now. I think its
a better choice than executing the mv command.

Unless there is some programming mistake in distkeys that Ruby 1.9 brings
to light I bet this is an upstream bug.

I am willing to forward / report upstream as well, but first wanted to have
this tracked in Debian BTS.


ms@mango:~> apt-show-versions | grep ruby-net
ruby-net-sftp/wheezy uptodate 1:2.0.5-3
ruby-net-ssh/wheezy uptodate 1:2.5.2-2
ruby-net-ssh-gateway/wheezy uptodate 1.1.0-2

Thanks,
Martin

-- System Information:
Debian Release: 7.1
  APT prefers stable
  APT policy: (500, 'stable'), (350, 'unstable'), (110, 'experimental')
Architecture: amd64 (x86_64)
Foreign Architectures: i386

Kernel: Linux 3.9-1-amd64 (SMP w/4 CPU cores)
Locale: LANG=de_DE.UTF-8, LC_CTYPE=de_DE.UTF-8 (charmap=UTF-8)
Shell: /bin/sh linked to /bin/dash

Versions of packages ruby-net-ssh depends on:
ii  ruby                          1:1.9.3
ii  ruby1.8 [ruby-interpreter]    1.8.7.358-7
ii  ruby1.9.1 [ruby-interpreter]  1.9.3.194-8.1

ruby-net-ssh recommends no packages.

ruby-net-ssh suggests no packages.

-- no debconf information

Message #10 received at 714606@bugs.debian.org (full text, mbox, reply):

From: Martin Steigerwald <ms@teamix.de>

To: 714606@bugs.debian.org

Subject: Reported upstream

Date: Mon, 1 Jul 2013 13:26:37 +0200

Hi!

ruby-net-ssh: can't add a new key into hash during iteration during 
ssh.exec
https://github.com/net-ssh/net-ssh/issues/110

Workaround with SFTP does work but needs removing the file prior to 
renaming it due to:

https://bugzilla.mindrot.org/show_bug.cgi?id=2123

Thanks,
-- 
Martin Steigerwald - teamix GmbH - http://www.teamix.de
gpg: 19E3 8D42 896F D004 08AC A0CA 1E10 C593 0399 AE90

Message #17 received at 714606@bugs.debian.org (full text, mbox, reply):

From: Martin Steigerwald <ms@teamix.de>

To: <debian-ruby@lists.debian.org>

Cc: <714606@bugs.debian.org>

Subject: help with fixing ruby-net-ssh: can't add a new key into hash during iteration during ssh.exec

Date: Tue, 14 Apr 2015 12:36:33 +0200

Cc'ing the bug report as well, feel free to drop the cc for discussion on mailing list.


Hi!

I seek help with fixing

https://bugs.debian.org/714606

aka

https://github.com/net-ssh/net-ssh/issues/110


The error message on trying ssh.exec is:

/usr/lib/ruby/vendor_ruby/net/ssh/connection/session.rb:305:in `open_channel': can't add a new key into hash during iteration (RuntimeError)


Here is the offending source:

https://github.com/net-ssh/net-ssh/blob/master/lib/net/ssh/connection/session.rb#L306


As far as I get the error is due to method exed in same file using

open_channel do |channel|

which then puts the assignment

channels[local_id] = channel

in open_channel into an iteration.

But I asked on #ruby-de and the do / end there is a block, not an iteration.

However in the backtrace there is

/usr/lib/ruby/vendor_ruby/net/ssh/connection/session.rb:305:in `open_channel': can't add a new key into hash during iteration (RuntimeError)
        from /usr/lib/ruby/vendor_ruby/net/ssh/connection/session.rb:329:in `exec'
        from /usr/lib/ruby/vendor_ruby/net/ssh/connection/session.rb:363:in `exec!'
        from /homelokal/ms/Debian/distkeys/distkeys.git/distkeys:174:in `block in commit'
        from /usr/lib/ruby/vendor_ruby/net/sftp/request.rb:87:in `call'
        from /usr/lib/ruby/vendor_ruby/net/sftp/request.rb:87:in `respond_to'
        from /usr/lib/ruby/vendor_ruby/net/sftp/session.rb:948:in `dispatch_request'
        from /usr/lib/ruby/vendor_ruby/net/sftp/session.rb:911:in `when_channel_polled'
        from /usr/lib/ruby/vendor_ruby/net/ssh/connection/channel.rb:311:in `call'
        from /usr/lib/ruby/vendor_ruby/net/ssh/connection/channel.rb:311:in `process'
        from /usr/lib/ruby/vendor_ruby/net/ssh/connection/session.rb:222:in `block in preprocess'
        from /usr/lib/ruby/vendor_ruby/net/ssh/connection/session.rb:222:in `each'
        from /usr/lib/ruby/vendor_ruby/net/ssh/connection/session.rb:222:in `preprocess'
        from /usr/lib/ruby/vendor_ruby/net/ssh/connection/session.rb:205:in `process'
        from /usr/lib/ruby/vendor_ruby/net/ssh/connection/session.rb:169:in `block in loop'
        from /usr/lib/ruby/vendor_ruby/net/ssh/connection/session.rb:169:in `loop'
        from /usr/lib/ruby/vendor_ruby/net/ssh/connection/session.rb:169:in `loop'
        from /usr/lib/ruby/vendor_ruby/net/ssh/connection/channel.rb:269:in `wait'
        from /usr/lib/ruby/vendor_ruby/net/ssh/connection/session.rb:364:in `exec!'
        from /homelokal/ms/Debian/distkeys/distkeys.git/distkeys:184:in `commit'
        from /homelokal/ms/Debian/distkeys/distkeys.git/distkeys:628:in `handle_host'
        from /homelokal/ms/Debian/distkeys/distkeys.git/distkeys:703:in `block in handle_gwhost'
        from /homelokal/ms/Debian/distkeys/distkeys.git/distkeys:686:in `each'
        from /homelokal/ms/Debian/distkeys/distkeys.git/distkeys:686:in `handle_gwhost'
        from /homelokal/ms/Debian/distkeys/distkeys.git/distkeys:718:in `loop'
        from /homelokal/ms/Debian/distkeys/distkeys.git/distkeys:828:in `<main>'

an each in session.rb, line 222.


Which looks quite central to the working of ruby-net-ssh to me

    # This is called internally as part of #process. It dispatches any
    # available incoming packets, and then runs Net::SSH::Connection::Channel#process
    # for any active channels. If a block is given, it is invoked at the
    # start of the method and again at the end, and if the block ever returns
    # false, this method returns false. Otherwise, it returns true.
    def preprocess
      return false if block_given? && !yield(self)
      dispatch_incoming_packets
      channels.each { |id, channel| channel.process unless channel.closing? }
      return false if block_given? && !yield(self)
      return true
    end



The calling site inside distkeys is:

https://github.com/teamix/distkeys/blob/master/distkeys#L174



Do you see a way to fix this up without changing the semantics of open_channel?


Another conclusion would be to treat ruby-net-ssh as unfit for release
with Debian Jessie, as a central functionality just does not work with Ruby
1.9+, and Jessie does not have Ruby 1.8 anymore.

But even tough upstream appears to be basically unmaintained (see note
about maintenance note at https://github.com/net-ssh/net-ssh) I think it would
be nice to have this fixed. But anyway, if its unmaintained it may be better
to remove it? I don't know about any other free as in freedom alternative to
it tough.

Thanks,

-- 
Martin Steigerwald  | Consultant / Trainer

teamix GmbH
Südwestpark 43
90449 Nürnberg

Tel.:  +49 911 30999 55 | Fax: +49 911 30999 99
mail: martin.steigerwald@teamix.de | web:  http://www.teamix.de | blog: http://blog.teamix.de

Amtsgericht Nürnberg, HRB 18320 | Geschäftsführer: Oliver Kügow, Richard Müller


** Data Management Day | 29.04.2015 bei teamix **
Jetzt anmelden unter www.teamix.de/CommVault

Message #22 received at 714606@bugs.debian.org (full text, mbox, reply):

From: Martin Steigerwald <ms@teamix.de>

To: <debian-ruby@lists.debian.org>

Cc: <714606@bugs.debian.org>

Subject: Re: help with fixing ruby-net-ssh: can't add a new key into hash during iteration during ssh.exec

Date: Tue, 14 Apr 2015 13:00:19 +0200

Am Dienstag, 14. April 2015, 12:36:33 schrieb Martin Steigerwald:
> Cc'ing the bug report as well, feel free to drop the cc for discussion on mailing list.
> 
> 
> Hi!
> 
> I seek help with fixing
> 
> https://bugs.debian.org/714606
> 
> aka
> 
> https://github.com/net-ssh/net-ssh/issues/110
> 
> 
> The error message on trying ssh.exec is:
> 
> /usr/lib/ruby/vendor_ruby/net/ssh/connection/session.rb:305:in `open_channel': can't add a new key into hash during iteration (RuntimeError)
> 
> 
> Here is the offending source:
> 
> https://github.com/net-ssh/net-ssh/blob/master/lib/net/ssh/connection/session.rb#L306
> 
> 
> As far as I get the error is due to method exed in same file using
> 
> open_channel do |channel|
> 
> which then puts the assignment
> 
> channels[local_id] = channel
> 
> in open_channel into an iteration.
> 
> But I asked on #ruby-de and the do / end there is a block, not an iteration.
> 
> However in the backtrace there is
> 
> /usr/lib/ruby/vendor_ruby/net/ssh/connection/session.rb:305:in `open_channel': can't add a new key into hash during iteration (RuntimeError)
>         from /usr/lib/ruby/vendor_ruby/net/ssh/connection/session.rb:329:in `exec'
>         from /usr/lib/ruby/vendor_ruby/net/ssh/connection/session.rb:363:in `exec!'
>         from /homelokal/ms/Debian/distkeys/distkeys.git/distkeys:174:in `block in commit'
>         from /usr/lib/ruby/vendor_ruby/net/sftp/request.rb:87:in `call'
>         from /usr/lib/ruby/vendor_ruby/net/sftp/request.rb:87:in `respond_to'
>         from /usr/lib/ruby/vendor_ruby/net/sftp/session.rb:948:in `dispatch_request'
>         from /usr/lib/ruby/vendor_ruby/net/sftp/session.rb:911:in `when_channel_polled'
>         from /usr/lib/ruby/vendor_ruby/net/ssh/connection/channel.rb:311:in `call'
>         from /usr/lib/ruby/vendor_ruby/net/ssh/connection/channel.rb:311:in `process'
>         from /usr/lib/ruby/vendor_ruby/net/ssh/connection/session.rb:222:in `block in preprocess'
>         from /usr/lib/ruby/vendor_ruby/net/ssh/connection/session.rb:222:in `each'
>         from /usr/lib/ruby/vendor_ruby/net/ssh/connection/session.rb:222:in `preprocess'
>         from /usr/lib/ruby/vendor_ruby/net/ssh/connection/session.rb:205:in `process'
>         from /usr/lib/ruby/vendor_ruby/net/ssh/connection/session.rb:169:in `block in loop'
>         from /usr/lib/ruby/vendor_ruby/net/ssh/connection/session.rb:169:in `loop'
>         from /usr/lib/ruby/vendor_ruby/net/ssh/connection/session.rb:169:in `loop'
>         from /usr/lib/ruby/vendor_ruby/net/ssh/connection/channel.rb:269:in `wait'
>         from /usr/lib/ruby/vendor_ruby/net/ssh/connection/session.rb:364:in `exec!'
>         from /homelokal/ms/Debian/distkeys/distkeys.git/distkeys:184:in `commit'
>         from /homelokal/ms/Debian/distkeys/distkeys.git/distkeys:628:in `handle_host'
>         from /homelokal/ms/Debian/distkeys/distkeys.git/distkeys:703:in `block in handle_gwhost'
>         from /homelokal/ms/Debian/distkeys/distkeys.git/distkeys:686:in `each'
>         from /homelokal/ms/Debian/distkeys/distkeys.git/distkeys:686:in `handle_gwhost'
>         from /homelokal/ms/Debian/distkeys/distkeys.git/distkeys:718:in `loop'
>         from /homelokal/ms/Debian/distkeys/distkeys.git/distkeys:828:in `<main>'
> 
> an each in session.rb, line 222.
> 
> 
> Which looks quite central to the working of ruby-net-ssh to me
> 
>     # This is called internally as part of #process. It dispatches any
>     # available incoming packets, and then runs Net::SSH::Connection::Channel#process
>     # for any active channels. If a block is given, it is invoked at the
>     # start of the method and again at the end, and if the block ever returns
>     # false, this method returns false. Otherwise, it returns true.
>     def preprocess
>       return false if block_given? && !yield(self)
>       dispatch_incoming_packets
>       channels.each { |id, channel| channel.process unless channel.closing? }
>       return false if block_given? && !yield(self)
>       return true
>     end
> 
> 
> 
> The calling site inside distkeys is:
> 
> https://github.com/teamix/distkeys/blob/master/distkeys#L174
> 
> 
> 
> Do you see a way to fix this up without changing the semantics of open_channel?

Okay, I now got further help from #ruby-de and there is a fix I can do inside distkeys:

		ok=false
		@sftp.lstat( ".ssh" ) do |response| ok = response.ok?; end
		if not ok then
				puts "~/.ssh does not seem to exist, creating it with 700..."
				@ssh.exec!( "mkdir ~/.ssh" )
				@ssh.exec!( "chmod 700 ~/.ssh" )
		end

i.e. first finish sftp then to the ssh exec stuff.

Pushed:
https://github.com/teamix/distkeys/commit/1092384f54d6531ce1106c4fe7b2f6833a2bba5b


I can now fix all other occurences of this in my script.

I am not sure whether it is a work-around or whether it is a valid
contraint to be taken into account when using ruby-net-ssh. To me it
feels like a work-around, but well… if it works this way.

If you still have an idea how to fix it in ruby-net-ssh, please tell me.

Thanks,

-- 
Martin Steigerwald  | Consultant / Trainer

teamix GmbH
Südwestpark 43
90449 Nürnberg

Tel.:  +49 911 30999 55 | Fax: +49 911 30999 99
mail: martin.steigerwald@teamix.de | web:  http://www.teamix.de | blog: http://blog.teamix.de

Amtsgericht Nürnberg, HRB 18320 | Geschäftsführer: Oliver Kügow, Richard Müller


** Data Management Day | 29.04.2015 bei teamix **
Jetzt anmelden unter www.teamix.de/CommVault

Message #27 received at 714606@bugs.debian.org (full text, mbox, reply):

From: David Suarez <david.sephirot@gmail.com>

To: Martin Steigerwald <ms@teamix.de>

Cc: debian-ruby@lists.debian.org, 714606@bugs.debian.org

Subject: Re: help with fixing ruby-net-ssh: can't add a new key into hash during iteration during ssh.exec

Date: Tue, 14 Apr 2015 14:39:04 +0200

Hi,

2015-04-14 13:00 GMT+02:00 Martin Steigerwald <ms@teamix.de>:
> Am Dienstag, 14. April 2015, 12:36:33 schrieb Martin Steigerwald:
>> Cc'ing the bug report as well, feel free to drop the cc for discussion on mailing list.
>> an each in session.rb, line 222.
>>
>>
>> Which looks quite central to the working of ruby-net-ssh to me
>>
>>     # This is called internally as part of #process. It dispatches any
>>     # available incoming packets, and then runs Net::SSH::Connection::Channel#process
>>     # for any active channels. If a block is given, it is invoked at the
>>     # start of the method and again at the end, and if the block ever returns
>>     # false, this method returns false. Otherwise, it returns true.
>>     def preprocess
>>       return false if block_given? && !yield(self)
>>       dispatch_incoming_packets
>>       channels.each { |id, channel| channel.process unless channel.closing? }
>>       return false if block_given? && !yield(self)
>>       return true
>>     end
>>
>>
>>
>> The calling site inside distkeys is:
>>
>> https://github.com/teamix/distkeys/blob/master/distkeys#L174
>>

You are right, 'channels' Hash is modified inside 'channels.each' call.

> I am not sure whether it is a work-around or whether it is a valid
> contraint to be taken into account when using ruby-net-ssh. To me it
> feels like a work-around, but well… if it works this way.
>
> If you still have an idea how to fix it in ruby-net-ssh, please tell me.

Seems like a valid constrain, due that the problem arise when you are
trying to open a new channel (ssh.exec!) inside the processing block
of the another channel (sftp.lstat) in the same ssh session.

One fix could be that instead os reusing the actual ssh connection to
open the sftp one ""@sftp = Net::SFTP::Session.new(@ssh)"", create a
new sftp connection ""Net::SFTP.start(host, user, options)"".

Cheers,

  David

Message #32 received at 714606@bugs.debian.org (full text, mbox, reply):

From: Martin Steigerwald <ms@teamix.de>

To: David Suarez <david.sephirot@gmail.com>

Cc: <debian-ruby@lists.debian.org>, <714606@bugs.debian.org>

Subject: Re: help with fixing ruby-net-ssh: can't add a new key into hash during iteration during ssh.exec

Date: Tue, 14 Apr 2015 15:12:57 +0200

Am Dienstag, 14. April 2015, 14:39:04 schrieb David Suarez:
> Hi,
> 
> 2015-04-14 13:00 GMT+02:00 Martin Steigerwald <ms@teamix.de>:
> > Am Dienstag, 14. April 2015, 12:36:33 schrieb Martin Steigerwald:
> >> Cc'ing the bug report as well, feel free to drop the cc for discussion on mailing list.
> >> an each in session.rb, line 222.
> >>
> >>
> >> Which looks quite central to the working of ruby-net-ssh to me
> >>
> >>     # This is called internally as part of #process. It dispatches any
> >>     # available incoming packets, and then runs Net::SSH::Connection::Channel#process
> >>     # for any active channels. If a block is given, it is invoked at the
> >>     # start of the method and again at the end, and if the block ever returns
> >>     # false, this method returns false. Otherwise, it returns true.
> >>     def preprocess
> >>       return false if block_given? && !yield(self)
> >>       dispatch_incoming_packets
> >>       channels.each { |id, channel| channel.process unless channel.closing? }
> >>       return false if block_given? && !yield(self)
> >>       return true
> >>     end
> >>
> >>
> >>
> >> The calling site inside distkeys is:
> >>
> >> https://github.com/teamix/distkeys/blob/master/distkeys#L174
> >>
> 
> You are right, 'channels' Hash is modified inside 'channels.each' call.
> 
> > I am not sure whether it is a work-around or whether it is a valid
> > contraint to be taken into account when using ruby-net-ssh. To me it
> > feels like a work-around, but well… if it works this way.
> >
> > If you still have an idea how to fix it in ruby-net-ssh, please tell me.
> 
> Seems like a valid constrain, due that the problem arise when you are
> trying to open a new channel (ssh.exec!) inside the processing block
> of the another channel (sftp.lstat) in the same ssh session.
> 
> One fix could be that instead os reusing the actual ssh connection to
> open the sftp one ""@sftp = Net::SFTP::Session.new(@ssh)"", create a
> new sftp connection ""Net::SFTP.start(host, user, options)"".

Thanks.

I now just finish the sftp operation before doing the ssh.exec calls and
this appears to work. Just uploaded distkeys-1.1 to github including 1.1
debian package source. This also fixes the same for the replacing of
authorized_keys with authorized_keys-new which was racy before due to
work-arounding this issue by deleting first and then sftp.rename() which
in current ruby-net-sftp cannot overwrite a file.

This appears to work just nice. And I also documented this behavior in
the script.

Will test internally for a while and then probably reapproach with request
for sponsoring for distkeys. Maybe one day will get distkeys into Debian,
I think its quite useful, cause it can put ssh keys to hosts behind
firewalls using ssh port forwarding automatically. After Jessie release :).

As for this bug, if you think its a valid constraint, feel free to close it.

I still think it would be good to at least have this documented somewhere
with ruby-net-ssh, cause the interference between ssh and sftp calls are not
that obvious but due to a implementation detail.

But I can try to send a documentation patch upstream.


-- 
Martin Steigerwald  | Consultant / Trainer

teamix GmbH
Südwestpark 43
90449 Nürnberg

Tel.:  +49 911 30999 55 | Fax: +49 911 30999 99
mail: martin.steigerwald@teamix.de | web:  http://www.teamix.de | blog: http://blog.teamix.de

Amtsgericht Nürnberg, HRB 18320 | Geschäftsführer: Oliver Kügow, Richard Müller


** Data Management Day | 29.04.2015 bei teamix **
Jetzt anmelden unter www.teamix.de/CommVault

Message #39 received at 714606-done@bugs.debian.org (full text, mbox, reply):

From: Lucas Nussbaum <lucas@debian.org>

To: Martin Steigerwald <ms@teamix.de>, 714606-done@bugs.debian.org

Cc: David Suarez <david.sephirot@gmail.com>, debian-ruby@lists.debian.org

Subject: Re: Bug#714606: help with fixing ruby-net-ssh: can't add a new key into hash during iteration during ssh.exec

Date: Thu, 28 Jul 2016 08:49:53 +0200

On 14/04/15 at 15:12 +0200, Martin Steigerwald wrote:
> Am Dienstag, 14. April 2015, 14:39:04 schrieb David Suarez:
> > Hi,
> > 
> > 2015-04-14 13:00 GMT+02:00 Martin Steigerwald <ms@teamix.de>:
> > > Am Dienstag, 14. April 2015, 12:36:33 schrieb Martin Steigerwald:
> > >> Cc'ing the bug report as well, feel free to drop the cc for discussion on mailing list.
> > >> an each in session.rb, line 222.
> > >>
> > >>
> > >> Which looks quite central to the working of ruby-net-ssh to me
> > >>
> > >>     # This is called internally as part of #process. It dispatches any
> > >>     # available incoming packets, and then runs Net::SSH::Connection::Channel#process
> > >>     # for any active channels. If a block is given, it is invoked at the
> > >>     # start of the method and again at the end, and if the block ever returns
> > >>     # false, this method returns false. Otherwise, it returns true.
> > >>     def preprocess
> > >>       return false if block_given? && !yield(self)
> > >>       dispatch_incoming_packets
> > >>       channels.each { |id, channel| channel.process unless channel.closing? }
> > >>       return false if block_given? && !yield(self)
> > >>       return true
> > >>     end
> > >>
> > >>
> > >>
> > >> The calling site inside distkeys is:
> > >>
> > >> https://github.com/teamix/distkeys/blob/master/distkeys#L174
> > >>
> > 
> > You are right, 'channels' Hash is modified inside 'channels.each' call.
> > 
> > > I am not sure whether it is a work-around or whether it is a valid
> > > contraint to be taken into account when using ruby-net-ssh. To me it
> > > feels like a work-around, but well… if it works this way.
> > >
> > > If you still have an idea how to fix it in ruby-net-ssh, please tell me.
> > 
> > Seems like a valid constrain, due that the problem arise when you are
> > trying to open a new channel (ssh.exec!) inside the processing block
> > of the another channel (sftp.lstat) in the same ssh session.
> > 
> > One fix could be that instead os reusing the actual ssh connection to
> > open the sftp one ""@sftp = Net::SFTP::Session.new(@ssh)"", create a
> > new sftp connection ""Net::SFTP.start(host, user, options)"".
> 
> Thanks.
> 
> I now just finish the sftp operation before doing the ssh.exec calls and
> this appears to work. Just uploaded distkeys-1.1 to github including 1.1
> debian package source. This also fixes the same for the replacing of
> authorized_keys with authorized_keys-new which was racy before due to
> work-arounding this issue by deleting first and then sftp.rename() which
> in current ruby-net-sftp cannot overwrite a file.
> 
> This appears to work just nice. And I also documented this behavior in
> the script.
> 
> Will test internally for a while and then probably reapproach with request
> for sponsoring for distkeys. Maybe one day will get distkeys into Debian,
> I think its quite useful, cause it can put ssh keys to hosts behind
> firewalls using ssh port forwarding automatically. After Jessie release :).
> 
> As for this bug, if you think its a valid constraint, feel free to close it.
> 
> I still think it would be good to at least have this documented somewhere
> with ruby-net-ssh, cause the interference between ssh and sftp calls are not
> that obvious but due to a implementation detail.
> 
> But I can try to send a documentation patch upstream.

Hi,

I don't think that this is a bug in ruby-net-ssh, as the inability to
modify a hash during iteration is a Ruby limitation. See e.g:

irb(main):005:0> h = { :a => 1}
=> {:a=>1}
irb(main):006:0> h.each_pair { |k, v| h[:b] = 2 }
RuntimeError: can't add a new key into hash during iteration
	from (irb):6:in `block in irb_binding'
	from (irb):6:in `each_pair'
	from (irb):6
	from /usr/bin/irb:11:in `<main>'

I'm closing this bug.

Lucas

Send a report that this bug log contains spam.