Debian Bug report logs - #474094
Directory name completion fails if name contains spaces and is quoted

Toggle useless messages

Message #5 received at submit@bugs.debian.org (full text, mbox, reply):

From: "Jake K" <jakekaa@gmail.com>

To: submit@bugs.debian.org

Subject: Directory name completion fails if name contains spaces and is quoted

Date: Thu, 3 Apr 2008 12:38:01 +0300

Package: bash-completion
Version: 20060301-4

Completion fails if  directory name contains spaces and is quoted:

$ mkdir temp
$ cd temp
/temp$ mkdir 'album first'
/temp$ mkdir 'album second'
/temp$ mkdir 'first'
/temp$ cd 'al[TAB] -> cd 'album[TAB]TAB]
album firts album second first
/temp$ cd 'album f[TAB] -> cd 'first\ <ctrl-c>
/temp$ cd 'album s[TAB][TAB] -> does not complete

I think cd 'album f[TAB] should complete to cd 'album first'/ and cd
'album s[TAB][TAB] to cd 'album second'/.
I have updated /etc/bash_completion with the maintainer's one.

I am using Debian sid, bash 3.1dfsg-9 and ucf 3.005.

Message #10 received at 474094@bugs.debian.org (full text, mbox, reply):

From: Morita Sho <morita-pub-en-debian@inz.sakura.ne.jp>

To: Debian Bug Tracking System <474094@bugs.debian.org>

Subject: bash-completion: Suggestion to improve _get_cword() for #474094

Date: Fri, 02 May 2008 13:22:11 +0900

Package: bash-completion
Version: 20060301-4
Followup-For: Bug #474094

Hi,

Today I installed bash-completion and I got same bug.

This bug also affects to file name as well as directory name.

$ touch "foo bar.zip"
$ ls
foo bar.zip
$ unzip "foo b[TAB]  ->  does not work

In addition to spaces, it is caused by other characters specified in $COMP_WORDBREAKS variable.

$ echo -n "$COMP_WORDBREAKS" | hexdump -C
00000000  20 09 0a 22 27 3e 3c 3d  3b 7c 26 28 3a           | .."'><=;|&(:|
0000000d
$ touch "foo(barbarbar).avi" "123=456.avi" "abc&def.avi"
$ ls
123=456.avi  abc&def.avi  foo(barbarbar).avi
$ mplayer "123=4[TAB]    ->  does not work
$ mplayer "abc&d[TAB]    ->  does not work
$ mplayer "foo(bar[TAB]  ->  does not work

Although quoted by " or ' does not work, quoted by \ works correctly.
$ unzip foo\ b[TAB]   ->  foo\ bar.zip 

It seems that the condition to produce this bug is as follow:
* the name contains one of $COMP_WORDBREAKS characters,
* and is quoted by " or '
* and press TAB after $COMP_WORDBREAKS character.


After some investigation for /etc/bash_completion,
I found the function _get_cword() has a problem.

The function _get_cword() was originally introduced to fix following bug in ubuntu:
Bug #139666 in bash-completion (Ubuntu): “bash completion doesn't work in the middle of a word”
https://bugs.launchpad.net/ubuntu/+source/bash-completion/+bug/139666

The function _get_cword() returns the current word that cursor is pointed.
The problem is, _get_cword considers only \ quoting, not considers " or ' quoting.
So that, _get_cword thinks the word  "foo b  is 2 words, '"foo' and 'b'.
That is a reason why " or ' quoting not work although \ quoting works.


I have 2 ideas to fix the problem.
1) revert to old (before #139666) behaviour.
_get_cword() {
  printf "%s" "${COMP_WORDS[$COMP_CWORD]}";
}

2) improve current behaviour to handle " and ' quoting correctly.
# Quote regular expression meta characters
_quotemeta()
{
  printf "%s" "$( printf "%s" "$1" | sed 's/\\/\\\\/g; s/[.*[$^]/\\&/g; s/[?+{()|]/[&]/g' )"
}

# Get the word to complete
# This is nicer than ${COMP_WORDS[$COMP_CWORD]}, since it handles cases
# where the user is completing in the middle of a word.
# (For example, if the line is "ls foobar",
# and the cursor is here -------->   ^
# it will complete just "foo", not "foobar", which is what the user wants.)
_get_cword()
{
  local pattern
  local i

  pattern=$'[ \t\n]*'
  for (( i=0; i < $COMP_CWORD; i++ )); do
    pattern="${pattern}$( _quotemeta "${COMP_WORDS[$i]}" )"$'[ \t\n]\+'
  done

  local offset_for_cword="$( expr "$COMP_LINE" : "$pattern" )"
  local point_on_cword=$(( $COMP_POINT - $offset_for_cword ));
  local LC_CTYPE=C
  printf "%s" "${COMP_LINE:$offset_for_cword:$point_on_cword}"
}


Note: I think it is better to use printf instead of echo.
echo prints empty string in case of echo "$str" and $str is "-e" or "-n" (a string that is option for echo.)
So if we use echo, completion for a string that start with "-e" or "-n" works incorrectly.


Regards,

-- System Information:
Debian Release: lenny/sid
  APT prefers unstable
  APT policy: (500, 'unstable')
Architecture: i386 (i686)

Kernel: Linux 2.6.25-1-686 (SMP w/1 CPU core)
Locale: LANG=ja_JP.UTF-8, LC_CTYPE=ja_JP.UTF-8 (charmap=UTF-8)
Shell: /bin/sh linked to /bin/bash

Versions of packages bash-completion depends on:
ii  bash                          3.2-2      The GNU Bourne Again SHell
ii  ucf                           3.006      Update Configuration File: preserv

bash-completion recommends no packages.

-- no debconf information

Message #17 received at 474094-quiet@bugs.debian.org (full text, mbox, reply):

From: David Paleino <d.paleino@gmail.com>

To: 474094-quiet@bugs.debian.org

Cc: Bash-Completion Developers <bash-completion-devel@lists.alioth.debian.org>, Jake K <jakekaa@gmail.com>, Morita Sho <morita-pub-en-debian@inz.sakura.ne.jp>

Subject: Re: Directory name completion fails if name contains spaces and is quoted

Date: Sun, 11 May 2008 17:26:34 +0200

[Message part 1 (text/plain, inline)]

tags 474094 confirmed patch
thanks

Hi Jake, hi Morita,

On Thu, 3 Apr 2008 12:38:01 +0300, Jake K wrote:

> Completion fails if  directory name contains spaces and is quoted: [..]


On Fri, 02 May 2008 13:22:11 +0900, Morita Sho wrote:

> Today I installed bash-completion and I got same bug.
> [..]
> After some investigation for /etc/bash_completion,
> I found the function _get_cword() has a problem.
> 
> The function _get_cword() was originally introduced to fix following bug in
> ubuntu: Bug #139666 in bash-completion (Ubuntu): “bash completion doesn't work
> in the middle of a word”
> https://bugs.launchpad.net/ubuntu/+source/bash-completion/+bug/139666
> 
> The function _get_cword() returns the current word that cursor is pointed.
> The problem is, _get_cword considers only \ quoting, not considers " or '
> quoting. So that, _get_cword thinks the word  "foo b  is 2 words, '"foo' and
> 'b'. That is a reason why " or ' quoting not work although \ quoting works.
> 
> 
> I have 2 ideas to fix the problem. [..]

Thanks for the report, and thanks Morita for the ideas posted.
We (the Bash-Completion Team, CCed) have taken upstream development, and we're
currently working on a new release (which is mainly bug-fix).

Morita, you can have a look at what _get_cword looks like at our Bzr
repository [0]. That is the result of a merge of Ubuntu's, Debian's and
upstream's code. I'm currently looking for a way to merge your code with this,
but it won't be that simple ;) -- jokes aside, I believe we *really* need to
clean up that code a bit.

In the meanwhile, I'll wait for the response of other Bash-Completion
developers.

Thanks for the patch (and the report),
David

[0] $ bzr co http://bzr.debian.org/bash-completion/current/ bash-completion

-- 
 . ''`.  Debian maintainer | http://wiki.debian.org/DavidPaleino
 : :'  : Linuxer #334216 --|-- http://www.hanskalabs.net/
 `. `'`  GPG: 1392B174 ----|---- http://snipr.com/qa_page
   `-   2BAB C625 4E66 E7B8 450A C3E1 E6AA 9017 1392 B174

[signature.asc (application/pgp-signature, attachment)]

Message #24 received at 474094-quiet@bugs.debian.org (full text, mbox, reply):

From: David Paleino <d.paleino@gmail.com>

To: 474094-quiet@bugs.debian.org

Cc: Bash-Completion Developers <bash-completion-devel@lists.alioth.debian.org>, Jake K <jakekaa@gmail.com>, Morita Sho <morita-pub-en-debian@inz.sakura.ne.jp>

Subject: Re: Directory name completion fails if name contains spaces and is quoted

Date: Sun, 11 May 2008 18:00:55 +0200

[Message part 1 (text/plain, inline)]

On Sun, 11 May 2008 17:26:34 +0200, David Paleino wrote:

> On Fri, 02 May 2008 13:22:11 +0900, Morita Sho wrote:
> 
> > The function _get_cword() returns the current word that cursor is pointed.
> > The problem is, _get_cword considers only \ quoting, not considers " or '
> > quoting. So that, _get_cword thinks the word  "foo b  is 2 words, '"foo' and
> > 'b'. That is a reason why " or ' quoting not work although \ quoting works.
> > 
> > 
> > I have 2 ideas to fix the problem. [..]
> 
> Thanks for the report, and thanks Morita for the ideas posted.
> We (the Bash-Completion Team, CCed) have taken upstream development, and we're
> currently working on a new release (which is mainly bug-fix).

I've currently patched _get_cword()... please try to enclose into quotes the
last line of the function:

echo "$cur"

instead of

echo $cur


This works for me (and, yes, it also considers " and ' as quoting characters
now).

Tagging the bug as "pending", waiting for a reply.


Kindly,
David

-- 
 . ''`.  Debian maintainer | http://wiki.debian.org/DavidPaleino
 : :'  : Linuxer #334216 --|-- http://www.hanskalabs.net/
 `. `'`  GPG: 1392B174 ----|---- http://snipr.com/qa_page
   `-   2BAB C625 4E66 E7B8 450A C3E1 E6AA 9017 1392 B174

[signature.asc (application/pgp-signature, attachment)]

Message #29 received at 474094-quiet@bugs.debian.org (full text, mbox, reply):

From: Morita Sho <morita-pub-en-debian@inz.sakura.ne.jp>

To: David Paleino <d.paleino@gmail.com>

Cc: 474094-quiet@bugs.debian.org, Bash-Completion Developers <bash-completion-devel@lists.alioth.debian.org>, Jake K <jakekaa@gmail.com>

Subject: Re: Directory name completion fails if name contains spaces and is quoted

Date: Mon, 12 May 2008 05:36:13 +0900

Hi,

On Sun, 11 May 2008 17:26:34 +0200, David Paleino wrote:

> Morita, you can have a look at what _get_cword looks like at our Bzr
> repository [0]. That is the result of a merge of Ubuntu's, Debian's and
> upstream's code. I'm currently looking for a way to merge your code with this,
> but it won't be that simple ;) -- jokes aside, I believe we *really* need to
> clean up that code a bit.

> [0] $ bzr co http://bzr.debian.org/bash-completion/current/ bash-completion

OK, I'll have a look at that.

Because I have never used bzr, I installed bzr.
But when I try to checkout, I had an error:

  $ bzr co http://bzr.debian.org/bash-completion/current/ bash-completion
  bzr: ERROR: Transport error: Server refuses to fullfil the request

A log file ~/.bzr.log told me

  failed to import pycurl: No module named pycurl

so I installed python-pycurl, and try checkout again, it works.

I'm not sure what's wrong, but I can start to look latest code.


First of all, this command

  $ source bash_completion

shows some error. so I need to fix it. [1]

Plus, once bash_completion is sourced, they set variables BASH_COMPLETION and 
BASH_COMPLETION_DIR,
and when bash_completion is sourced next time, they stop to reading the code. 
(Is this intended behavior?)
It is not useful for developing, I need to uncomment following line:

  readonly BASH_COMPLETION BASH_COMPLETION_DIR


On Sun, 11 May 2008 18:00:55 +0200, David Paleino wrote:

> I've currently patched _get_cword()... please try to enclose into quotes the
> last line of the function:
>
> echo "$cur"
>
> instead of
>
> echo $cur
>
>
> This works for me (and, yes, it also considers " and ' as quoting characters
> now).
>
> Tagging the bug as "pending", waiting for a reply.

Hmmmm... I can see $cur has been enclosed into quotes in latest code, but

  $ source bash_completion
  $ mkdir "FOO BAR"
  $ cd "FOO B[TAB]

is still not works for me...

IMHO, the difference between "$cur" and $cur is whether word splitting is occurs 
or not,
and word splitting does not considers quoting.
I think, although changing $cur to "$cur" resolves some space-related problems,
it will not resolves quote-related problems.


Thanks,

[1]

--- bash_completion	2008-05-11 16:35:16 +0000
+++ bash_completion	2008-05-11 18:06:47 +0000
@@ -2560,7 +2560,7 @@
 	    fi
 	    # append any available aliases from config files
 	    if [ ${#config[@]} -gt 0 ] && [ -n "$aliases" ]; then
-	    local host_aliases=$( sed -ne 
's/^[Hh][Oo][Ss][Tt]\([Nn][Aa][Mm][Ee]\)\?['"$'\t '"']*\([^*?]*\)$/\2/p' 
${config[@]} )" )
+	    local host_aliases=$( sed -ne 
's/^[Hh][Oo][Ss][Tt]\([Nn][Aa][Mm][Ee]\)\?['"$'\t '"']*\([^*?]*\)$/\2/p' 
${config[@]} )
 	    hosts=$( compgen -W "$host_aliases" -- $ocur )
 		COMPREPLY=( ${COMPREPLY[@]} $hosts )
 	    fi
@@ -4272,7 +4272,6 @@
 					half-configured|config-files" | \
 		grep "Package: $1" | cut -d\  -f2
 }
-}

 # Debian dpkg(8) completion
 #



-- 
Morita Sho <morita-pub-en-debian@inz.sakura.ne.jp>

Message #34 received at 474094-quiet@bugs.debian.org (full text, mbox, reply):

From: Morita Sho <morita-pub-en-debian@inz.sakura.ne.jp>

To: David Paleino <d.paleino@gmail.com>

Cc: 474094-quiet@bugs.debian.org, Bash-Completion Developers <bash-completion-devel@lists.alioth.debian.org>, Jake K <jakekaa@gmail.com>

Subject: Re: Directory name completion fails if name contains spaces and is quoted

Date: Mon, 12 May 2008 11:59:15 +0900

Hi,

Let me explain about the function _get_cword I suggested and my opinion for that.

Previously, _get_cword was same as ${COMP_WORDS[$COMP_CWORD]}.
But it does not work in the middle of a word.
To work the completion in the middle of a word, we need to know the current 
cursor position.
Bash provides $COMP_POINT, this variable hold the current cursor position for 
the current command line($COMP_LINE).
Unfortunately, Bash does not provide the current cursor position for the current 
word.

I think it is a reason why the current _get_cword was switched to $COMP_LINE 
from ${COMP_WORDS[$COMP_CWORD]}.
The current _get_cword works as follow:
 1) Get the substring from the beginning of the current command line to the 
current cursor position.
 2) For that substring, find the last COMP_WORDBREAKS character that is not 
quoted by '\',
    and returns substring after the COMP_WORDBREAKS character found to the end 
of the string.

The problem of this way is handling for " and ' quoting.
To handle such quoting correctly, we need to parse the command line correctly.
It means we need to implement a command line parsing code as same as Bash does.
It is very hard to do in shell script.
I think it is better to use $COMP_WORDS instead.
Since $COMP_WORDS is an array consisting of words that splitted by Bash's parser,
so we don't need to consider for quoting.

However, $COMP_POINT is the position for $COMP_LINE,
and Bash does not provide the current cursor position for $COMP_WORDS.
If we know the current cursor position for the current word,
it is easy to do like this:
  ${COMP_WORDS[$COMP_CWORD]:0:$cursor_position_for_current_word}
But how to converting "the current cursor position for the command line" to "the 
current cursor position for the current word" ?


My idea (and what my _get_cword does) is as follow:

1) Find the offset for ${COMP_WORDS[$COMP_CWORD]} in $COMP_LINE.
   To do this, construct a regexp pattern like this:

/$IFS*${COMP_WORD[0]}$IFS+${COMP_WORD[1]}$IFS+${COMP_WORD[2]}$IFS+....${COMP_WORD[$COMP_CWORD-1]}$IFS+/
2) and then perform pattern matching against $COMP_LINE using expr(1)
     expr "$COMP_LINE" : "$pattern"
   this returns the number of characters matched.
   It value is the offset for ${COMP_WORDS[$COMP_CWORD]} in $COMP_LINE.
3) Subtract that value from $COMP_POINT and I get the cursor position for the 
current word.
4) Finally, get the substring from the beginning of the current word to the 
cursor position for the current word.
     ${COMP_LINE:$offset_for_current_word:$cursor_position_for_current_word}
    or
     ${COMP_WORDS[$COMP_CWORD]:0:$cursor_position_for_current_word}

That's all for explanation.


By the way, I cleaned up the code I posted.
* Removed unnesessary printf.
* Removed _quote_meta function.
* Slightly changed the loop condition from "i < $COMP_CWORD" to "i <= 
$COMP_CWORD - 1".
  I think it is more clear since it constructs the pattern for ${COMP_WORD[0 .. 
$COMP_CWORD-1]}.
* Append + for each arguments in expr(1) to pass an arbitrary string safely.

_get_cword()
{
  local pattern
  local i

  pattern=$'[ \t\n]*'
  for (( i=0; i <= $COMP_CWORD - 1; i++ )); do
    pattern="${pattern}$( printf "%s" "${COMP_WORDS[$i]}" | sed 's/\\/\\\\/g; 
s/[.*[$^]/\\&/g; s/[?+{()|]/[&]/g' )"$'[ \t\n]\+'
  done

  local LC_CTYPE=C
  printf "%s" "${COMP_WORDS[$COMP_CWORD]:0:$(( $COMP_POINT - $( expr + 
"$COMP_LINE" : + "$pattern" ) ))}"
}



FYI,
"local LC_CTYPE=C" may not work as expected.
See #478096 for details.
#478096 - bash: local does not work for some Bash variables - Debian Bug report logs
http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=478096


Sorry for my poor English.
I tried to write a correct sentence, but my skill was not enough.
Ask me if you have any question.

Thanks,

-- 
Morita Sho <morita-pub-en-debian@inz.sakura.ne.jp>

Message #39 received at 474094-close@bugs.debian.org (full text, mbox, reply):

From: Luk Claes <luk@debian.org>

To: 474094-close@bugs.debian.org

Subject: Bug#474094: fixed in bash-completion 20080617

Date: Sat, 21 Jun 2008 21:02:04 +0000

Source: bash-completion
Source-Version: 20080617

We believe that the bug you reported is fixed in the latest version of
bash-completion, which is due to be installed in the Debian FTP archive:

bash-completion_20080617.dsc
  to pool/main/b/bash-completion/bash-completion_20080617.dsc
bash-completion_20080617.tar.gz
  to pool/main/b/bash-completion/bash-completion_20080617.tar.gz
bash-completion_20080617_all.deb
  to pool/main/b/bash-completion/bash-completion_20080617_all.deb



A summary of the changes between this version and the previous one is
attached.

Thank you for reporting the bug, which will now be closed.  If you
have further comments please address them to 474094@bugs.debian.org,
and the maintainer will reopen the bug report if appropriate.

Debian distribution maintenance software
pp.
Luk Claes <luk@debian.org> (supplier of updated bash-completion package)

(This message was generated automatically at their request; if you
believe that there is a problem with it please contact the archive
administrators by mailing ftpmaster@debian.org)


-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Format: 1.8
Date: Sat, 21 Jun 2008 21:59:43 +0200
Source: bash-completion
Binary: bash-completion
Architecture: source all
Version: 20080617
Distribution: unstable
Urgency: low
Maintainer: Bash Completion Maintainers <bash-completion-devel@lists.debian.org>
Changed-By: Luk Claes <luk@debian.org>
Description: 
 bash-completion - programmable completion for the bash shell
Closes: 347316 365658 376433 394636 404976 409423 416655 424736 428641 432289 441017 455510 468254 470539 470742 472132 472940 474094 474517 474974 475242 478596 480879
Changes: 
 bash-completion (20080617) unstable; urgency=low
 .
   [ David Paleino ]
   * New upstream release
     - add more completions to aptitude (Closes: #432289)
     - fixed UTF-8 problem with _get_cword(), thanks to
       Andrei Paskevich (Closes: #472132)
     - fixed autoremove completion, thanks to Flavio Visentin
       (Closes: #474974)
     - cmf and CMF added to playmidi completion (Closes: #365658)
     - added rrdtool completion, thanks to Justin Pryzby (Closes: #428641)
     - added OpenDocument completion for unzip/zipinfo (.od{f,g,p,s,t})
       (Closes: #472940)
     - fixed escaping problems with job control (i.e. disown, jobs, bg,
       fg): the argument is now surrounded by "" (Closes: #347316)
     - make mkdir complete also on filenames (Closes: #376433)
     - {bz,z}{cat,cmp,diff,egrep,fgrep,grep,less,more} now should complete
       on all filenames, not just compressed archives (just commented out)
       (Closes: #455510)
     - fixes Perl completion (Closes: #470742)
     - fixes get_cword -> _get_cword typo (Closes: #478596)
     - fixes _get_cword() function to properly handle filenames with
       whitespaces (Closes: #394636, #468254, #474094)
     - added .pdf.bz2 completion to evince (Closes: #424736)
     - added .svg completion to display (Closes: #441017)
     - added .m2ts completion to mplayer (Closes: #480879)
     - added extra/dh_bash-completion to ease future rewrite of bc.
   * debian/copyright - now in a fancier machine-parsable format.
   * debian/control:
     - added myself to Uploaders
     - debhelper Build-Depends updated to >= 6.
   * debian/watch:
     - improved current watch line regex
     - added (commented out) probable future watch line
   * debian/compat bumped to 6
   * debian/dirs, debian/install and debian/links added
   * debian/rules:
     - refactored to make use of debian/{dirs,install,links}
 .
   [ Steve Kemp ]
   * Applied patch to fix completion of umount command.
     (Closes: #470539)
   * Fixed the completion of Perl manpages.
     (Closes: #404976)
   * Added 'aif' to the filenames offed for completion for mplayer.
     (Closes: #474517)
   * Allow tsocks completion.
     (Closes: #409423)
   * Update mutt completion to handle local usernames.
     (Closes: #416655)
   * Update apt-get completion to include the flag "--no-install-recommends"
     (Closes: #475242)
Checksums-Sha1: 
 a267866b73de860449ddbc4374faa0b27cb14652 854 bash-completion_20080617.dsc
 d0fc2f36ee0d896f98c1e5dfd49e8623bb936aaa 97143 bash-completion_20080617.tar.gz
 91dc4ad723ecbff302047187e65f6aefedf5ebbf 83372 bash-completion_20080617_all.deb
Checksums-Sha256: 
 9aa0ed60aa9fc88fcc3a542a0d296ead12c23dbae025d158ff414267b22c960b 854 bash-completion_20080617.dsc
 57cd6ee1abc39182183eebeeb1378bdd2438363bf766d69dcabd0320378b6b22 97143 bash-completion_20080617.tar.gz
 e88bd724c2e44629c27fe0346369fe450f0cae12f846ae08fc3794b7dcb1ae32 83372 bash-completion_20080617_all.deb
Files: 
 058c2c5c7a2da8c58f3fbf4f117caf26 854 shells standard bash-completion_20080617.dsc
 902c045bb9fce960571f44200b476cec 97143 shells standard bash-completion_20080617.tar.gz
 7997031028650ef5f5ae30bf1ba5488f 83372 shells standard bash-completion_20080617_all.deb

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.9 (GNU/Linux)

iEYEARECAAYFAkhdZpQACgkQ5UTeB5t8Mo2n+ACcC7WR8B82ilcQ7iB69fQ/SLQ7
BAwAn3aAUrB08bRSg5QD3/wF9qdPGtC0
=CoOt
-----END PGP SIGNATURE-----

Send a report that this bug log contains spam.