Debian Bug report logs - #483860
posh: "local" can't be used as a function name

Toggle useless messages

Message #5 received at submit@bugs.debian.org (full text, mbox, reply):

From: Stephane Chazelas <stephane_chazelas@yahoo.fr>

To: Debian Bug Tracking System <submit@bugs.debian.org>

Subject: posh: "local" can't be used as a function name

Date: Sat, 31 May 2008 18:21:35 +0100

Package: posh
Version: 0.6.7
Severity: normal


$ local() { echo a; }
$ local a
$ "local" a
$

"local" is not a POSIX command, but I suspect it might be a
"debian policy" extension to POSIX which may explain why posh
has it.

Anyway, the above breaks POSIX conformance I think. POSIX only
allows "select" and "function" as possible non-standard
keywords.

If "local" were a keyword in posh, I would expect the first line
above to give an error and the 3rd line to output "a\n".

Note that the behavior of "local" differs in *every* shell that
has such a builtin.

Note that given that "local" is not POSIX and can be written as
a POSIX shell function with little effort if need be, it
shouldn't be needed as a builtin, IMO.

See for instance:

# this is like the "local" in pdksh, posh or ash, and unlike in
# bash or zsh (that initialise the variable to an empty string)
# or ksh93 or bash (that don't parse it as a builtin).
#
# variables starting with _l are reserved.
# functions to be declared as:
# declare funcname; f_funcname() { ... ; }

local() {
  for _lvar do
    if eval "[ -z \"\${_l${_l}${_lvar%%=*}++}\" ]"; then
      eval "_l$_l=\"\${_l$_l} \${_lvar%%=*}\""
      eval "_l$_l${_lvar%%=*}=\${${_lvar%%=*}++\$${_lvar%%=*}}"
    fi
    case $_lvar in
      (*=*) eval "${_lvar%%=*}=\${_lvar#*=}"
    esac
  done
}

call() {
  _l=$((${_l:-0} + 1))
  unset "_l$_l"
  _lfname=${1#f_}
  _lfstack=$_lfstack+$_lfname
  "$@"
  _lfstack=${_lfstack%+*}
  _lfname=${_lfstack##*+}
  local IFS=" "
  eval "_lvar=\${_l$_l}"
  for _lvar in $_lvar; do
    eval "$_lvar=\${_l$_l$_lvar}"
    if eval "[ -z \"\${$_lvar}\" ]"; then
      unset "$_lvar"
    else
      eval "$_lvar=\${$_lvar#+}"
    fi
    unset "_l$_l$_lvar"
  done
  unset "_l$_l"
  _l=$(($_l - 1))
}

declare() { eval "$1() { call f_$1 \"\$@\"; }"; }


# usage example:

declare f; f_f() {
  local i
  i=$(($i + 1))
  echo $_lfstack $i
  [ "$i" -ge 10 ] || f
  echo $_lfstack $i
}

f

# Or:

IFS=+
call eval '
  local IFS=:
  set -f
  printf "<%s>\n" $PATH
'
echo "$IFS"

# or
call . some-file
# to have variables local during the execution of the sourced
# file...


-- System Information:
Debian Release: lenny/sid
  APT prefers unstable
  APT policy: (500, 'unstable')
Architecture: i386 (i686)

Kernel: Linux 2.6.25-rc8 (PREEMPT)
Locale: LANG=en_GB.ISO-8859-15, LC_CTYPE=en_US.ISO-8859-15 (charmap=ISO-8859-15)
Shell: /bin/sh linked to /bin/bash

Versions of packages posh depends on:
ii  debconf [debconf-2.0]         1.5.22     Debian configuration management sy
ii  libc6                         2.7-11     GNU C Library: Shared libraries

posh recommends no packages.

-- debconf-show failed

Message #10 received at 483860@bugs.debian.org (full text, mbox, reply):

From: Clint Adams <schizo@debian.org>

To: Stephane Chazelas <stephane_chazelas@yahoo.fr>, 483860@bugs.debian.org

Subject: Re: Bug#483860: posh: "local" can't be used as a function name

Date: Sun, 1 Jun 2008 02:26:37 +0000

On Sat, May 31, 2008 at 06:21:35PM +0100, Stephane Chazelas wrote:
> $ local() { echo a; }
> $ local a
> $ "local" a
> $
[...]
> Anyway, the above breaks POSIX conformance I think. POSIX only
> allows "select" and "function" as possible non-standard
> keywords.

"local" was incorrectly marked as a special built-in, since
it was intended to behave similarly to export/readonly.

> If "local" were a keyword in posh, I would expect the first line
> above to give an error and the 3rd line to output "a\n".

It's not a keyword, it's a builtin; what is your basis for the
quoting behavior?

> Note that given that "local" is not POSIX and can be written as
> a POSIX shell function with little effort if need be, it
> shouldn't be needed as a builtin, IMO.

Hardcoding such a function definition into the shell seems less
robust than as a builtin.

Message #15 received at 483860-close@bugs.debian.org (full text, mbox, reply):

From: Clint Adams <schizo@debian.org>

To: 483860-close@bugs.debian.org

Subject: Bug#483860: fixed in posh 0.6.8

Date: Sun, 01 Jun 2008 02:47:02 +0000

Source: posh
Source-Version: 0.6.8

We believe that the bug you reported is fixed in the latest version of
posh, which is due to be installed in the Debian FTP archive:

posh_0.6.8.dsc
  to pool/main/p/posh/posh_0.6.8.dsc
posh_0.6.8.tar.gz
  to pool/main/p/posh/posh_0.6.8.tar.gz
posh_0.6.8_amd64.deb
  to pool/main/p/posh/posh_0.6.8_amd64.deb



A summary of the changes between this version and the previous one is
attached.

Thank you for reporting the bug, which will now be closed.  If you
have further comments please address them to 483860@bugs.debian.org,
and the maintainer will reopen the bug report if appropriate.

Debian distribution maintenance software
pp.
Clint Adams <schizo@debian.org> (supplier of updated posh package)

(This message was generated automatically at their request; if you
believe that there is a problem with it please contact the archive
administrators by mailing ftpmaster@debian.org)


-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Format: 1.8
Date: Sat, 31 May 2008 22:29:43 -0400
Source: posh
Binary: posh
Architecture: source amd64
Version: 0.6.8
Distribution: unstable
Urgency: medium
Maintainer: Clint Adams <schizo@debian.org>
Changed-By: Clint Adams <schizo@debian.org>
Description: 
 posh       - Policy-compliant Ordinary SHell
Closes: 483860
Changes: 
 posh (0.6.8) unstable; urgency=medium
 .
   * Treat "local" builtin as non-special.  closes: #483860.
   * Don't use ENV for regression-52 test.
Checksums-Sha1: 
 ab83a1e2ab536b51cedb5a964c0b25b8f4f186ec 824 posh_0.6.8.dsc
 6234ecdc69958b6e9715e46734710369c69a8f12 415751 posh_0.6.8.tar.gz
 bf629d715a5337345a4127aa021e00a82ab9ffae 87694 posh_0.6.8_amd64.deb
Checksums-Sha256: 
 431912fcced63e09c7a262dd69770daca363c6ab16bb63bf4fe2fdac68ecedaf 824 posh_0.6.8.dsc
 d5563e5f8669e8957efd3e2f1518f415eeb01e1d9001c2ff812e71570f06a787 415751 posh_0.6.8.tar.gz
 4f7b1d2c43dd55ba0419b1b78103b55a32cb16cc0c5b7a65d967f32eb4cb8b92 87694 posh_0.6.8_amd64.deb
Files: 
 103797a557c977c6eb93508485a00caf 824 shells optional posh_0.6.8.dsc
 c2403054ae48f4b8bc5ee0528bfc4836 415751 shells optional posh_0.6.8.tar.gz
 42f8303a9b54741c2ac7ebc66ef14885 87694 shells optional posh_0.6.8_amd64.deb

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.6 (GNU/Linux)
Comment: Debian!

iD8DBQFIQgtM5m0u66uWM3ARArkVAKCYw2g4vjninK4LSFzwHTOxKFiY/QCgtth1
l6Z4ThvJRWF0cJS5Rfurfjw=
=qVZX
-----END PGP SIGNATURE-----

Message #20 received at 483860@bugs.debian.org (full text, mbox, reply):

From: Stephane Chazelas <Stephane_Chazelas@yahoo.fr>

To: Clint Adams <schizo@debian.org>

Cc: 483860@bugs.debian.org

Subject: Re: Bug#483860: posh: "local" can't be used as a function name

Date: Sun, 1 Jun 2008 12:11:39 +0100

On Sun, Jun 01, 2008 at 02:26:37AM +0000, Clint Adams wrote:
> On Sat, May 31, 2008 at 06:21:35PM +0100, Stephane Chazelas wrote:
> > $ local() { echo a; }
> > $ local a
> > $ "local" a
> > $
> [...]
> > Anyway, the above breaks POSIX conformance I think. POSIX only
> > allows "select" and "function" as possible non-standard
> > keywords.
> 
> "local" was incorrectly marked as a special built-in, since
> it was intended to behave similarly to export/readonly.

Hi Clint,

then, posh has another problem with the other special builtins.
As far, as I can tell, POSIX doesn't say that special builtin
names can't be used as functions.

But then ksh93 or ash don't allow it either...

If posh doesn't allow it, it should report an error on the
function declaration like ash or ksh93 IMO.

> > If "local" were a keyword in posh, I would expect the first line
> > above to give an error and the 3rd line to output "a\n".
> 
> It's not a keyword, it's a builtin; what is your basis for the
> quoting behavior?

keywords are only recognised as keywords when not quoted as the
quote removal is performed after the recognition of keywords.

See also SUSv3:

   The following words may be recognized as reserved words on
   some implementations (when none of the characters are
   quoted), causing unspecified results:

           [[ ]] function select


> > Note that given that "local" is not POSIX and can be written as
> > a POSIX shell function with little effort if need be, it
> > shouldn't be needed as a builtin, IMO.
> 
> Hardcoding such a function definition into the shell seems less
> robust than as a builtin.

Sorry I wasn't clear.

It is not what I meant. That was a critic of debian policy
rather than posh's.

I meant that encouraging people to write non-portable scripts by
using the non-POSIX "local" was a bad idea given that there are
POSIX alternatives to it. That is, script-writer can implement
their own version of "local" (which will be more portable as the
behavior of the built-in equivalent vary from shell to shell and
can be used is other places than just functions) in the unlikely
event they really need local scope in a sh shell script and
still be portable. But the same goes for "echo -n". I don't see
the point, given that there is a standard alternative,
especially when you consider how unportable echo is.

-- 
Stéphane

Message #25 received at 483860@bugs.debian.org (full text, mbox, reply):

From: Clint Adams <schizo@debian.org>

To: Stephane Chazelas <Stephane_Chazelas@yahoo.fr>

Cc: 483860@bugs.debian.org

Subject: Re: Bug#483860: posh: "local" can't be used as a function name

Date: Sun, 1 Jun 2008 13:15:24 +0000

On Sun, Jun 01, 2008 at 12:11:39PM +0100, Stephane Chazelas wrote:
> then, posh has another problem with the other special builtins.
> As far, as I can tell, POSIX doesn't say that special builtin
> names can't be used as functions.

It says that the command search order is special builtins, then
functions, then regular builtins, then non-built-in commands.

> If posh doesn't allow it, it should report an error on the
> function declaration like ash or ksh93 IMO.

That seems reasonable to me right now.

> > It's not a keyword, it's a builtin; what is your basis for the
> > quoting behavior?
> 
> keywords are only recognised as keywords when not quoted as the
> quote removal is performed after the recognition of keywords.

Again, "local" is not a reserved word or keyword.

> I meant that encouraging people to write non-portable scripts by
> using the non-POSIX "local" was a bad idea given that there are
> POSIX alternatives to it. That is, script-writer can implement
> their own version of "local" (which will be more portable as the
> behavior of the built-in equivalent vary from shell to shell and
> can be used is other places than just functions) in the unlikely
> event they really need local scope in a sh shell script and
> still be portable. But the same goes for "echo -n". I don't see
> the point, given that there is a standard alternative,
> especially when you consider how unportable echo is.

The point is to accommodate existing scripts rather than having
people fix them (which seems to make them very angry and confused
for no apparent reason).

This particular issue would more productively be discussed in a
bug on the debian-policy package or on the
debian-policy@lists.debian.org mailing list.

Message #30 received at 483860@bugs.debian.org (full text, mbox, reply):

From: Stephane Chazelas <Stephane_Chazelas@yahoo.fr>

To: Clint Adams <schizo@debian.org>

Cc: 483860@bugs.debian.org

Subject: Re: Bug#483860: posh: "local" can't be used as a function name

Date: Mon, 2 Jun 2008 11:58:59 +0100

On Sun, Jun 01, 2008 at 01:15:24PM +0000, Clint Adams wrote:
> On Sun, Jun 01, 2008 at 12:11:39PM +0100, Stephane Chazelas wrote:
> > then, posh has another problem with the other special builtins.
> > As far, as I can tell, POSIX doesn't say that special builtin
> > names can't be used as functions.
> 
> It says that the command search order is special builtins, then
> functions, then regular builtins, then non-built-in commands.

Oh thanks. Sorry, I had missed that. What a strange requirement!
Looks like bash and zsh authors found it strange as well as they
don't implement it when not called as sh (for zsh, even when
called as sh).

Another strange requirement that I see no shell implements even
posh, is that if a builtin (such as "[" or "echo" or ":") is not
found in $PATH, its invocation should fail!?!

$ posh -c 'PATH=/; echo "foo"'
foo

should have given something like:
posh: echo: not found
instead if I read the standard correctly.

[...]
> > > It's not a keyword, it's a builtin; what is your basis for the
> > > quoting behavior?
> > 
> > keywords are only recognised as keywords when not quoted as the
> > quote removal is performed after the recognition of keywords.
> 
> Again, "local" is not a reserved word or keyword.

I understood that, I was only answering your question: "what is
your basis for the question behavior?".

[...]
> This particular issue would more productively be discussed in a
> bug on the debian-policy package or on the
> debian-policy@lists.debian.org mailing list.

agreed, that's just me needing my daily ranting ;).

Cheers,
Stéphane

Message #35 received at 483860@bugs.debian.org (full text, mbox, reply):

From: Clint Adams <schizo@debian.org>

To: Stephane Chazelas <Stephane_Chazelas@yahoo.fr>, 483860@bugs.debian.org

Subject: Re: Bug#483860: posh: "local" can't be used as a function name

Date: Mon, 2 Jun 2008 15:04:55 +0000

On Mon, Jun 02, 2008 at 11:58:59AM +0100, Stephane Chazelas wrote:
> Another strange requirement that I see no shell implements even
> posh, is that if a builtin (such as "[" or "echo" or ":") is not
> found in $PATH, its invocation should fail!?!

Could you point me to that in the standard?

Message #40 received at 483860@bugs.debian.org (full text, mbox, reply):

From: Stephane Chazelas <Stephane_Chazelas@yahoo.fr>

To: Clint Adams <schizo@debian.org>

Cc: 483860@bugs.debian.org

Subject: Re: Bug#483860: posh: "local" can't be used as a function name

Date: Mon, 2 Jun 2008 17:12:45 +0100

On Mon, Jun 02, 2008 at 03:04:55PM +0000, Clint Adams wrote:
> On Mon, Jun 02, 2008 at 11:58:59AM +0100, Stephane Chazelas wrote:
> > Another strange requirement that I see no shell implements even
> > posh, is that if a builtin (such as "[" or "echo" or ":") is not
> > found in $PATH, its invocation should fail!?!
> 
> Could you point me to that in the standard?

This is so weird that I'm suspecting that either I misinterpret
it or I missed something.

Here is the text, please tell me if you understand it
differently:

http://www.opengroup.org/onlinepubs/009695399/utilities/xcu_chap02.html#tag_02_09_01_01

SUS>   Command Search and Execution
SUS>
SUS>If a simple command results in a command name and an optional list of
SUS>arguments, the following actions shall be performed:
SUS>
SUS> 1. If the command name does not contain any slashes, the first successful
SUS>    step in the following sequence shall occur:
SUS>
SUS>      a. If the command name matches the name of a special built-in
SUS>         utility, that special built-in utility shall be invoked.
SUS>
SUS>      b. If the command name matches the name of a function known to this
SUS>         shell, the function shall be invoked as described in Function
SUS>         Definition Command. If the implementation has provided a standard
SUS>         utility in the form of a function, it shall not be recognized at
SUS>         this point. It shall be invoked in conjunction with the path
SUS>         search in step 1d.
SUS>
SUS>      c. If the command name matches the name of a utility listed in the
SUS>         following table, that utility shall be invoked.
SUS>
SUS>                   alias   false   jobs   read    wait
SUS>                   bg      fc      kill   true
SUS>                   cd      fg      newgrp umask
SUS>                   command getopts pwd    unalias
SUS>
SUS>
SUS>      d. Otherwise, the command shall be searched for using the PATH
SUS>         environment variable as described in the Base Definitions volume
SUS>         of IEEE Std 1003.1-2001, Chapter 8, Environment Variables:
SUS>
SUS>          i. If the search is successful:
SUS>
SUS>               a. If the system has implemented the utility as a regular
SUS>                  built-in or as a shell function, it shall be invoked at
SUS>                  this point in the path search.
SUS>
SUS>               b. Otherwise, the shell executes the utility in a separate
SUS>                  utility environment (see Shell Execution Environment)
SUS>                  with actions equivalent to calling the execve() function
SUS>                  as defined in the System Interfaces volume of
SUS>                  IEEE Std 1003.1-2001 with the path argument set to the
SUS>                  pathname resulting from the search, arg0 set to the
SUS>                  command name, and the remaining arguments set to the
SUS>                  operands, if any.
SUS>
SUS>                  If the execve() function fails due to an error
SUS>                  equivalent to the [ENOEXEC] error defined in the System
SUS>                  Interfaces volume of IEEE Std 1003.1-2001, the shell
SUS>                  shall execute a command equivalent to having a shell
SUS>                  invoked with the pathname resulting from the search as
SUS>                  its first operand, with any remaining arguments passed
SUS>                  to the new shell, except that the value of "$0" in the
SUS>                  new shell may be set to the command name. If the
SUS>                  executable file is not a text file, the shell may bypass
SUS>                  this command execution. In this case, it shall write an
SUS>                  error message, and shall return an exit status of 126.
SUS>
SUS>             Once a utility has been searched for and found (either as a
SUS>             result of this specific search or as part of an unspecified
SUS>             shell start-up activity), an implementation may remember its
SUS>             location and need not search for the utility again unless the
SUS>             PATH variable has been the subject of an assignment. If the
SUS>             remembered location fails for a subsequent invocation, the
SUS>             shell shall repeat the search to find the new location for
SUS>             the utility, if any.
SUS>
SUS>          ii. If the search is unsuccessful, the command shall fail with
SUS>              an exit status of 127 and the shell shall write an error
SUS>              message.

Given that I've never seen a /bin/: (as : is built in every
shell), that would imply that ":" would never work.

regards,
Stéphane

Message #45 received at 483860@bugs.debian.org (full text, mbox, reply):

From: Stephane Chazelas <Stephane_Chazelas@yahoo.fr>

To: Clint Adams <schizo@debian.org>

Cc: 483860@bugs.debian.org

Subject: Re: Bug#483860: posh: "local" can't be used as a function name

Date: Tue, 10 Jun 2008 10:39:32 +0100

On Mon, Jun 02, 2008 at 05:12:45PM +0100, Stephane Chazelas wrote:
> On Mon, Jun 02, 2008 at 03:04:55PM +0000, Clint Adams wrote:
> > On Mon, Jun 02, 2008 at 11:58:59AM +0100, Stephane Chazelas wrote:
> > > Another strange requirement that I see no shell implements even
> > > posh, is that if a builtin (such as "[" or "echo" or ":") is not
> > > found in $PATH, its invocation should fail!?!
> > 
> > Could you point me to that in the standard?
> 
> This is so weird that I'm suspecting that either I misinterpret
> it or I missed something.
> 
> Here is the text, please tell me if you understand it
> differently:

Hi Clint,

> Given that I've never seen a /bin/: (as : is built in every
> shell), that would imply that ":" would never work.

":" (colon) is a special built-in, so it was a bad example, but
it is still true for "[", "echo", "printf"...

Do you read me the same way as I do?

> http://www.opengroup.org/onlinepubs/009695399/utilities/xcu_chap02.html#tag_02_09_01_01
> 
> SUS>   Command Search and Execution
> SUS>
> SUS>If a simple command results in a command name and an optional list of
> SUS>arguments, the following actions shall be performed:
> SUS>
> SUS> 1. If the command name does not contain any slashes, the first successful
> SUS>    step in the following sequence shall occur:
> SUS>
> SUS>      a. If the command name matches the name of a special built-in
> SUS>         utility, that special built-in utility shall be invoked.
> SUS>
> SUS>      b. If the command name matches the name of a function known to this
> SUS>         shell, the function shall be invoked as described in Function
> SUS>         Definition Command. If the implementation has provided a standard
> SUS>         utility in the form of a function, it shall not be recognized at
> SUS>         this point. It shall be invoked in conjunction with the path
> SUS>         search in step 1d.
> SUS>
> SUS>      c. If the command name matches the name of a utility listed in the
> SUS>         following table, that utility shall be invoked.
> SUS>
> SUS>                   alias   false   jobs   read    wait
> SUS>                   bg      fc      kill   true
> SUS>                   cd      fg      newgrp umask
> SUS>                   command getopts pwd    unalias
> SUS>
> SUS>
> SUS>      d. Otherwise, the command shall be searched for using the PATH
> SUS>         environment variable as described in the Base Definitions volume
> SUS>         of IEEE Std 1003.1-2001, Chapter 8, Environment Variables:
> SUS>
> SUS>          i. If the search is successful:
> SUS>
> SUS>               a. If the system has implemented the utility as a regular
> SUS>                  built-in or as a shell function, it shall be invoked at
> SUS>                  this point in the path search.
> SUS>
> SUS>               b. Otherwise, the shell executes the utility in a separate
> SUS>                  utility environment (see Shell Execution Environment)
> SUS>                  with actions equivalent to calling the execve() function
> SUS>                  as defined in the System Interfaces volume of
> SUS>                  IEEE Std 1003.1-2001 with the path argument set to the
> SUS>                  pathname resulting from the search, arg0 set to the
> SUS>                  command name, and the remaining arguments set to the
> SUS>                  operands, if any.
> SUS>
> SUS>                  If the execve() function fails due to an error
> SUS>                  equivalent to the [ENOEXEC] error defined in the System
> SUS>                  Interfaces volume of IEEE Std 1003.1-2001, the shell
> SUS>                  shall execute a command equivalent to having a shell
> SUS>                  invoked with the pathname resulting from the search as
> SUS>                  its first operand, with any remaining arguments passed
> SUS>                  to the new shell, except that the value of "$0" in the
> SUS>                  new shell may be set to the command name. If the
> SUS>                  executable file is not a text file, the shell may bypass
> SUS>                  this command execution. In this case, it shall write an
> SUS>                  error message, and shall return an exit status of 126.
> SUS>
> SUS>             Once a utility has been searched for and found (either as a
> SUS>             result of this specific search or as part of an unspecified
> SUS>             shell start-up activity), an implementation may remember its
> SUS>             location and need not search for the utility again unless the
> SUS>             PATH variable has been the subject of an assignment. If the
> SUS>             remembered location fails for a subsequent invocation, the
> SUS>             shell shall repeat the search to find the new location for
> SUS>             the utility, if any.
> SUS>
> SUS>          ii. If the search is unsuccessful, the command shall fail with
> SUS>              an exit status of 127 and the shell shall write an error
> SUS>              message.
> 
> Given that I've never seen a /bin/: (as : is built in every
> shell), that would imply that ":" would never work.

-- 
Stéphane

Send a report that this bug log contains spam.