Re: [PATCH] Keep expected behaviour for [a-z] and [A-z] (Bug 23393).

[Carlos O'Donell <carlos at redhat dot com>]

On 07/25/2018 10:16 PM, Jonathan Nieder wrote:
> Carlos O'Donell wrote:
>> On 07/25/2018 09:33 PM, Jonathan Nieder wrote:
> 
>>> The Debian system where it is most convenient for me to test has
>>> Debian's libc6 package, version 2.24-12.  [a-z] matches uppercase
>>> letters.  I've always considered that undesirable but I'm confused
>>> about the described regression.  Did one of Debian's patches to
>>> localedata cause it to pick up the regression early (by which I mean,
>>> more than 5 years ago)?
>>
>> It depends entirely on the locale you use. Some locales already have
>> [a-z] matching uppercase and have had it for years. The problem is that
>> this is new for en_US.UTF-8.
>>
>> Which locale did you use? en_US.UTF-8? If so, then yes, Debian must have
>> done something different with iso14651_t1_common to change this, or added
>> something else. I did a quick look at the debian patches for 2.24-12 and
>> didn't see anything that would change this materially for en_US.
> 
> I tried with the following locales:
> 
>  en_US:		matches (bad)
>  en_US.UTF-8:	matches (bad)
>  C:		does not match (good)
>  C.UTF-8:	does not match (good)
>  fr_CH:		matches (bad)
>  fr_CH.UTF-8:	matches (bad)
> 
> Looking over
> https://salsa.debian.org/glibc-team/glibc/tree/sid/debian/patches/localedata
> and https://salsa.debian.org/glibc-team/glibc/tree/sid/debian/patches/locale,
> I don't see any obvious culprits.  Anyway, please just take this as more
> feedback in favor of your approach.
> 
> See the user reports merged with https://bugs.debian.org/301717.

This is your shell doing the expanding, and worse doing it
differently from glibc.

My bash shell also handles [a-z] expansion differently given 
the locale data. It appears to be using collation sequence
i.e. the order in which the elements sort in. 

Using grep doesn't result in these matches.

The fix is this: `shopt -s globasciiranges`, and we should
make it the default from now on. The option turns on rational
ranges for bash. Florian found this out when digging into
the issue.

We have a lot of cleanup to do to get rational ranges on
at each step of expansion.

Cheers,
Carlos.