Re: [PATCH v10] Locales: Cyrillic -> ASCII transliteration table [BZ #2872]

[Rafal Luzynski <digitalfreak at lingonborough dot com>]

19.11.2018 12:10 Egor Kobylkin <egor@kobylkin.com> wrote:
> 
> Changelog v10:
> * Removed ISO 9.1995 GOST 7.79-2000 System A (transliteration to Latin
> with diacritics) as conflicting with System B within glibc mechanics and
> not solving BZ #2872

I'm in favor of implementing System A and dropping System B instead.
If I understand correctly, System A is actually ISO 9, therefore it is
international, universal, and neutral, while System B is a GOST standard
and therefore used only in Russia (also adopted in several other countries
as well).

It's true that we can't handle both System A and System B.  What we
would like to have is:


             System A
          /============> OUTPUT: Latin with diacritics
  INPUT <    System B
          \============> OUTPUT: Plain ASCII (fallback)

That means: use one system but if the output can't handle it then switch
to another system.

But what we can actually have is either:

          System A                                    Fallback
  INPUT ============> OUTPUT: Latin with diacritics ============> Plain
ASCII

or:

          System B
  INPUT ============> OUTPUT: Plain ASCII

That means, we can only provide a fallback for individual characters,
we can't provide a fallback algorithm (that is, we can't switch to
transliterating 'Х' as 'X' instead of 'H' just because we can't
transliterate
'Ш' as 'Š' and switch to 'SH' instead).

Wouldn't it be better to implement ISO 9 (System A) instead and provide
a fallback ASCII transliteration which could be similar but not identical
to System B?  Is it necessary to provide plain ASCII transliteration
conforming to System B even if that means that we would have not to
implement System A?  If yes, would it be correct to provide System B
for ru_RU (and maybe few more locales) but include System A in all other
locales (except few which we exclude already)?

> * Edited below email, commit message, comment in translit_cyrillic to
> reflect System A removal
> * Removed <U0423><U0301> and <U0443><U0301> (Cyrillic U with acute,
> using composition) as composing is not covered by current glibc
> conversion mechanics

OK, thank you, I like this change.

> [...]
> The transliteration of Cyrillic to ASCII according to GOST 7.79-2000
> System B represents what is actually called transcription (preserving
> phonemes), while System A is the transliteration (preserving graphemes).
> There is no meaningful way to preserve graphemes converting Cyrillic to
> ASCII and thus the System B is chosen. [11]

I'm not sure it should be actually called transcription.  IIUC,
transcription
reflects pronunciation, something we can't easily implement in glibc.
As long as we convert letters to letters (or group of letters to group
of letters) without taking pronunciation into account it should be
called transliteration.  OTOH, I agree that it is rather uncommon in
Russian language to find an example where pronunciation is not perfectly
reflected in spelling.

> +% Generated from UnicodeData.txt with a spreadsheet referenced
> +% in that bugs doclet

The previous versions of your patch had "in that bug's doclet" here
which I think is correct.

I like the version 9 of your patch more so I'm going to write a more
thorough review of it.

Regards,

Rafal