Re: glibc git commit hooks update.

[Carlos O'Donell <codonell at redhat dot com>]

On 5/16/19 4:57 PM, Florian Weimer wrote:
> * Joseph Myers:
> 
>> On Wed, 17 Apr 2019, Carlos O'Donell wrote:
>>
>>>> One problem with the old hooks was that they did not generate a
>>>> Content-Type header, which was unhelpful when the mails also weren't pure
>>>> ASCII.  I see the new ones are generating 'Content-Type: text/plain;
>>>> charset="us-ascii"'.  Will they also be smart about specifying an
>>>> appropriate character set (so UTF-8 if the commit message / author name /
>>>> diff contents are valid UTF-8 but not ASCII, for example)?
>>>
>>> All of this is handled by python's email package and some handling
>>> on the hooks part. If everything is ASCII then we don't do any special
>>> encoding and just send ASCII. Otherwise we choose UTF-8 first and if
>>> that fails a decode test then we fallback to ISO-8859-1.
>>
>> This does not seem to be working as intended.  See e.g. 
>> https://sourceware.org/ml/glibc-cvs/2019-q2/msg00147.html (UTF-8 bytes 
>> marked as us-ascii, so ’ appears as â??, for example).
> 
> I expect that this is unrelated to the change and was this way before.
> I haven't modified those parts.
> 
> Defaulting to charset=utf-8 is probably the best option here, even
> though the message text might not be UTF-8 always.  I think this
> requires modifications to the Python scripts.

Correct.

Today we have:

hooks/updates/emails.py:
201         e_msg = MIMEText(self.__email_body_with_diff)

We need something like this instead:

	full_text = self.__email_body_with_Diff
	encoding = None
	for potential_encoding in ('UTF-8', 'iso-8859-1')
		try:
			full_text.decode(potential_encoding)
			encoding = potential_encoding
			break
		except
			pass
	if encoding is None:
		encoding = 'us-ascii'
	e_msg = MIMEText(full_text, 'plain', encoding)
	... Rest of the handling ...

-- 
Cheers,
Carlos.