Re: [Locale] IBM437 encoding not supported in wconv

From: Arnold, Stefan (Stefan.Arnold_at_[hidden])
Date: 2022-06-14 07:36:24

Let me restate my question in another way:

What would be necessary in order to support IBM-437 (and maybe further Win32 codepages) in boost::locale with Win32 API backend? Would it be as easy as adding entries in this mapping: https://github.com/boostorg/locale/blob/5789ae2532abbc3d9f5908b7c71ab079455d75a5/src/encoding/wconv_codepage.ipp#L42, plus adding some tests? How could I contribute to this undertaking?

Or are there any restrictions making such an addition difficult? For example, ICU seems to support IBM-437 as well (https://icu4c-demos.unicode.org/icu-bin/convexp?conv=ibm-437_P100-1995&s=ALL), so IBM-437 would not be restricted to the Win32 API backend.

Note that Microsoft currently lists 152 code page identifiers (https://docs.microsoft.com/en-us/windows/win32/intl/code-page-identifiers), while boost::locale exposes only 48 of them in the mapping mentioned above.

-----Original Message-----
From: Arnold, Stefan
Sent: Wednesday, June 8, 2022 8:07 PM
To: boost-users_at_[hidden]
Subject: [Locale] IBM437 encoding not supported in wconv

Why is the IBM437 encoding not supported in boost::locale with the Win32 API ("wconv") backend?

The following call will fail with a boost::locale::conv::invalid_charset_error when using the Win32 API backend:

boost::locale::conv::to_utf<char>("abc", "IBM437")

This is because "ibm437" is not contained in the all_windows_encodings variable in wconv_codepage.ipp (nor is "cp437" or another alias).

As per Win32 API documentation (https://docs.microsoft.com/en-us/windows/win32/api/stringapiset/nf-stringapiset-multibytetowidechar, https://docs.microsoft.com/en-us/windows/win32/intl/code-page-identifiers), this encoding is supported in Win32 API. Furthermore, IBM437 is the legacy encoding of filenames in ZIP files, so it should be quite common.