Hong Kong Supplementary Character Set

From HandWiki
Short description: Extended character encoding for Standard Cantonese
HKSCS Big-5 extension
MIME / IANABig5-HKSCS
Alias(es)big5hk, csBig5HKSCS
Language(s)Traditional Chinese, Cantonese
Classification8-bit CJK DBCS
ExtendsBig5 ETen

The Hong Kong Supplementary Character Set (香港增補字符集; commonly abbreviated to HKSCS) is a set of Chinese characters – 4,702 in total in the initial release—used in Cantonese, as well as when writing the names of some places in Hong Kong (whether in written Cantonese or standard written Chinese sentences).[1]

It evolved from the preceding Government Chinese Character Set (政府通用字庫) or GCCS. GCCS is a set of supplementary Chinese characters coded in the user-defined areas of the Big5 character set. It was originally used within the Hong Kong Government and later used by the public. It later evolved into Hong Kong Supplementary Character Set when the characters in the set were submitted to ISO-10646 for coding.

History

Due to the inherent differences between standard written Chinese and written Cantonese, the Government of Hong Kong recognised the need for a standardised set of proprietary characters that would allow for the streamlining of electronic communication; at the time, the Big5 Chinese encoding scheme did not contain a vast majority of these characters (some were erroneously cross-listed with similar characters).

The Government Chinese Character Set (政府通用字庫) or GCCS was thus developed by the government. The character set consists of Chinese characters commonly used in Hong Kong. Some characters are Cantonese-specific, while some are alternative forms of characters. The set is not well-organised and the characters are not closely examined.

Subsequently, the HKSCS-1999 (HKSCS 1999 specification) was developed. Following its acceptance, newer revisions were released in 2001 (adding 116 new characters) and in 2004 (adding 123 new characters), totalling 4,941 characters. 106 GCCS characters were removed in HKSCS-1999 as a result of unification, and their Big5 code points are reserved for compatibility.[2][3] Retired "not verifiable" GCCS characters are found in UTC Sources (UTC-00877–UTC-00898),[4] where they are sourced from Adobe-CNS1-1,[5] an Adobe-CNS1 supplement implemented to support GCCS.[6]

The HKSCS is encoded in Big5 (Big5-HKSCS,[7] big5hk[8]) and ISO 10646 (Unicode). Starting from HKSCS-2004, all characters previously using the Private Use Area section of Unicode are remapped, with many of them reassigned to Extension B Block or Supplementary Ideographic Plane Compatibility Block.[9] However, to preserve compatibility with programs that generated PUA code points, the allocated code points are reserved, and no new characters will be mapped to PUA.

Similarly to Hong Kong's situation, there are also characters that are needed by Macao but included in neither Big5 nor HKSCS, hence, the Macao Supplementary Character Set was developed, building on HKSCS with additional Unicode-mapped characters. The first batch of 121 MSCS characters were submitted for addition to or horizontal extension in Unicode (as appropriate) in 2009,[10] and the first final version of MSCS was established in 2020.[11]

Versions

The HKSCS has gone through a few iterations.[12]

Version Total characters Publish date
GCCS 3,049 1995
HKSCS-1999 4,702 09/1999
HKSCS-2001 4,818 12/2001
HKSCS-2004 4,941 05/2005
HKSCS-2008 5,009 12/2009
HKSCS-2016 5,033 05/2017

The last edition of HKSCS to encode all of its characters in Big5 was HKSCS-2008, while the characters added in HKSCS-2016 are mapped to Unicode only (as a CJK Unified Ideographs horizontal glyph extension where appropriate).[11]

Compatibility

Operating systems

In Microsoft Windows 98, NT 4.0, 2000, XP, HKSCS support can be enabled using Microsoft's patch. In Microsoft's implementation, application using code page 950 automatically uses a hidden code page 951 table for the Big5 encoding of the HKSCS extensions. The table supports all code points in HKSCS-2001, except for the compatibility code points specified by the standard.[13] In addition, the MingLiU font is altered using Microsoft's patch. This patch is known to create conflicts in applications such as Microsoft Office, or any application using fonts supporting simplified Chinese characters (e.g.: SimSun). If the target environment contains custom font mapped to the code points affected by Microsoft's patch, the custom fonts can undo Microsoft's patch. Furthermore, the patch breaks EUDC Editor supplied with the affected versions of Windows.[14] Starting with Windows Vista, HKSCS-2004 characters are only supported as Unicode 4.1 or later.[15] All characters are assigned standard, non-PUA codepoints. The characters are displayed with the MingLiU font, and these characters can be entered via the keyboard. The patch that provides Big5 encoding of HKSCS is unsupported in Windows Vista and later. A utility provided by Microsoft is available to convert HKSCS and Unicode PUA-encoded characters to Unicode 4.1 version.[16] In 2010, Microsoft published a HKSCS-2004 patch for Windows XP and Windows Server 2003.[17] It replaces Windows XP version of MingLiu, PMingLiu, and MingLiu_HKSCS (if HKSCS-2001 patch was applied) with Windows 7 version of MingLiu, PMingLiu and MingLiu_HKSCS. In addition, MingLiU-ExtB, MingLiU_HKSCS-ExtB and PMingLiU-ExtB fonts will be added onto target system. However, IME is not updated as it was in the case of HKSCS-2001 patch, and the fonts are from pre-release of Windows 7. For earlier versions of the OS, HKSCS support requires the use of Microsoft's patch, or the Hong Kong government's Digital 21's utilities.

IBM assigns CCSID 5471 to the HKSCS-2001 Big5 code page (with CPGID 1374 as CCSID 5470 as the double byte component),[18][19] CCSID 9567 to the HKSCS-2004 code page (with CPGID 1374 as CCSID 9566 as the double byte component),[20] and CCSID 13663 to the HKSCS-2008 code page (with CPGID 1374 as CCSID 13662 as the double byte component),[21] while CCSID 1375 (with CPGID 1374 as CCSID 1374 as its double byte component) is assigned to a growing HKSCS code page, currently equivalent to CCSID 13663.[22]

HKSCS support was added to glibc in 2000, but it has not been updated since then. HKSCS-2004 support is handled as Unicode 4.1 and later. For freedesktop.org setup, AR PL ShanHeiSun Uni font fully supports HKSCS-2004 since 0.1-0.dot.1, with latest revision of HKSCS-2004 supported in version 0.1.20060903-1. Modern desktop distributions (e.g. Ubuntu) include Arphic Technology's HKSCS-compliant UKai and UMing fonts out of the box when Traditional Chinese Language support is selected during installation. They can also be installed manually at a later time.

Mac OS X 10.0–10.2 supports HKSCS-1999. 10.3–10.4 supports HKSCS-2001. Some of the letters added to HKSCS-2004 is supported via Unicode PUA in OS X 10.4. Starting with OS X 10.5, all the HKSCS-2004 characters are supported via standard Unicode 4.1 code points.

Applications and the Web

Mozilla 1.5 and above supports HKSCS, with HKSCS-2004 support added into Gecko 1.8.1 code base.[23] Unlike the above-mentioned patch, Mozilla uses its own code page table. However, the fix for bug 343129 does not support characters mapped to code points above Basic Multilingual Plane.[24]

QT 3.x-based applications (e.g.: KDE) only support characters mapped to code points FFFF or lower. In QT4, characters outside BMP are supported via surrogates. Big5-HKSCS Text Codec supports HKSCS-1999 back in Qt-2.3.x, but it was too late in Qt development schedule to be officially included in the Qt-2.3.x series, so it was officially supported in Qt-3.0.1. HKSCS-2001 support was added in Qt-3.0.5.[25]

GNOME supports HKSCS characters in Unicode ranges, except those mapped to the Basic Multilingual Plane compatibility block. Patches to support characters mapped to above Basic Multilingual Plane was introduced during Pango 1.1.[26]

The WHATWG Encoding Standard (used by HTML5) includes HKSCS in its definition of Big5 (used even with the plain Big5 label). However, only its decoder uses all HKSCS extensions, while its encoder explicitly excludes those with lead bytes below 0xA1 (thus excluding most of the HKSCS extensions but including, for example, those inherited from Big5 ETEN).[27] Newer browsers follow this standard, including Firefox.

See also

References

  1. ↑ FAQs about GovHK Online Services – Other Technical Questions and Trouble Shooting
  2. ↑ "Big5CMP.txt". http://www.ogcio.gov.hk/en/business/tech_promotion/ccli/terms/doc/big5cmp.txt.  Found at Mapping table - HKSCS-2008
  3. ↑ "HKSCS-2004 Annex IV. Compatibility Points for GCCS". http://www.ogcio.gov.hk/en/business/tech_promotion/ccli/terms/doc/e_annex4_2004.pdf. 
  4. ↑ "Group:Big5-GCCS外字". http://glyphwiki.org/wiki/Group:Big5-GCCS外字. 
  5. ↑ "U-source glyphs". https://www.unicode.org/Public/UCD/latest/ucd/USourceGlyphs.pdf. 
  6. ↑ "The Adobe-CNS1-6 Character Collection". https://www.adobe.com/content/dam/Adobe/en/devnet/font/pdfs/5080.Adobe-CNS1-6.pdf. 
  7. ↑ "Character Sets". IANA. https://www.iana.org/assignments/character-sets/character-sets.xhtml. 
  8. ↑ "SDK components". http://infocenter.sybase.com/help/topic/com.sybase.infocenter.dc34789.1550/html/ocsinunx/CIHEBHFB.htm. 
  9. ↑ "Big5-HKSCS:2004". http://moztw.org/docs/big5/table/hkscs2004.txt. 
  10. ↑ Computer Chinese Characters Encoding Workgroup (2009-06-12). "Submission of Characters from Macao Information Systems Character Set". http://appsrv.cse.cuhk.edu.hk/~irg/irg/irg32/IRGN1580MacaoCharsFromMISCS.pdf. 
  11. ↑ 11.0 11.1 Macao Special Administrative Region Government (2020-06-11). "Submission of Macao's Vertical Extension (UNC Characters), Horizontal Extension, and IVSes Registration for MSCS". https://appsrv.cse.cuhk.edu.hk/~irg/irg/irg53/IRGN2430.pdf. 
  12. ↑ "OGCIO - Development of HKSCS". https://www.ogcio.gov.hk/en/business/tech_promotion/ccli/hkscs/development.htm. 
  13. ↑ Steele, Shawn. "CP 951 & HKSCS". MS Dev Blog. https://blogs.msdn.microsoft.com/shawnste/2007/03/12/cp-951-hkscs/. 
  14. ↑ 華通資訊網: 小心！有人悄悄換掉了你的Windows系統字型
  15. ↑ Microsoft: Hong Kong Supplementary Character Set – Support for Windows Platform
  16. ↑ Microsoft Character Code Conversion Routines For HKSCS-2004
  17. ↑ Windows XP Font Pack for ISO 10646:2003 + Amendment 1 Traditional Chinese Support
  18. ↑ "CCSID 5471: Mixed Big-5 ext for HKSCS-2001". IBM Globalization - Coded character set identifiers. IBM. http://www-01.ibm.com/software/globalization/ccsid/ccsid5471.html. 
  19. ↑ International Components for Unicode (ICU), ibm-5471_P100-2006.ucm, 2007-05-09, https://github.com/unicode-org/icu/blob/master/icu4c/source/data/mappings/ibm-5471_P100-2006.ucm 
  20. ↑ "CCSID 9567: Mixed Big-5 ext for HKSCS-2004". IBM Globalization - Coded character set identifiers. IBM. http://www-01.ibm.com/software/globalization/ccsid/ccsid9567.html. 
  21. ↑ "CCSID 13663: Mixed Big-5 ext for HKSCS-2008". IBM Globalization - Coded character set identifiers. IBM. http://www-01.ibm.com/software/globalization/ccsid/ccsid13663.html. 
  22. ↑ "CCSID 1375: Mixed Big-5 ext for HKSCS". IBM Globalization - Coded character set identifiers. IBM. http://www-01.ibm.com/software/globalization/ccsid/ccsid1375.html. 
  23. ↑ Mozilla.org: Bug 343129 – Big5-HKSCS 2004 <==> Unicode Table Update
  24. ↑ Bug 162431 – add non-BMP Unicode (plane 1 and above. surrogate) support to charset encoder/decoder
  25. ↑ "Qt 4.7: Big5-HKSCS Text Codec". http://doc.qt.nokia.com/latest/codec-big5hkscs.html. 
  26. ↑ Bug 101081 – Non-BMP (plane 1 thru plane 16) characters are not supported
  27. ↑ van Kesteren, Anne. "Encoding Standard". WHATWG. https://encoding.spec.whatwg.org/. 

External links



Retrieved from "https://handwiki.org/wiki/index.php?title=Hong_Kong_Supplementary_Character_Set&oldid=3361435"