Iyini i-Unicode: Umhlahlandlela ophelele, ukusetshenziswa, kanye nokubhala ngekhodi

Informatec Digital » Izihloko » Iyini i-Unicode: Umhlahlandlela ophelele, ukusetshenziswa, kanye nokubhala ngekhodi

I-Unicode ulimi oluvamile olukhulunywa amakhompyutha lapho ephatha umbhalo : inika inombolo ehlukile kumlingiswa ngamunye kanye nophawu, kusukela cishe kunoma yiluphi uhlelo lokubhala, ukuze igcinwe, icutshungulwe, futhi yabelwane ngayo kalula phakathi kwamapulatifomu namazwe.

Le nqubo yavela ukuze inqobe ukulinganiselwa kwamasethi amadala ezinhlamvu ( isethi ye-ASCII , amakhasi ekhodi, i-EBCDIC, njll.), ayehluleka noma ayengahambisani, futhi namuhla ivumelaniswe ne-ISO/IEC 10646, igcina ama-algorithms (njenge-bidirectional eyodwa) futhi ichaza izakhiwo kanye nemithetho yokulungisa ukuze konke kusebenze ngokuqhubekayo.

Iyini i-Unicode futhi kungani ibaluleke kangaka?

I-Unicode iyindinganiso yokufaka ikhodi yezinhlamvu eguquguqukayo futhi ehlala ishintsha echaza uhlamvu ngalunye ngegama, iphuzu lekhodi, kanye nesethi yezakhiwo (iskripthi, isigaba, ukuqondiswa, icala, njll.). IKomidi Lobuchwepheshe be-Unicode (UTC), ngaphakathi kwe-Unicode Consortium, liyigcina ihambisana nendinganiso ye-ISO/IEC 10646.

Umgomo wayo ukuhlukahluka, ukufana, kanye nokuhlukile : i-repertoire ebanzi evumela ukushintshaniswa okucacile kombhalo wezilimi eziningi, ngemithetho ecacile nephindaphindwayo. Ngenxa yalokhu, ubuchwepheshe besimanje (izinhlelo zokusebenza, iziphequluli, i-XML, i-Java, izizindalwazi) bungaxuba izikripthi zesiLatini nesi-Arabhu, ama-ideogram e-CJK, ama-emoji, kanye nezimpawu zobuchwepheshe noma zomculo ngaphakathi kwedokhumenti efanayo.

Izinhlamvu, ama-glyphs namaphoyinti ekhodi

Ku-Unicode, uhlamvu luyiyunithi yolwazi engabonakali, kanti iphuzu lekhodi liyisihlonzi salo sezinombolo . I- glyph iyisimo esibonakalayo (lokho okubonayo esikrinini), okuncike kufonti . Uhlamvu olulodwa lungaba nama-glyph amaningi, futhi ngezinye izikhathi i-glyph imelela uhlamvu olungaphezu kolulodwa.

Umbhalo ojwayelekile wephuzu lekhodi ngu- U+XXXX ku-hexadecimal . Izibonelo ezibonisa izinto ezihlaziyiwe: uhlamvu oluncane u-'l' ngu-U+006C, uhlamvu oluncane olubhalwe ngaphambilini u-'ü' ngu-U+00FC, u-'é' ngu-U+00E9 kanye ne-beta yesiGreki: uhlamvu olukhulu u-U+0392 kanye nohlamvu oluncane u-U+03B2 (kweminye imibhalo u-'β' ucashunwe no-U+0392, kodwa leyo khodi ihambisana nohlamvu olukhulu u-'Β').

Isikhala sekhodi ye-Unicode sinezikhundla ezingaba ngu-1.114.112 (kufika ku-U+10FFFF) , ezihlelwe ukumboza nokuhlukanisa zonke izinhlelo zokubhala nezimpawu, kanye namashumi ezinkulungwane zemisebenzi ephumelelayo nekhulayo kunguqulo ngayinye.

Izinhlelo, izindawo namabhlogo

I-Unicode ihlukanisa isikhala sayo sezinhlamvu sibe yizindawo eziyi-17 ezinamakhodi afinyelela ku-65.536 ngalinye . Le nhlangano yenza kube lula ukuqoqa izikripthi nezimpawu ezihlobene bese uthola ngokushesha lokho okufunayo.

Izinhlobo ezifanele kakhulu : i-Basic Multilingual Plane (BMP, plane 0) ihlanganisa cishe zonke izinhlamvu zanamuhla nezimpawu eziningi; i-Supplementary Multilingual Plane (SMP, plane 1) igcina imibhalo yomlando nezimpawu zobuchwepheshe (isb., zomculo nezezibalo); i-Supplementary Ideographic Plane (SIP, plane 2) yandisa ama-ideogram e-CJK; i-plane 14 (SSP) ifaka amalebula akhethekile; kanti i-plane 15 ne-16 zenzelwe ukusetshenziswa kwangasese.

Amabhulokhi nezindawo : amamephu ahlukaniswe ngokungakahleleki abe yizindawo futhi ngokusemthethweni abe amabhulokhi ahambisanayo. Amabhulokhi asetshenziselwa ukubala nokubhala izinhlamvu, yize zingahambisani njalo namaqoqo ezilimi anencazelo.

CJK ideograms kanye nephrojekthi ye-Unihan

Ama-ideogram ase-East Asian (Han) ahlanganiswe ku-Unicode ngokuhlukahluka kwesitayela ngokwesifunda , kodwa wonke abhekisela kumlingiswa ofanayo ongaqondakali. Ukuphathwa kwawo kuphathwa yi-Ideographic Rapporteur Group (IRG), iqembu le-ISO/IEC JTC1/SC2/WG2 elinokumelwa okuvela eShayina, eJapan, eKorea, eVietnam, eHong Kong, eMacau, eSingapore, e-USA, nakwamanye.

Isizindalwazi se-Unihan sihlanganisa ulwazi olusizayo (ukufundwa, izincazelo, ukulingana) olubalulekile ekusingatheni la ma-ideogram ngezilimi ezahlukene kanye nezindinganiso zomlando noma zezinkampani.

Amabhulokhi kanye nezandiso ze-CJK eziyinhloko :

• I-CJK Unified Ideographs (BMP, U+4E00–U+9FFF): 20.992 izinhlamvu ezivame ukusetshenziswa.
• Isandiso A (BMP, U+3400–U+4DBF): 6.592 ama-ideogram angavamile.
• Izandiso ze-B–H: ku-SMP/SIP/TIP, bengeza amashumi ezinkulungwane ngaphezulu (B: U+20000–U+2A6DF, 42.720; C: U+2A700–U+2B73F, 4.154; D: U+2B740–U+2B81F, U+U+U222; 2; F: U+820CEB2–U+5.762EBEF, 2 G: U+0–U+2F, 7.473;
• Amanye amabhulokhi e-CJK ahlobene: Ama-Kangxi radicals (U+2F00–U+2FDF), izimpawu ze-CJK nezimpawu zokuloba (U+3000–U+303F), amafomethi ahambisanayo nokuhambisana, Isithasiselo se-ideogram yokuvumelana, njll.

I-Unicode ilindele ukuthi ukufakwa kwama-ideogram ngeke kube nomkhawulo ophelele futhi icabanga ngezindlela ezifana nokulandelana kwencazelo yama-ideographic ukumela izimpawu ezingafakwanga ikhodi ngokuzihlukanisa zibe izingxenye ezikhona (ngezixwayiso: ngaphandle kokuqhekeka kwe-canonical noma iziqinisekiso ekusebenzeni okufana nokusesha noma uku-oda).

Amafomu okufaka ikhodi: UTF-8, UTF-16 kanye ne-UTF-32

I-Unicode ichaza amafomu okuguqulwa (i-UTF) aguqula amaphuzu ekhodi abe amayunithi okugcina ukuze isofthiwe ikwazi ukucubungula umbhalo ngempumelelo ngokuya ngomongo wawo.

I-UTF-8 iyifomethi yokufaka ikhodi enobude obuguquguqukayo, eqondiswe ku-byte ehambisana ne-ASCII kububanzi be-U+0000–U+007F ku-byte eyodwa. Indinganiso yamanje isebenzisa ama-byte angu-1 kuya kwangu-4 ngohlamvu ngalunye; eminye imibhalo emidala ikhuluma ngo-1–6, kodwa ku-Unicode yanamuhla ingu-1–4. Yifomethi evelele kuwebhu.

I-UTF-16 isebenzisa amayunithi angu-16-bit

(iyunithi yekhodi eyodwa noma ezimbili ngohlamvu ngalunye) : izinhlamvu eziningi ze-BMP zingena kuyunithi eyodwa; izinhlamvu ezengeziwe zisebenzisa amabhangqa angaphansi kwe-U+D800–U+DFFF.

I-UTF-32 ubude obungaguquki: amabhayithi ama-4 ngohlamvu ngalunye, kulula kodwa kudinga isikhala esiningi; kuyasiza lapho ukukhomba izinhlamvu ngqo kubalulekile futhi inkumbulo ingeyona inkinga.

Indlela amabhithi asakazwa ngayo ku-UTF-8

Ifomethi ye-UTF-8 isabalalisa ama-code point bits ngokulandelana kwama-byte angu-1 kuya kwangu-4 anama-header abonakalayo, okugwema ukungaqondakali futhi kusiza ekutholakaleni kwemingcele yezinhlamvu.

Ububanzi be-Unicode	Iphethini elincane	Amabhayithi
U+0000..U+007F	0xxxxxx	1
U+0080..U+07FF	110yyyyy 10xxxxxx	2
U+0800..U+FFFF	1110zzzz 10yyyyyy 10xxxxxx	3
U+010000..U+10FFFF	11110uuu 10uuuzzzz 10yyyyyy 10xxxxxx	4

Inzuzo ebalulekile ye-UTF-8 ukuthi igwema izinkinga zokuhleleka kwe-byte (endianness) futhi ivumela ukucutshungulwa okuphumelelayo kakhulu kwemisakazo yombhalo, ngaphezu kokuhambisana ne-backward ne-ASCII.

Izikimu zokufaka ikhodi, ukuphela, kanye ne-BOM

Ngaphezu kwamafomu e-UTF, i-Unicode ichaza izinhlelo zokulandelela ezixazulula ukuthi ama-byte adluliselwa kanjani phakathi kwezinhlelo ezine-endianness ehlukene nokuthi izici ezifana ne-byte order zisayinwa kanjani.

• UTF-8: ukuphela akusebenzi. Ingathwalwa I-Byte Order Mark (BOM) njengeseluleko, nakuba kungadingeki noma kunganconywa ngokuzenzakalelayo.
• UTF-16: I-BE/LE (enkulu/encane-endian) okuhlukile kanye ne-BOM yokuzithandela (uma ingekho futhi ingachazwa umthetho olandelwayo, i-big-endian iyacatshangwa).
• UTF-32: Izinhlobo ze-BE/LE ezinemithetho efanayo; I-BOM ivunyelwe njengophawu lwe-oda.

Kukhona futhi izinhlobo ezithile ezifana ne-UTF-16BE/UTF-16LE kanye ne-UTF-32BE/UTF-32LE (ngaphandle kwe-BOM ngokwesivumelwano), kanye nezinye izikhodi zokuhambisana zomlando ezifana ne-UTF-7 noma i-UTF-EBCDIC, ngaphezu kwe-GB18030 (isiShayina esilingana ne-UTF-8 ngokusekelwa kwesiShayina esilula nesendabuko).

Ukujwayela, ukwakheka nokulingana

Izinhlamvu eziningi zingamelwa njengezibhalwe kusengaphambili noma njengezilandelaniso zesisekelo + amamaki okuhlanganisa . Izibonelo zakudala ezivela ezintweni ezibuyekeziwe: i-'Ä' ebhalwe kusengaphambili ingu-U+00C4, kuyilapho ifomu elibolile lingu-'A' (U+0041) + diaeresis (U+0308). I-'ỗ' yaseVietnam ingamelwa njengo-'o' (U+006F) + circumflex (U+0302) + tilde (U+0303).

I-Unicode ichaza amafomu ajwayelekile kanye nezinhlobo ezimbili zokulingana : okungokomthetho (okuqukethwe okufanayo okubalulekile) kanye nokuhambisana (amafomu angabonakala afana kodwa anomehluko we-semantic). Ukumiswa kwesimo kuqinisekisa ukuqhathaniswa kwezintambo okuthembekile futhi kunciphisa ukuphindaphindwa.

I-algorithm ye-Bidirectional nezinhlamvu ezikhethekile

Kuma-scripts asuka kwesokudla kuya kwesobunxele, njengesi-Arabhu noma isiHebheru, i-Unicode ifaka i -algorithm ejwayelekile neshintshashintshayo ye-bidirectional (Bidi) (isb., ukubuyekezwa ku-6.3), ukuze imibhalo exubile nesiLatini nesi-Arabhu imelwe ngokulandelana okulungile kokubona.

Amakilasi amakhodi okufanele aziwe ngokwezinto ezitholiwe: izinhlamvu zesithombe (izinhlamvu, izimpawu, izimpawu), ukufometha (izinhlamvu ezingabonakali ezithinta ukucutshungulwa: ukuhlukana komugqa we-U+2028, ukuhlukana kwesigaba se-U+2029, isikhala esiqinile se-U+00A0), amakhodi okulawula azuzwe njengefa ukuze avumelane (amabanga U+0000–U+001F, U+007F, U+0080–U+009F), ukusetshenziswa kwangasese, izikhundla ezigciniwe, izithasiselo (U+D800–U+DFFF ze-UTF-16) kanye nezingezona izinhlamvu (U+FFFE, U+FFFF endizeni ngayinye).

I-Unicode, ISO/IEC 10646 namanye amazinga (ASCII, ANSI, amakhasi ekhodi)

I-Unicode ivumelaniswe ne-ISO/IEC 10646 (UCS) futhi igcina amamephu ezindinganiso zangaphambilini (ASCII, ISO 8859-1, ANSI Z39.64, JIS X 0208, KS X 1001, GB 2312, GB 18030, HKSCS, CNS 11643, njll.), ngaphezu kokubhuka izikhala zokusetshenziswa kwangasese ngabakhiqizi.

I-ASCII vs. I-Unicode : I-ASCII iyisethi yezinhlamvu ezingama-7-bit enezinhlamvu eziyi-128 , eyanele isiNgisi esiyisisekelo, kodwa ayanele izilimi ezinezimpawu ezihlukanisayo, ama-ideogram, noma ama-emoji . I-Unicode ihlanganisa izinhlamvu ezingaphezu kuka-140.000 futhi iyaqhubeka nokukhula, ngekhodi engu-8/16/32-bit ngokwefomethi ye-UTF-12.

Amakhasi e-ANSI kanye nekhodi : 'I-ANSI' ivame ukubhekisela kumakhasi ekhodi e-Windows angu-8-bit alinganiselwe futhi angahambisani. Ikhompyutha esebenzisa i-OEM-Latin II evula umbhalo ku-IBM EBCDIC-Cyrillic izobona izinhlamvu ezingalungile. I-Unicode inqoba le nkinga ngesethi eyodwa yezinhlamvu kanye nokuguqulwa okungenakulahlekelwa phakathi kwamafomu ayo.

Ukuqaliswa ezinhlelweni (Windows, Solaris) kanye nokuguqulwa

I-Windows isebenzisa i-UTF-16 ngaphakathi kuma-API ayo esimanje futhi inikeza izici ezifana ne -MultiByteToWideChar kanye ne -WideCharToMultiByte zokuguqula phakathi kwamakhasi e-Unicode kanye nekhodi (SBCS/DBCS/MBCS). Uma uphoqelekile ukuguqula ube ikhasi lekhodi, kungase kube nokulahleka kwedatha uma ingakwazi ukumela zonke izinhlamvu.

Izinhlelo zokusebenza ezintsha ku-Windows kufanele zisebenzise i-UTF-16 ngaphakathi futhi zishiye ukuguqulwa kufinyelele emaphethelweni (I/O, amaphrothokholi), zinciphisa ukonakala. Noma kunjalo, i-Windows igcina ukuhambisana namakhasi ekhodi lapho kungagwemeki khona.

I-Oracle Solaris 11, ngokusho kwedokhumenti ebuyekeziwe, isekela i-Unicode 6.0 kanye ne-ISO/IEC 10646:2011 ezingeni lesistimu, isebenzisa i-UTF-8 njengefomethi ezenzakalelayo kumasethi ayo epharamitha, esindisa izinkinga zoku-oda nge-byte futhi ilondoloze ngokusobala i-ASCII.

I-Unicode in Practice: Iwebhu, Amadokhumenti, Nokuhlela

Kuwebhu, kuvamile ukukhonza nokugcina okuqukethwe ku-UTF-8 . Ku-HTML, memezela '<meta charset="UTF-8">' ukuqinisekisa ukuhambisana, futhi usebenzise izinhlaka zezinombolo ze-hex uma kudingeka (isibonelo: i-euro '&#x20AC;').

Ku-Word, ungafaka kalula izimpawu ze-Unicode ngokubeka i-cursor, uye ku-Insert > Symbol, noma ngokuthayipha ikhodi bese ucindezela u-Alt+X; lokhu kuguqula inani libe uhlamvu lwayo, kulandela izinqubo ezijwayelekile. Bheka futhi uhlu lwamakhodi e-Alt wokufaka izimpawu kusuka kukhibhodi.

Ezilimini zokuhlela : ku-Python 3, izintambo sezivele ziyi-Unicode; ku-Java naku-C#, ungasebenzisa izinhlamvu ze-escape '\uXXXX'; ku-HTML, '&#xXXX;'. Into ebalulekile ukuthi ukuhlela, ukuhlanganisa, kanye nokudlulisa kusebenzise ikhodi efanayo ukugwema amaphutha.

Ukukopisha nokunamathisela izimpawu kusebenza uma yonke intambo iku-Unicode . Uma izingxenye zayo zisebenzisa ikhodi ehlukile, kungase kuvele izimpawu zemibuzo, izikwele, noma ezinye izimpawu ezingajwayelekile.

Isizindalwazi sohlamvu (UCD), izakhiwo nezigaba

I-Unicode Character Database (UCD) ishicilela, ngephuzu ngalinye lekhodi, igama layo, isigaba, iskripthi, izakhiwo zecala, ukuqondiswa, nezinye izici . Lolu lwazi lubalulekile ekwenzeni izinjini kanye nokucubungula umbhalo kusebenze kahle.

Ngenxa yalezi zakhiwo , izingxenye ezahlukene kanye nama-algorithms (njengokuhlunga, ukuhlukaniswa kwamagama, noma ukuguqulwa kosonhlamvukazi/osonhlamvukazi abancane) kungasebenza ngokuhambisana nedatha efanayo.

Izinguqulo nezandiso zezinga: okuvelele

I-Unicode ikhula ngenguqulo ngayinye , ifaka izikripthi ezintsha, izimpawu, nama-emoji. Izinyathelo ezibalulekile zifaka phakathi inguqulo 1.0 ngo-1991 enezinhlamvu ezingu-7.161, kanye nokwandiswa okulandelayo okungeze izinkulungwane zezimpawu ezintsha, ama-ideogram, ama-emoji, kanye nezikripthi.

Isibonelo, ngo-2022 , inguqulo 15.0 yengeze ama-ideogram e-CJK engeziwe angu-4.192 nezinye izakhi, yafinyelela cishe izinhlamvu ezingu-149.186.

Amathuluzi okuhlola ithebula le-Unicode

Kunezinsiza zamahhala zokusesha nokuhlaziya izinhlamvu : I-Unibook Character Browser, i-SYMBL, kanye ne-Branah.com zikuvumela ukuthi ubuze ngezakhiwo, amagama, kanye namabhulokhi ezinhlamvu, okwenza kube lula ngabathuthukisi nabadali bokuqukethwe ukuthola ulwazi abaludingayo ngokushesha.

Sebenzisa izimo: amafomu ekheli, ukuhwebelana kwamanye amazwe, kanye nebhizinisi

Ukuvumela abasebenzisi ukuthi bafake amakheli ngolimi lwabo kanye nesikripthi kusiza ukunciphisa amaphutha nokuthuthukisa ulwazi lomsebenzisi. Ngaphezu kwalokho, ukusebenzisana kwe-Unicode kuvimbela izinkinga zokuguqulwa phakathi kwezinhlelo ezahlukene, kugcina ubuqotho bombhalo kulo lonke uchungechunge lwedijithali.

Ngenxa yalesi sizathu, i-Unicode iyisici esiyisisekelo esiqinisekisa ukuthi umbhalo uhlala ungowokugcina kuyo yonke ingqalasizinda yedijithali , kusukela kumafomu ewebhu kuya ezinhlelweni zedatha kanye namadokhumenti aphrintiwe, kungakhathaliseki ukuthi usebenzisa u-'ñ', isi-Arabhu, isiShayina noma ama-emoji emshweni ofanayo.