Iyini i-UTF-8: incazelo, ukusebenza, amaphutha kanye nokusetshenziswa

I-Informatec Digital » Ikhompyutha » Kuyini i-UTF-8: incazelo, ukusebenza, amaphutha kanye nokusetshenziswa

Uma ufunda lesi sihloko namuhla ngaphandle kokubona izimpawu ezingajwayelekile, kungenxa ye-UTF-8 . Lokhu kubhala ikhodi kuvumela izinhlamvu, ama-accent, izimpawu zobuchwepheshe, ngisho nama-emoji ukuthi aboniswe ngendlela efanayo kunoma yisiphi isiphequluli sesimanje, uhlelo lokusebenza, noma iklayenti le-imeyili. Kuyindinganiso ebanzi kakhulu kuwebhu futhi iyisisekelo sokuxhumana kwedijithali njengoba sisazi.

Uma idivayisi ibonisa umbhalo, empeleni icubungula izinombolo . Lezi zinombolo zingamaphuzu ekhodi achazwa yi- Unicode standard , futhi ukuze siziguqule zibe amabhayithi ahamba ngenethiwekhi noma agcinwe kufayela, senza ushintsho: UTF-8 . Emigqeni elandelayo, uzoqonda ukuthi iyini, isebenza kanjani, ukuthi kungani yaba yijwayelekile, izinzuzo zayo kanye nemikhawulo, nokuthi ungagwema kanjani amaphutha avamile.

Iyini i-UTF-8?

I-UTF-8 (ifomethi yokuguqulwa kwe-Unicode engu-8-bit) iyindlela yokuguqula amaphuzu ekhodi ye-Unicode abe yi-byte sequences . Isici sayo esiyinhloko ukuthi isebenzisa ubude obuguquguqukayo : ezinye izinhlamvu zihlala ku-byte eyi-1, kanti ezinye zidinga ama-byte angu-2, 3, noma angu-4. Lokhu kuvumela imibhalo emincane enezinhlamvu ezilula zesiLatini , kodwa futhi ingamelela noma yimuphi umlingiswa ovela ku-repertoire ye-Unicode.

Ihambisana ngokugcwele ne -ASCII : izinhlamvu zokuqala ezingu-128 (U+0000 kuya ku-U+007F) zifakwe ikhodi nge-byte eyodwa efana ne-ASCII engu-7-bit. Lokhu kwenza kube lula ukuguquka kusuka ezinhlelweni ezindala futhi kuchaza impumelelo yayo enkulu ku-inthanethi, ku-imeyili, kanye nakuphrothokholi ye-IETF.

I-UTF-8 ivelele ngokuqina kwayo : ifaka ama-bits okuvumelanisa avumela ukuhlonza okuthembekile kokuqala kophawu ngalunye. Le mpahla yokuvumelanisa ngokwayo yenza kube lula ukubona ukuthi ukulandelana "kubukeka" njenge-UTF-8 , okuwusizo kakhulu kumathuluzi kanye nabahlaziyi.

I-Unicode: Isisekelo Sakho Konke

I-Unicode iyindinganiso evamile enikeza inombolo ehlukile kumlingiswa ngamunye , kungakhathaliseki ulimi, ipulatifomu, noma uhlelo lokusebenza. Le nombolo ibizwa ngokuthi iphuzu lekhodi futhi ivame ukubhalwa nge-hexadecimal ngefomethi ethi U+XXXX (noma izinombolo ezengeziwe uma kudingeka).

Isibonelo, Usonhlamvukazi “A” uthi U+0041Ku-HTML singabhekisela kuyo ngokuthi A. Ikhompyutha yakho "ayicabangi" ngo-A njengohlamvu, kodwa njengenombolo 65, bese ukubhala ngekhodi (okufana ne-UTF-8) kunquma indlela yokumela leyo nombolo ngamabhayithi.

Uma ufuna ukubona ukuthi i-Unicode ihumusha kanjani izinhlamvu ku-PC yakho , ku-Windows ungabamba inkinobho ethi Alt bese uthayipha ikhodi yezinombolo yedesimali kukhiphedi yezinombolo: isibonelo, i-Alt+65 ibuyisela u-“A” (bheka uhlu oluphelele lwamakhodi e-Alt ). Kuyindlela enqamulelayo yakudala ekhombisa ukuthi amakhodi angaphansi kwezinhlamvu ozibonayo.

Umlando omncane: ukuthi i-UTF-8 yazalwa kanjani

I-UTF-8 yasungulwa nguKen Thompson ngaphansi kokuqondiswa kukaRob Pike ngoSepthemba 2, 1992. Bayisebenzisa ohlelweni lokusebenza lwe-Plan 9 oluvela eBell Labs futhi bayiveza ngokusemthethweni e -USENIX (San Diego, ngoJanuwari 1993) . Ngesikhathi sokumiswa kwayo, exhaswe yi- X/Open Joint Internationalization Group (XOJIG) , yayaziwa ngamagama anjenge-FSS/UTF kanye ne-UTF-2 ngaphambi kokuhlanganiswa njenge-UTF-8.

Lo mklamo uxazulule izinkinga ezingokoqobo ezazihlupha imizamo yangaphambilini yokufaka ikhodi yendawo yonke: ukuhambisana kwe-ASCII, ukuzivumelanisa, ukungabi nama-byte adlulanayo, kanye nokulula kokuthola amaphutha. Lokhu kulinganisela kwenze kwaba yindinganiso yangempela yewebhu.

Isebenza kanjani i-UTF-8 ngaphansi kweHood

I-UTF-8 ihlanganisa izinhlamvu ngokwe-byte ezidingekayo ukuze zifakwe ikhodi . Inani lama-byte lincike kuphela ephuzwini lekhodi ye-Unicode futhi lilandela amaphethini amancane abonisa ubude bochungechunge.

• Ibhayithi elingu-1 (U+0000 ukuya ku-U+007F): Izinhlamvu ze-ASCII. Ifomethi: 0xxxxxxx. Ibhithi ebaluleke kakhulu ngu-0, okuyinto iqinisekisa ukuhambisana okuqondile ne-ASCII.
• 2 amabhayithi (U+0080 ukuya ku-U+07FF): Ifomethi 110yyyyy 10xxxxxx. Isetshenziswa kuma-alfabhethi amaningi ase-Europe anezinhlamvu zezinhlamvu nezinye ezifana nesiGreki, isiCyrillic, isiHebheru noma isi-Arabhu..
• Amabhayithi ama-3 (U+0800 ukuya ku-U+FFFF): Ifomethi 1110zzzz 10yyyyyy 10xxxxxx. Kufaka phakathi uhlelo oluyisisekelo lwezilimi eziningi (BMP), ne-CJK (isiShayina, isiJapane, isiKorea), izimpawu zobuchwepheshe nezinhlamvu ezisetshenziswa kakhulu.
• Amabhayithi angu-4 (U+10000 ukuya ku-U+10FFFF): Ifomethi 11110uuu 10uuzzzz 10yyyyyy 10xxxxxx. Imele izindiza ezengeziwe: izimpawu zezibalo ezithuthukisiwe, imibhalo yomlando, izimpawu ze-ideographic ezingajwayelekile, njll.

Isihluthulelo sokuzivumelanisa sisezinhlakeni zekhanda : 0 ye-ASCII; 110 yama-byte amabili; 1110 kwamathathu; 11110 kwama-byte amane. Ama-byte okuqhubeka aqala ngo-10. Ngenxa yalokhu, i-byte yokuqhubeka ayikwazi ukuvela njenge-byte yokuqala , futhi ukulandelana okusebenzayo akusoze kwaba yi-substring ye-long (isimiso esingahlangani).

Ukulingana ne-UTF-16 kanye namapheya alandelayo

I-UTF-16 imele amaphoyinti ekhodi ye-BMP aneyunithi yamabhithi angu-16 kanye namaphuzu ngenhla kwe-U+FFFF nge amapheya abambele ebangeni D800–DFFF. Esikhundleni salokho, I-UTF-8 ihlezi ibhala amaphoyinti ekhodi yangempela, hhayi amayunithi e-UTF-16, agwema ukudideka nokumiselela.

Ngokomlando, amanye ama-draft avumela ama-byte angu-5 noma angu-6 ku-UTF-8 ukuthi amboze ububanzi obubanzi, kodwa i-Unicode kanye ne-RFC 3629 zikhawulela i-UTF-8 ibe yi-byte engu-4 . I-ISO/IEC icabangele izinketho ezibanzi, kodwa lezi aziyona ingxenye yendinganiso yamanje.

Isibonelo esisebenzayo: i-ñ

Uhlamvu “ñ” lunephoyinti lekhodi U+00F1, ewela phakathi kwebanga lamabhayithi amabili. Ngokulandela iphethini, ibhalwa njenge 110xxxxx 10xxxxxx. Ukumelwa kwayo kwe-UTF-8 kungu-0xC3 0xB1Ukukhipha amakhodi kuyinqubo ehlehlayo: ukufunda izingcezu eziwusizo nokwakha kabusha iphoyinti lekhodi yasekuqaleni.

Izinzuzo kanye nemikhawulo ye-UTF-8

Izinzuzo eziyinhloko :

• Ukusekelwa kwe-ASCII: Imibhalo ye-ASCII ivumelekile ku-UTF-8 ngaphandle kwezinguquko.
• Universal: ingamela noma yiziphi izinhlamvu ze-Unicode, okuhlanganisa nezimpawu zobuchwepheshe nama-emoji.
• Ukusebenza kahle emibhalweni yesiLatini: uma usebenzisa i-byte engu-1 ye-ASCII, yonga isikhala uma iqhathaniswa ne-UTF-16 ngezilimi eziningi zaseNtshonalanga.
• Ukuzivumelanisa nokutholwa: amaphethini amancane avumela bona iziqalo zezinhlamvu futhi uqinisekise ukulandelana kalula.

Imikhawulo kanye nokushintshana :

• Imibhalo ye-CJK ithatha isikhala esiningi kunemibhalo ye-UTF-16, lapho eziningi zalezo zinhlamvu zingena kumabhayithi alungisiwe angu-2.
• Izindleko zokubala: ukuba nobude obuguquguqukayo, okunye ukusebenza (isb., “iya kuhlamvu n”) kudinga ukudlula kusukela ekuqaleni, futhi imisebenzi ethile ingase isheshe ku-UTF-16/UTF-32.

I-BOM (I-Byte Order Mark) ku-UTF-8

I-UTF-8 ayidingi i-BOM ngoba i-byte order ayishintshi incazelo yamanani (iyunithi encane kunazo zonke i-byte). Noma kunjalo, Kukhona i-BOM yokuzikhethela, uhlamvu U+FEFF olufakwe njenge EF BB BF ekuqaleni kwefayela noma ukusakaza, okungasetshenziswa ukukhombisa “lena i-Unicode/UTF-8”.

Izindlela ezinhle kakhulu : Uma kuvela ekuqaleni, ezinye izinhlelo ziyakwamukela kanti ezinye zikuphatha ngokoqobo. Ezihlanganisweni, kuyalulekwa ukususa ama-BOM aphakathi . Ukukufaka akuyona impoqo, futhi ukusebenza kwayo ku-UTF-8 kunqunyelwe uma kuqhathaniswa ne-UTF-16/UTF-32, lapho kuphawula khona i-endianness.

Amaphutha okubhala amakhodi ajwayelekile nendlela yokubhekana nawo

I-decoder enamandla ye-UTF-8 kumele yenqabe ama-sequences angalungile noma iwashintshe nge-U+FFFD (REPLASEMENT CHARACTER) noma iphawule iphutha. Amaphutha avame kakhulu yilawa:

• Ukulandelana okunqanyuliwe: i-multibyte lead byte ngaphandle kokuqhubeka okwanele.
• Khulula amabhayithi aqhubekayo: vela 10xxxxxx ngaphandle kwebhayithi yokuhola evumelekile.
• Ubude obude: ukubhala ngekhodi ngamabhayithi amaningi kunokudingekile; isibonelo, ukuzama ukufaka ikhodi ye-ASCII ngamabhayithi angu-2 (0xC0 y 0xC1 azivumelekile).
• Ubude obunqatshelwe: iqala ukuphakamisa amabhayithi angu-5 noma angu-6 (0xF8-0xFD azivumelekile ku-UTF-8 ejwayelekile).
• Amanani angaphandle kwebanga le-Unicode: ayisekelwe ngaphezu kwe-U+10FFFF; amanani athile (0xF5-0xF7 njengeziqalo) azivumelekile.
• I-UTF-16 i-surrogate pairs: D800–DFFF akuwona amakhodi ekhodi avumelekile ku-Unicode; akumele zivele zifakwe ikhodi ku-UTF-8.

Uma ubona uhlamvu “�” esikrinini , kungenzeka kakhulu ukuthi kungenxa yokungafani kokubhala ikhodi noma ifayela eligcinwe ekhasini lekhodi ehlukile. Isixazululo ukuphoqa ukufaka ikhodi ye-UTF-8 kusukela ekuqaleni kuya ekugcineni (ifayela, iseva, isizindalwazi, ama-header e-HTTP).

I-UTF-8 kuwebhu naku-imeyili

Ikhasi le-HTML lidinga kuphela ukumemezela ikhodi eyodwa . Ikhodi enconyiwe, yokuhambisana kanye nobubanzi, yi-UTF-8. Faka i-meta tag elandelayo kusihloko ngokushesha okukhulu:

<meta charset="UTF-8">

Yibeke ekuqaleni kwethegi ethi `<head>` ukuze isiphequluli siyifunde ngaphambi kokucubungula idokhumenti. Lokhu kuvimbela ukungalingani kanye nezinhlamvu "eziphukile". Ukwamukelwa kwe-UTF-8 kuwebhu kuyamangaza ; isetshenziswa iningi lamawebhusayithi amanje.

Ku-imeyili, i-UTF-8 isekelwa kakhulu futhi inconywa yizinhlangano ezifana ne-Internet Mail Consortium. Ukuhlela amaklayenti e-imeyili ukuze asebenzise i-UTF-8 kunciphisa izinkinga lapho kushintshana imiyalezo nabantu abakhuluma ezinye izilimi.

I-UTF-8, UTF-16, ne-UTF-32: Uyini umehluko?

I-UTF-8 : Ukufaka ikhodi ngobude obuguquguqukayo kumayunithi angu-8-bit; ilungele iwebhu , isebenza kahle kakhulu nge-ASCII nezilimi zaseNtshonalanga. Ukuhambisana okuhle kakhulu nokutholwa kwamaphutha.

I-UTF-16 : ubude obuguquguqukayo kumayunithi angu-16-bit; isebenzisa ama-surrogate pairs e-U+10000 nangaphezulu. Ngokuvamile kuyazuzisa lapho izinhlamvu ezingezona eze-ASCII zibusa kakhulu , futhi isetshenziswa kuma-API amaningi namapulatifomu (isibonelo, i-Windows isebenza ngendlela yemvelo ku-UTF-16 ).

I-UTF-32 : ubude obuhleliwe bama-bits angu-32 ngohlamvu ngalunye; kulula kakhulu ukukhomba , kodwa kuthatha isikhala esiningi. Igcinelwe izimo lapho usayizi ulandela ubulula bokucubungula.

Izinhlobonhlobo ezingahambisani: CESU-8 kanye “ne-Modified UTF-8”

I-CESU-8 ifaka amayunithi e-UTF-16 ngqo (kufaka phakathi ama-surrogate pairs) esikhundleni sokufaka amakhodi amaphuzu ekhodi, ngaleyo ndlela ihluke ku-UTF-8 ejwayelekile yezinhlamvu ezingaphezu kwe-U+FFFF. Amanye amapulatifomu omlando ayisebenzise: I-Oracle 8 yayinikeza ngaphansi kwegama elithi UTF-8, futhi kusukela ku-Oracle 9, yengeze i-UTF-8 ejwayelekile ngaphansi kwegama elihlukile. I-Java ne-Tcl basebenzise i-CESU-8 ezimweni ezithile.

I-UTF-8 eguquliwe (isibonelo, ezindaweni zeJava) imele uhlamvu lwe-NUL (U+0000) njengo-0xC0 0x80 esikhundleni sika-0x00. Igwema i-null byte kuma-string e-C kodwa ayihambisani nezindinganiso ze-UTF-8. Ukusetshenziswa okuningi kwale nguqulo "eguquliwe" nakho kuyahambisana ne-CESU-8.

I-UTF-8 ku-Windows kanye nama-API: Amakhasi Ekhodi Nokuguqulwa

IWindows isebenza ngaphakathi ku-UTF-16 (WCHAR), kodwa kusukela Windows 10 inguqulo 1903 ungakwazi phoqa i-UTF-8 njengekhasi lekhodi yenqubo nge-manifest yohlelo lokusebenza (property activeCodePage). Lokhu kwenza kube lula ngekhodi yefa esebenzisa “-A” APIs ukusebenza nge-UTF-8.

Ama-API -A vs -W: -A kuncike ekhasini lekhodi ye-ANSI kumisiwe (kungaba yi-CP_UTF8), kuyilapho i- -W basebenzisa UTF-16. Ukusebenzisana, I-MultiByteToWideChar y I-WideCharToMultiByte ikuvumela ukuthi uguqule phakathi kwe-UTF-8 ne-UTF-16; usa CP_UTF8 futhi, uma kufanele, MB_ERR_INVALID_CHARS ukuthola amaphutha okokufaka.

Ukuhambisana kwangempela: iziphequluli namasistimu

I-UTF-8 iyahambisana neziphequluli zesimanje (i-Chrome, i-Firefox, i-Safari, i-Edge, i-Opera, kanye nezinguqulo zakamuva ze-Internet Explorer) kanye nezinhlelo eziningi zokusebenza (i-Windows, i-Linux, i-macOS, i-Android, i-iOS). Ngaphandle kokuthi unesofthiwe endala kakhulu, akufanele ube nezinkinga.

Ungawaguqula kanjani amafayela abe yi-UTF-8

Ku-Windows (Notepad) : vula ifayela, iya ku-"File > Save As…", bese ku-"Encoding" khetha i-UTF-8 . Londoloza ngegama elisha uma ufuna ukugcina elokuqala.

Ku-macOS (TextEdit) : Ku-“TextEdit > Izintandokazi > Vula & Gcina”, khetha i-Unicode (UTF-8) uma ulondoloza. Bese uthumela ifayela ngaleyo ndlela ivuliwe.

Kwi-Linux: nge-terminal ongayisebenzisa iconv. Isibonelo: iconv -f <codificación_origen> -t UTF-8 <entrada> -o <salida>. Hlola kamuva ukuthi uhlelo lokusebenza oluyidlayo futhi lulindele i-UTF-8.

Ungabona kanjani ukuthi ifayela liku-UTF-8? Abahleli abaningi banamuhla babonisa lokhu kubha yesimo. Uma ubona izinhlamvu ezingajwayelekile ezifana no-“�”, ama-accent onakele, noma u-“ñ/ç” oboniswe ngendlela engafanele, hlola ukufakwa kwekhodi kwefayela kanye nezilungiselelo zomhleli/iseva/isizindalwazi.

Imikhuba emihle yokugwema izimanga

Memezela i-UTF-8 kusenesikhathi ngangokunokwenzeka kuma-header e-HTML kanye ne-HTTP. Qondanisa ukufaka ikhodi kuyo yonke i-stack (amafayela omthombo, amathempulethi, isizindalwazi, kanye nokuxhumeka). Gwema ukuxuba ukufaka ikhodi ekhasini elifanayo noma ukugeleza, bese usebenzisa amathuluzi aqinisekisa/alungisa ukufakwa.

Okokuhlanganisa nama-API, hlala ucacise umbhalo wekhodi kumaheda (Content-Type: application/json; charset=UTF-8, Ngokwesibonelo). Hlola ngedatha yezilimi eziningi (ama-accents, i-CJK, ama-emoji) ukuze uthole amaphuzu abuthakathaka ngaphambi kokukhiqiza.

I-UTF-8 inqobe ngoba ilinganisela ukuhambisana, ukusebenza kahle, kanye nokufinyelela . Kuyindlela ewusizo kakhulu yokuqinisekisa ukuthi umbhalo uhamba kahle kuwo wonke amasiko, izinhlelo, kanye nezinhlelo zokusebenza, kungakhathaliseki ukuthi uhlanganisa ama-accent, izimpawu zobuchwepheshe, noma imibhalo engeyona yesiLatini.

I-athikili ehlobene:

Iyini i-Unicode: Umhlahlandlela ophelele, ukusetshenziswa, kanye nokubhala ngekhodi