Izidingo zehadiwe zokusebenzisa i-Ollama ngaphandle kokumiswa

Informatec Digital » Izinsiza » Izidingo zehadiwe zokusebenzisa i-Ollama ngaphandle kokumiswa

Uma ucabanga ukusebenzisa amamodeli obuhlakani bokwenziwa kukhompyutha yakho, uzohlangana no-Ollama ngokushesha noma kamuva. Futhi yilapho umbuzo omkhulu uphakama khona: yiziphi izidingo zehadiwe engizidingayo ukuze amamodeli asebenze kahle futhi ngaphandle kokungingiza? Akwanele ukuthi aqale nje; isihluthulelo ukuthi angasetshenziswa ngokunethezeka emisebenzini yansuku zonke, futhi ukuqonda izinhlobo ezahlukene zehadiwe yekhompyutha kubalulekile.

Kuyo yonke le ndatshana sizobona ngokuningiliziwe ukuthi i-Llama yenzani, ukuthi izinhlobo ezahlukene zamamodeli (7B, 13B, 70B, njll.) zidingani, ukuthi i-CPU, i-GPU, i-RAM kanye nediski kuthinta kanjani ukusebenza, nokuthi yiziphi izilungiselelo ezifanele kuye ngecala lakho , kungakhathaliseki ukuthi ufuna umsizi wombhalo olula noma uhlose ukusebenzisa izilo ezifana ne-Llama 3 ngamapharamitha angamashumi ezigidigidi noma amamodeli ombono kanye ne-OCR.

Uyini u-Ollama futhi kungani ihadiwe yenza umehluko omkhulu kangaka?

I-Ollama empeleni iyiklayenti lemodeli yolimi elikuvumela ukuthi usebenzise ama-LLM endaweni yakho emshinini wakho, ngaphandle kokuthembela efwini. Isebenzisa izinjini ezifana ne -llama.cpp ukuze ithole ukuqonda futhi ihlanganisa konke ukuguquguquka ngethuluzi elilula, nge-CLI kanye ne-REST API, futhi ikusiza ukuthi uqonde imiqondo yamanethiwekhi e-neural okwenziwa ngemuva kwamamodeli.

Indima yayo ukuba "isikhungo somyalo" lapho ulanda khona, uphatha, futhi usebenzise khona amamodeli afana ne-Llama 3, i-Mistral, i-Gemma, i-Phi, i-Qwen, i-DeepSeek, noma amamodeli e-multimodal afana ne-Llava . Ubuhle ukuthi ungawasebenzisa ngokuphelele ungaxhunyiwe ku-inthanethi, ugcine idatha yakho endaweni futhi ngaphandle kokukhokhela ithokheni ngayinye njengoba kunjalo ngama-API efu.

Manje, yize i-Ollama ngokwayo ilula futhi ingadingi okuningi, amamodeli ewasebenzisayo adinga kakhulu izinsiza . I-LLM ngayinye inamapharamitha ayizigidi noma izigidigidi, ahumushela kuma-gigabytes enkumbulo nesitoreji, kanye nomthwalo osindayo ku-CPU, futhi uma unayo, i-GPU.

Yingakho, uma umuntu ezama ukusebenzisa imodeli enkulu (isibonelo, i-70B Llama) ohlelweni olune-CPU enamandla kodwa i-GPU ehlukile kanye ne-RAM eyanele, umphumela uvame ukuba "uyasebenza," kodwa kuhamba kancane kangangokuthi akusizi ngalutho . Isihluthulelo ukulinganisela kahle i-CPU, i-GPU, i-RAM, i-hard drive, kanye nohlobo lwemodeli.

Izinhlobo zamamodeli ku-Ollama nokuthi zithinta kanjani izidingo

Emtatsheni wezincwadi wase-Ollama uzobona amamodeli ahlelwe ngemindeni nosayizi: 1B, 2B, 4B, 7B, 13B, 30B, 65B, 70B, 405B… . Leyo nombolo (B yezigidigidi) ikhombisa inani elilinganiselwe lamapharamitha, futhi ingenye yezinto ezinquma kakhulu ihadiwe edingekayo.

Singawahlukanisa ngendlela evamile ngezigaba ezine , okusiza kakhulu ekulinganiseni ukuthi yimuphi umshini owudingayo ukuze uzizwe ukhululekile ngeqembu ngalinye lamamodeli kanye nezilinganiso:

• Amamodeli amancane (270M – 4B): yakhelwe imishini enesizotha (ama-laptop alula, ngisho namaselula noma ama-mini-PC). Kuyashesha, kodwa kunekhono elincane lokucabanga.
• Amamodeli amancane (4B - 14B): ifaneleka njenge amamodeli "asekhaya" alinganisiweKuhle kakhulu engxoxweni evamile, imisebenzi yasehhovisi, usizo lokubhala ikhodi elula, njll.
• Amamodeli aphakathi nendawo (14B - 70B)Sebevele badlala kuligi ehlukile; Badinga ihadiwe enamandla., i-RAM eningi, futhi uma kungenzeka, i-GPU ene-VRAM eningi.
• Amamodeli amakhulu (> 70B)Ziyizilwane eziklanyelwe ingqalasizinda engathi sína kakhulu (Ama-GPU aphezulu, amakhadi ehluzo amaningi, amaseva azinikele, ama-Mac aphezulu asetshenziswa kahle, njll.).

Ngaphezu kosayizi, i-quantization nayo iyasebenza : ku-Ollama uzobona izijobelelo ezifana no- q4_K_M, q5_1, q3_K_S, q8_0, f16 , njll. Lawa mafomethi abonisa ukuthi izisindo zemodeli zicindezelwe kangakanani .

• FP16 / FP32 (f16, f32): kucindezelwe kancane, ukusetshenziswa kwememori okunekhwalithi ephezulu kodwa okunonyaI-7B ku-FP16 ingafinyelela ku-VRAM engaphezu kuka-20 GB.
• Q4 (q4_0, q4_K_M…): Ukulinganisa okungu-4-bit, ukunciphisa usayizi omkhulu okunomthelela omaphakathi kwikhwalithiNgokuvamile "yizindawo ezimnandi".
• Umbuzo 3, Umbuzo 2 (q3_K_S, q2_K…): ukulinganisa okunamandla kakhulu, usayizi omncane kakhulu ukuze uthole ukulahlekelwa okuncane kokunembaIwusizo kwihadiwe elinganiselwe kakhulu.
• Q5, Q6, Q8: izinyathelo eziphakathi nendawo phakathi kokucindezela okunamandla kanye ne-FP16; Ikhwalithi ephezulu, ukusetshenziswa okuphezulu.

Umphumela osebenzayo ucacile: imodeli efanayo ye-7B ingadla cishe ama-GB angu-26 ku-FP16 noma cishe ama-GB angu-4 ku-Q4 . Lokhu kuhunyushwa ngqo ku-GPU VRAM oyidingayo kanye nenani le-RAM okumele lisekele umthwalo womsebenzi.

Izidingo zehadiwe ezincane nezinconywayo ze-Ollama kunethiwekhi yendawo

Uma ukhathazekile ngokuthi ikhompyutha yakho ingakwazi yini ukuphatha i-Ollama, impendulo ivame ukuba yebo; umbuzo uwukuthi iyiphi imodeli ongayisebenzisa kahle . Ake sihlukanise ngezingxenye: i-RAM, i-CPU, i-GPU, kanye ne-hard drive, ngezincomo ezingokoqobo ezisekelwe kokuhlangenwe nakho okusebenzayo kanye nemibhalo evela eziqondisweni eziningana ezikhethekile.

I-RAM: umthombo obalulekile kakhulu

I-RAM iyisithiyo esiyinhloko lapho kuxoxwa ngama-LLM endawo. Ngokuvamile, singacabangela la mabanga:

• I-8 GB ye-RAM: isisekelo esisebenzayo. Ivumela amamodeli amancane (1B, 3B, uhlobo oluthile olulinganiselwe kakhulu lwe-7B)Kodwa-ke, uzobona ukulinganiselwa, ikakhulukazi uma uhlelo kanye nesiphequluli sezivele zisebenzisa imemori eningi. Kungenzeka ukuthi konke kuzohamba kancane futhi kube nokulibaziseka okwengeziwe.
• I-16 GB ye-RAM: indinganiso enengqondo namuhla. Ilungele amamodeli angu-7B ngisho nangu-13B alinganiswe ku-Q4Ikakhulukazi uma usebenzisa ama-GPU. Ungasebenza ngezingxoxo eziyinkimbinkimbi ngaphandle kokuthi uhlelo lwehlise ijubane.
• 32 GB we-RAM noma ngaphezuluKunconywa uma ufuna amamodeli aphakathi nendawo (30B, 40B, 70B) noma ukwenza izinto ezisindayo njengezimo ezinde kakhulu, amamodeli amaningana kumaseva afanayo, anabasebenzisi abaningi noma amathuluzi ezithombe avulekile ohlobo lwe-WebUI ku-Ollama.

Khumbula ukuthi i-RAM ayisetshenziswa yidivayisi uqobo kuphela: uhlelo lokusebenza, isiphequluli, i-IDE, i-Docker, i-Open WebUI, nezinye izinhlelo zokusebenza nazo ziyayisebenzisa . Uma ufuna ukukhulula imemori ezimweni ezithile, ungafunda ukuthi unganciphisa kanjani ukusetshenziswa kwe-RAM ezinhlelweni zokusebenza ezifana nesiphequluli sakho. Ukuze kusetshenziswe kakhulu, i-16 GB okwamanje iyinto encane enconywayo, kanti i-32 GB ingaphezu kwenele.

I-CPU: Imiyalelo yesimanje kanye nenani lama-core

I-Ollama ingasebenza ku-CPU yodwa, kodwa ulwazi luyahlukahluka kakhulu kuye ngokuthi iprosesa iyasebenza yini. Okubaluleke kakhulu kunenani lama-cores ukusekelwa kwemiyalelo ethuthukisiwe njenge-AVX2 kanye, okungcono nakakhulu, i-AVX-512 , esheshisa ukusebenza kwe-matrix kanye ne-vector okusetshenziswa kakhulu kuma-LLM.

Isiqondiso esinengqondo singaba:

• Okuncane okwamukelekayoI-CPU yesimanje ene-quad-core (isibonelo, iprosesa ye-Intel i5 yesizukulwane sakamuva noma i-Ryzen efanayo) enokusekelwa kwe-AVX2. Uzokwazi ukwenza lokhu Sebenzisa amamodeli e-7B ngesineke, ikakhulukazi uma elinganiswe kahle.
• Kunconyiwe: uhlobo lwakamuva lweprosesa Isizukulwane se-Intel se-11 noma kamuva noma i-AMD Zen4, nge Ama-cores angu-8 noma ngaphezulu kanye nokusekelwa kwe-AVX-512 lapho kungenzeka khona. Ngale ndlela uthola izikhathi zokuphendula ezithuthukisiwe kanye nokunciphisa ukuvinjelwa, ngisho nangama-GPU.

Uma uhlelo lwakho luwukusebenzisa amamodeli amakhulu kakhulu (isibonelo, ukuzama i-70B Llama 3 ene-CPU ne-GPU encane), i-CPU izoba nzima futhi uzobona izikhathi zokukhiqiza amathokheni aphezulu kakhulu . Kulezi zimo, indlela enengqondo kakhulu ukukhetha amamodeli amancane noma ukutshala imali ku-GPU efanelekile.

I-GPU ne-VRAM: kubaluleke nini futhi kungakanani okudingekayo

I-GPU ayiphoqelekile, kodwa yenza umehluko omkhulu. I-GPU efanelekile ene-VRAM eyanele ingaguqula isipiliyoni esihamba kancane sibe yinto esebenziseka kahle , ikakhulukazi ngamamodeli alinganisiwe angu-7B kuya ku-13B.

Njengereferensi ewusizo kakhulu , kumamodeli alinganisiwe (cishe i-Q4) into efana nale ingalinganiselwa:

• 7B → ~4 GB ye-VRAM
• 13B → ~8 GB ye-VRAM
• 30B → ~16 GB ye-VRAM
• 65-70B → ~32 GB ye-VRAM

Lezi yizilinganiso ezilinganiselwe, kodwa zibonisa ngokusobala ukuthi i-RTX 2060 SUPER GPU ene-8 GB ye-VRAM inamandla angaphezu kuka-7B futhi ingaphatha i-13B, kodwa ayifiki ku-70B ngisho noma ine-i9 kanye ne-64 GB ye-RAM. Kuleso simo, uhlelo luzophoqeleka ukusabalalisa umthwalo omningi phakathi kwe-RAM ne-CPU, futhi ukubambezeleka kuzophakama kakhulu.

Ngokwemibandela engokoqobo :

• cunt 4-6 GB ye-VRAM: Gxila ku amamodeli e-7B alinganiswe kahleZisebenza kahle kakhulu engxoxweni, ekubhaleni, kanye nasemisebenzini ejwayelekile.
• cunt 8-12 GB ye-VRAMUngasebenza ngokunethezeka nge 7B kanye no-13B ngisho nama-30B uma uzimisele ukuhamba kancane.
• cunt 20-24 GB ye-VRAMManje usungena endaweni ye- Amamodeli angu-30B-40B anesithunzi esikhulu, kanye ne-70B elinganiswe kakhulu, ikakhulukazi uma uyisekela nge-RAM enhle.
• cunt 32 GB ye-VRAM noma ngaphezulu: kunini lapho I-70B iqala ukubonakala inengqondo ngempela ukuze kusetshenziswe ngokuxhumana, uma nje bonke abanye beqembu behambisana.

Kumodeli ye-OCR noma amanye amamodeli akhethekile (isb., umbono), i-GPU ene-20-24 GB ye-VRAM iyisisekelo esiqinile kakhulu sokusebenza okubushelelezi , ikakhulukazi uma imodeli ifaka amashumi ezigidigidi zamapharamitha. Kuma-OCR alula (2B-7B) noma ama-variants ombono, i-8-12 GB kufanele yanele ngokuphelele.

Isitoreji sediski: ingakanani indawo ethathwa amamodeli

Ngokuphathelene nesikhala sediski, uhlelo lokusebenza lwe-Ollama ngokwalo luthatha okuncane kakhulu; okuthatha isikhala ngempela amamodeli. Endaweni eyisisekelo noma yokuhlola, cishe ama-GB angu-50 anele , kodwa uma uqala ukuqoqa amamodeli, isikhala sanda ngokushesha.

Njengesiqondiso esilula samamodeli alinganisiwe:

• Amamodeli amancane (1B-4B) → nxazonke 2 GB ngemodeli.
• Amamodeli aphakathi nendawo (7B-13B) → ngokuvamile I-4-8 GB ngemodeli ngokuya ngesilinganiso.
• Amamodeli amakhulu (30B-70B) → kalula I-16-40 GB ngamunye
• Amamodeli amakhulu kakhulu (> 100B) → kungadlula 200 GB ngemodeli futhi idlule ngisho nama-terabyte kwezinye izimo ezimbi kakhulu.

Okungcono kakhulu, sebenzisa i-SSD esheshayo (NVMe uma kungenzeka) ukuze ulayishe imodeli yokuqala ngokushesha. Ngaphezu kwalokho, i-Ollama ikuvumela ukuthi ushintshe indawo yokugcina imodeli usebenzisa i- OLLAMA_MODELS environment variable , ukuze usebenzise idrayivu enkulu yesibili futhi ukhulule isikhala kudrayivu yakho eyinhloko. Ukuze uthole ulwazi olwengeziwe mayelana nomthamo wesitoreji kanye nezinhlobo zedrayivu, bheka umhlahlandlela wehadiwe yesitoreji.

Izidingo ezithile zokusebenzisa amamodeli athile ne-Ollama

Nakuba imodeli ngayinye inezincazelo zayo, nge-ecosystem yamanje ye-Ollama kuneziqondiso ezicacile ezinganikezwa ngezigaba zokusetshenziswa ezijwayelekile: ingxoxo evamile, ukufaka ikhodi, amamodeli ombono/ama-OCR kanye namamodeli amakhulu ohlobo lwe-70B.

Amathempulethi engxoxo evamile (i-Llama, i-Mistral, i-Gemma, i-Qwen…)

Ukusetshenziswa okuvamile kwe-"ChatGPT yendawo" ngamamodeli aphakathi nendawo njenge-Llama 3.x 7B/8B, Mistral 7B, Gemma 2B/7B noma i-Qwen , ukusetha okunengqondo namuhla kungaba into efana nale:

• Okunconywayo okuncane:
  • I-CPU yesimanje ene-quad-core ene-AVX2.
  • I-16 GB ye-RAM.
  • Ayikho i-GPU noma i-GPU eyisisekelo ene-4-6 GB VRAM.
  • Okungenani i-SSD engu-50 GB yesistimu + imodeli eyodwa noma ezimbili.
• Ukucushwa okuhle kakhulu kokuba negumbi elikhulu eline-7B-13B:
  • I-CPU enama-cores angu-8 noma ngaphezulu (i-i7/i9 yesimanje noma i-Ryzen 7/9).
  • I-32 GB ye-RAM uma ufuna ukugcina izinto eziningi zivulekile.
  • GPU nge 8-12 GB ye-VRAM (RTX 3060/3070 noma okulingana nayo, i-AMD RX 6700 noma ngaphezulu, noma i-Mac ene-M1/M2/M3 esetshenziswa kahle).
  • I-SSD engu-1 TB uma uzoqoqa amamodeli.

Kulezi zimo, amamodeli e-7B ane-quantization ye-Q4_K_M noma ye-Q5_K_M asebenza kahle kakhulu futhi anikeza ikhwalithi eyanele yokusetshenziswa komuntu siqu, imibhalo yobuchwepheshe, imisebenzi yokufunda, noma ukwesekwa kokubhala.

Amamodeli okubhala ikhodi (i-DeepSeek, i-CodeLlama, i-Phi egxile kukhodi)

Amamodeli akhethekile ekuhleleni izinhlelo ngokuvamile anezidingo ezifanayo namamodeli engxoxo ajwayelekile anosayizi ofanayo , kodwa kuyalulekwa ukuthi ugcine imajini ethe xaxa ku -RAM kanye ne-VRAM uma uzowasebenzisa kanye ne-IDE esindayo kanye namaphrojekthi amaningi avulekile.

Isibonelo, ukusebenzisa into efana ne -DeepSeek-Coder (7B-8B) noma i-CodeLlama enobukhulu obufanayo ngaphansi kwezimo ezifanele , inhlanganisela enengqondo kakhulu ingaba:

• CPU ama-cores anamuhla angu-6-8.
• I-32 GB ye-RAM uma usebenza ngamathuluzi amaningi ngesikhathi esisodwa (i-IDE, isiphequluli esinamathebhu, i-Docker, njll.).
• I-GPU ene-VRAM okungenani engu-8 GB ukuhambisa imodeli kahle.

Isebenza futhi kwihadiwe engenamandla kakhulu, kodwa uzobona izikhathi zokuphendula ezithatha isikhathi eside lapho udala amabhlogo amade ekhodi noma wenza ukuhlaziya okuyinkimbinkimbi . Kumamodeli amancane njenge -Phi-4 Mini, izidingo ziphansi kakhulu, futhi asebenza kahle ngisho nasezinhlelweni ezine-RAM engu-16 GB kanye ne-GPU elula.

Amamodeli e-Vision kanye ne-OCR (Ukhiye, amamodeli e-OCR, amamodeli e-multimodal)

Amamodeli anamakhono okucubungula izithombe (umbono/i-OCR) njenge- Llama noma izinhlobo eziningi ze-Llama 3.x, kanye namamodeli athile e-OCR, anezela ungqimba olwengeziwe lobunzima. Ezingeni lehadiwe, afinyelela izidingo zemodeli yombhalo enobukhulu obufanayo, kodwa ngenzuzo enkulu yokusebenzisa i-GPU.

Uma sikhuluma ngemodeli ye-OCR ephakathi nendawo (ake sithi kububanzi be-7B-13B) futhi ufuna ukuyisebenzisa kahle endaweni ukuze uqaphe amadokhumenti, izithombe eziskeniwe, njll., kunengqondo ukucabangela into efana nalokhu:

• I-GPU ene-VRAM engu-20-24 GB kungakhathaliseki ukuthi imodeli inkulu ngempela noma uma ufuna ukushiya cishe konke ukucutshungulwa ekhadini.
• I-GPU ene-VRAM engu-8-12 GB Uma ukhetha izinhlobo ezilula nezilinganiselwe kahle, zizoqhubeka nokusebenza kahle uma nje ungasebenzisi ngokweqile usayizi wesithombe noma izimo ezinkulu.
• Ubuncane be-16 GB ye-RAM, yize i-32 GB inikeza i-margin ekhululekile kakhulu yokusetshenziswa kakhulu.
• i-CPU yesimanje ukuze ingavimbi lapho i-GPU ilayishwa.

Impendulo eqondile yombuzo ojwayelekile othi "ngingasebenzisa imodeli ye-OCR ku-GPU ene-20-24 GB ye-VRAM?" inguyebo , iyibanga elihle kakhulu lamamodeli ombono/ama-OCR aphakathi kuya kwamakhulu ku-Ollama , uma nje une-RAM eyanele kanye ne-CPU enhle.

Amamodeli amakhulu (i-Llama 3:70B kanye nokunye okufanayo)

Ukuzama ukusebenzisa i- 70KB Llama 3 ku-Windows nge-CPU enamandla kakhulu (isibonelo, isizukulwane se-11 i9) kanye ne -64GB ye-RAM kodwa i-GPU efana ne-8GB RTX 2060 SUPER iyisibonelo esiphelele sokuthi "yebo, kodwa cha." Imodeli ingase ilayishe ekugcineni, kodwa:

• Ingxenye yemodeli ayifaneleki ku-VRAM futhi incike kakhulu ku-RAM.
• I-CPU kufanele ithathe umsebenzi omningi wokuqagela.
• Isikhathi ngethokheni siyakhuphuka futhi isipiliyoni asisasebenzi nhlobo..

Ukuze i-70B ibe nomqondo ekhaya noma ezindaweni ezingezona ezobungcweti, udinga, okungenani , okuthile okufana nalokhu:

• I-RAM engu-32 GB njengesisekelo, i-64 GB uma ufuna i-headroom eyengeziwe.
• I-GPU ene-VRAM okungenani engu-24-32 GB ukulayisha iningi lemodeli ngesilinganiso esinengqondo (Q4_K_M noma esifanayo).
• I-CPU enamandla ephezulu enama-cores angu-8-16.

Uma ungahlangabezani nalezi zibalo, kungcono kakhulu ukusebenzisa amamodeli e-7B-13B alinganiswe kahle noma, uma udinga ngempela i-70B ukuze uthole ikhwalithi, cabanga ngeseva ekhethekile (yendawo noma esefwini), i-Mac enamandla kakhulu, noma ama-GPU amaningana asebenza ngesikhathi esisodwa.

Izidingo zokufaka i-Ollama ku-VPS noma kuseva

Enye inketho evamile ukufaka i-Ollama ku- VPS noma iseva ezinikele bese uyifinyelela nge-API noma isikhombimsebenzisi sewebhu (isibonelo, nge-Open WebUI). Lokhu akubandakanyi izinsiza kuphela kodwa futhi nesistimu yokusebenza kanye nezimvume.

Kuziqondiso ezivela kubahlinzeki abanjengoHostinger, izidingo ezilandelayo ezincane ziyanconywa ze-VPS ehloselwe i-Ollama:

• I-RAM: ubuncane be-16 GB ukuze amamodeli amancane/aphakathi angawugqibi kakhulu uhlelo.
• I-CPU: 4-8 vCoreskuye ngobukhulu bamamodeli kanye nenani labasebenzisi abasebenzisa kanyekanye.
• Isitoreji: okungenani i-12 GBKodwa-ke, empeleni kuyalulekwa ukuthi uhlose okungaphezulu (50-100 GB) uma uzozama amamodeli amaningana.
• Isistimu yokusebenza: ngaphezu kwakho konke Linux, ngokukhetha kwe Ubuntu 22.04 noma ngaphezulu, noma i-Debian ezinzile yakamuva.
• Ukufinyelela kwempande noma izimvume ze-sudo ukuze ufake ukuncika, lungiselela i-systemd, njll.

Uma i-VPS yakho ifaka i-NVIDIA GPU, kuzodingeka ufake futhi ulungiselele i-CUDA noma i-NVIDIA container toolkit uma usebenzisa i-Docker. Nge-AMD, i-ROCm ivame ukusetshenziswa ku-Linux, kanye nabashayeli be-Adrenalin abafanele ku-Windows. Ezindaweni ezingenayo i-GPU, iseva izoncika ku-CPU kanye ne-RAM, ngakho unganciphisi lezo zinsiza; ungayiphatha futhi ukude ngedeskithophu ekude uma udinga isikhombikubona sesithombe.

Izimo ezithile zehadiwe nokuthi yimaphi amamodeli okufanele uwasebenzise

Ukuqinisekisa ukuthi konke okungenhla akuhlali kungokwemfundiso nje, kungaba usizo ukubheka ezinye izinhlanganisela zehadiwe ezijwayelekile nokuthi yiziphi izinhlobo zamamodeli ezifanelana kahle esimweni ngasinye kusetshenziswa i-Ollama.

Ikhompyutha yedeskithophu noma yelaptop ephakathi nendawo

Ake sicabange ngeqembu elijwayelekile :

• i5 noma i-Ryzen 5 CPU kusukela eminyakeni embalwa edlule (ama-core angu-4-6).
• I-16 GB ye-RAM.
• I-GPU engu-4 GB ehlanganisiwe noma ezinikele.
• I-SSD engu-512GB.

Kulesi simo, into ehlakaniphile okufanele uyenze ukuhlose :

• Amamodeli e-1B-3B alinganisiwe (Gemma 2B, Phi-4 Mini, Llama 3.x 1B) ukuze kube nokugeleza okuphezulu.
• Amamodeli angu-7B ku-Q4 uma wamukela isikhathi eside sokuphendula.
• Usebenzisa i-Ollama nge-terminal futhi, uma ufuna i-web interface, Vula i-WebUI ngokucophelela ukuze ungagcwalisi i-RAM ngokweqile.

Ungaba nomsizi wakho wombhalo wendawo, wenze izifinyezo, wenze ukuhlaziya kanye nemisebenzi elula yokuhlela, kodwa akuyona indawo efanelekile yamamodeli angu-13B nangaphezulu.

Imishini ephakathi kuya kwephezulu egxile ku-AI yendawo

Lapha sikhuluma ngohlobo lwe-PC :

• I-CPU yesimanje ye-i7/i9 noma i-Ryzen 7/9, ama-core angu-8-16.
• I-32 GB ye-RAM.
• I-GPU ene-VRAM engu-12-24 GB (RTX 4070/4080, 3090, 4090, okulingana ne-AMD noma okufanayo).
• I-SSD engu-1-2 TB.

Lokhu kulungiselelwa kwandisa kakhulu amathuba amaningi :

• onobuhle 7B-13B ku-Q4/Q5 ngengxoxo, ikhodi, ukuhlaziywa kwedatha… ngezikhathi ezinhle kakhulu zokuphendula.
• Amamodeli angu-30B futhi abanye Isilinganiso esingu-70B uma wamukela ukubambezeleka okwengeziwe.
• Amamodeli we umbono/i-OCR ngobukhulu obuphakathi kusetshenziswa kakhulu i-GPU.

Luhlobo lomshini ongawusebenzisa ukusetha indawo ye-AI yendawo engathi sína, enamamodeli amaningi, isikhombikubona sewebhu, ukuhlanganiswa nge-REST API, kanye nomsebenzi wobungcweti ngaphandle kokuthembela kumasevisi angaphandle.

Iseva noma isiteshi sendawo "sesilo"

Engxenyeni ephezulu kukhona izindawo ezine:

• Ama-GPU amaningana ane-VRAM engu-24-48 GB ngayinye, noma eyodwa ephezulu.
• 64-128 GB RAM.
• Ama-CPU anama-core amaningi, njengezinhlobo zakamuva ze-Threadripper noma ze-Xeon.

Yilapho amamodeli amakhulu (>70B, MoE, vision heavy, njll.) eqala khona ukuba ngokoqobo, ngisho nangabasebenzisi abaningi ngesikhathi esisodwa noma ukuhlanganiswa okuyinkimbinkimbi. Kusobala ukuthi kuyisimo esibiza kakhulu, kodwa futhi sikuvumela ukuthi ube namakhono afana namanye ama-API ezentengiselwano, ngokulawula idatha okuphelele ngaphakathi kwengqalasizinda yakho.

Amathiphu awusizo okuthola okuningi kwihadiwe yakho ye-Ollama

Ngaphandle kokuthenga i-RAM eyengeziwe noma i-GPU engcono, kunezindlela eziningana ezikusiza ukuthi uthole okuningi kulokho osuvele unakho futhi ugweme izimanga lapho usebenzisa amamodeli amakhulu nge-Ollama.

Okokuqala, kubalulekile ukukhetha imodeli efanele ngokusekelwe ekusetshenzisweni kwakho okuhlosiwe : akunangqondo ukusebenzisa i-70B ukubhala ama-imeyili alula uma i-7B elungiswe kahle yanele ngokuphelele. Ngokufanayo, i-30B ayinangqondo uma i-GPU yakho ine-6GB ye-VRAM kuphela; i-7B izoba ukukhetha okungcono ku-Q4.

Esinye isinyathelo esibalulekile ukuzama amapharamitha esikhathi sokusebenza (izinga lokushisa, i-num_ctx, i-num_predict, njll.), kungaba ku-Modelfile noma nge-CLI/API. Ukusebenzisa izimo ezinkulu kakhulu ( i-num_ctx ye-32k noma ngaphezulu ) nge-RAM encane noma i-VRAM kuzonciphisa ijubane lesistimu yonke ngaphandle kokunikeza izinzuzo eziningi ezimweni eziningi.

Kuyanconywa futhi qapha ukuthi yimaphi amamodeli alayishwayo nokuthi kuyiphi iprosesa usebenzisa ollama psLapho uzobona ukuthi imodeli isebenza ngempela ku-GPU noma ku-CPU, nokuthi ilayishe usayizi ongakanani. Lungisa i-variable OLLAMA_KEEP_ALIVE Kusiza amamodeli ukuthi akhulule inkumbulo uma engasetshenziswa, ngaleyo ndlela akhulule izinsiza.

Okokugcina, khumbula ukuthi i-quantization iyisihlobo sakho : ukudala izinhlobo ze-Q4_K_M noma ze-Q5_K_M zemodeli yokuqala ye-FP16 kukuvumela ukuthi usebenzise ihadiwe ephansi kakhulu ngokulahlekelwa yikhwalithi evame ukungabonakali ukusetshenziswa kwangempela.

Ngemva kokucabangela konke lokhu, okucacile ukuthi i-Ollama akuyona ingxenye edingayo; amamodeli ayi- . Ukuqonda ukuthi usayizi, ukulinganisa, i-RAM, kanye ne-VRAM kuhlobene kanjani kukuvumela ukuthi ukhethe i-hardware efanele kanye nenhlanganisela ye-LLM yezidingo zakho: kusukela ku-laptop ene-16 GB esebenzisa i-7B elula kuya endaweni yokusebenza ene-24 GB GPU ephethe umbono oqinile kanye namamodeli e-OCR. Ngokulungisa ngokucophelela okulindelwe kanye namapharamitha, kungenzeka ngokuphelele ukuba ne-AI enamandla, yangasese esebenza emshinini wakho ngaphandle kwemali yanyanga zonke.