- I-vLLM ivelele ngokuguquguquka kwayo, ukuhlanganiswa kalula ne-Hugging Face, kanye nesistimu yememori esebenza kahle esebenzisa i-PagedAttention.
- I-TensorRT-LLM iyindlela engcono kakhulu ekusebenzeni okuluhlaza kubasebenzisi be-NVIDIA, yize kuyinkimbinkimbi kakhulu ukuyilungiselela futhi ingaguquguquki kakhulu.
- Kuma-benchmark aphezulu, izinjini ezifana ne-SGLang kanye ne-LMDeploy zisebenza kahle kakhulu kune-vLLM ngesivinini ngenxa yokulungiswa kwe-C++ okungokwemvelo kanye ne-overhead ephansi yokuhlelwa kwe-orchestrate.
Uma singena ezweni lokusebenzisa amamodeli olimi ngezinga elikhulu, enye yezinto ezivame kakhulu ukuthi singaphetha kanjani ngokushesha ngaphandle kokukhipha imali yethu. Ekuqaleni, ukuhambisa imodeli kusuka elabhorethri iye endaweni yangempela yokukhiqiza kuyinselele enkulu, ngoba ukusebenza kahle engqalasizinda ye-AI yikho okwenza umehluko phakathi kwensizakalo endiza kanye nensizakalo ephahlazeka ngaphansi kwethrafikhi enkulu.
Kulesi simo, kuvele amathuluzi ahlukahlukene aklanyelwe ukwandisa ukusebenza kwe-GPU, kanti i-vLLM ingenye yezithandwa kakhulu, yize incintisana ngqo nezixazululo ezivaliwe kakhulu noma ezikhethekile kakhulu. Ukuze ugweme ukukhetha ngokunganaki, kubalulekile ukuqonda ukuthi ukukhethwa kwenjini yokuphetha kuncike ngokuphelele ekutheni sibeka phambili ukulula kokufakwa, ukuhambisana nehadiwe ehlukahlukene, noma ukusebenza okungaxutshwe nalutho, okungaxutshwe nalutho.
i-vLLM: Indinganiso eguquguqukayo nevulekile
I-vLLM izisungule njengethuluzi elibalulekile ngenxa yokugxila kwayo ekuguquguqukeni. Amandla ayo amakhulu uhlelo lwe -PagedAttention , oluphatha inkumbulo ye-GPU ngendlela efanayo nenkumbulo ebonakalayo yezinhlelo zokusebenza. Lokhu kuvimbela isikhala esichithwa ngamathokheni angasebenzi, okuvumela amafasitela amakhulu omongo kanye nokucutshungulwa kwezicelo eziningi ngasikhathi sinye ngaphandle kokuphahlazeka kwesistimu.
- Izinzuzo eziyinhloko: Ivelele ngokuhlanganiswa kwayo ngqo ne-Hugging Face, okusiza kakhulu ukuhamba komsebenzi, kanye nekhono layo lokuphatha amaqoqo amakhulu edatha ngokusebenza kahle okumangalisayo.
- Amaphuzu abuthakathaka: Nakuba inamandla kakhulu, ingase ingafinyeleli ukusebenza okuphezulu kwamathuluzi aklanyelwe i-NVIDIA kuphela, futhi ukwesekwa kwayo kwe-CPU kusalokhu kulinganiselwe.
I-TensorRT-LLM: Izikhali ezisindayo ze-NVIDIA
Uma ufuna ukuvula wonke amandla okugcina avela ekhadini le-NVIDIA, i-TensorRT-LLM iyisinqumo esinengqondo. Akuyona injini yenhloso evamile, kodwa umtapo wolwazi okhethekile osebenzisa ukulungiswa kwegrafu ye-CUDA kanye nama-core ahlanganisiwe ukusheshisa ukubala. Yakhelwe ukuhlanganiswa kalula ne-Triton Inference Server kanye ne-NeMo, iyitshe eliyigugu lezindawo zebhizinisi esezivele zicwiliswe ohlelweni lwe-NVIDIA.
Ngokungafani ne-vLLM, i-TensorRT-LLM igxile ekwenzeni ngcono izinga le-kernel , isekela amafomethi e-quantization njenge-FP8 kanye ne-INT4. Kodwa-ke, la mandla afika ngentengo: ijika lokufunda liyinkimbinkimbi kakhulu, ukucushwa kunzima kakhulu, futhi kusobala ukuthi likhawulelwe kuphela kwihadiwe ye-NVIDIA , ngaphandle kwe-AMD nanoma yiziphi ezinye izindlela.
Ukungqubuzana kokusebenza: i-vLLM ngokumelene ne-LMDeploy kanye ne-SGLang
Ukuze uqonde ukuthi i-vLLM imi kuphi ngempela, kuyasiza ukuyiqhathanisa nezinye izinsimbi ezisindayo njenge-SGLang ne-LMDeploy kuhadiwe yanamuhla, ikakhulukazi i-NVIDIA H100. Ekuhlolweni kokusebenza okungahleliwe (amathokheni ngomzuzwana), kubonakale igebe elikhulu lokwakha . Ngenkathi i-SGLang ne-LMDeploy zifinyelela izibalo eziseduze ne-16.200 tok/s, i-vLLM, ngisho ne-FlashInfer, ijikeleza cishe i-12.500 tok/s.
Lo mehluko ongu-29% awubangelwa ukubala kwezibalo, kodwa kunalokho ngenxa yokuhlelwa kwe-overhead . I-SGLang isebenzisa i-RadixAttention ukuphatha amaphethini ayinkimbinkimbi, kanti i-LMDeploy ithembele ku-backend emsulwa ye-C++ (i-TurboMind) esusa umthwalo we-Python. I-vLLM, emzamweni wayo wokuhambisana nezakhiwo eziningi futhi inikeze ama-plugin aguquguqukayo, idela isivinini esithile ukuze ilondoloze ukuguquguquka.
Umhlahlandlela osheshayo wokukhetha injini yakho yokuqagela
Akukho sixazululo esisodwa, kodwa kukhona ithuluzi lazo zonke izimo. Uma udinga ukwenza i-prototype ngokushesha futhi ufuna imodeli yakho isebenze namuhla ngokufakwa okulula kwe-pip, i-vLLM ingumngani wakho omkhulu ngenxa ye-ecosystem yayo kanye nokusekelwa kwayo.
Uma uneqembu elizinikele lokuqagela kanye nethimba lobuchwepheshe elikwazi ukusingatha ukuncika okuyinkimbinkimbi, futhi ufuna ukusebenza okuphezulu , i-SGLang iyindlela okufanele uhambe ngayo. Kulabo abafuna ibhalansi phakathi kokukhiqiza okuzinzile nokusebenza kwe-H100 ngaphandle kokufakwa okuyinkimbinkimbi, i-LMDeploy iyindlela eqinile.
Izinto ezicatshangelwayo zobuchwepheshe kanye nokusetshenziswa kwazo
Ngesikhathi sokusetshenziswa, kunemininingwane engabangela izinkinga. Isibonelo, ukwaba u-95% wememori ye-GPU kuvame ukuholela emaphutheni esistimu lapho kuthwetshulwa igrafu ye-CUDA. Kungcono ukusebenzisa umkhawulo wokuphepha ongu-80% ukuqinisekisa ukuzinza.
Ukuze kube lula izinto, amapulatifomu afana neNorthflank akuvumela ukuthi usebenzise lezi zinjini ezitsheni ezine- GPU acceleration ngaphandle kokusetha ingqalasizinda ngesandla. Lokhu kwenza kube nokwenzeka ukuhlola i-vLLM kanye ne-TensorRT-LLM ngesikhathi esifanayo ukuze kuqhathaniswe ukuthi iyiphi esebenza kahle ngaphambi kokukala.
Isinqumo sokugcina sincike ekutholeni ibhalansi phakathi kokulula kokufakwa kanye nokwenza ngcono ihadiwe. I-vLLM ivelele ngokuhambisana kwayo kanye nobulula bayo , kuyilapho i-TensorRT-LLM kanye ne-SGLang ziphula izithiyo zokusebenza ezingqalasizinda ezisezingeni eliphezulu, zandisa ukuhlangana kwememori kanye nokusetshenziswa kwe-Tensor Cores ukukhipha inani elikhulu kuwo wonke amahora okusebenzisa ikhompyutha.
