large language models on Sadman Kabir Soumik

large language models on Sadman Kabir Soumik https://blog.sksoumik.com/tags/large-language-models/ Recent content in large language models on Sadman Kabir Soumik Hugo -- gohugo.io en Copyright © 2022, Sadman Kabir Soumik Fri, 15 May 2026 00:00:00 +0000 How vLLM Works for Serving Large Language Models https://blog.sksoumik.com/artificial-intelligence/how_vllm_works_for_serving_llms/ Fri, 15 May 2026 00:00:00 +0000 https://blog.sksoumik.com/artificial-intelligence/how_vllm_works_for_serving_llms/ You have downloaded an open source Large Language Model. Maybe it is Llama, Qwen, Gemma, or another model. You load it on an NVIDIA GPU and send a prompt: 1What is the capital of Bangladesh? The model answers: 1The capital of Bangladesh is Dhaka. Everything works. But then 100 users start sending requests at the same time. Suddenly things become harder. Some requests wait. GPU memory fills up. Your expensive GPU may not be used efficiently. How to Make LLM Inference Faster https://blog.sksoumik.com/artificial-intelligence/how-to-make-llm-inference-faster/ Sun, 12 Apr 2026 00:00:00 +0000 https://blog.sksoumik.com/artificial-intelligence/how-to-make-llm-inference-faster/ Running a Large Language Model is easy. Running it fast, at scale, and without wasting expensive GPUs is much harder. Imagine you deploy an open source LLM on an NVIDIA A100 GPU. Your API works, but users sometimes wait several seconds before they see the first word. When many users arrive at the same time, things get even slower. Buying more GPUs is one solution. But it is often not the best first solution.