https://blog.sksoumik.com/categories/system-design/ Recent content in system design on Sadman Kabir Soumik Hugo -- gohugo.io en Copyright © 2022, Sadman Kabir Soumik Wed, 12 Aug 2026 18:00:00 +0000 https://blog.sksoumik.com/artificial-intelligence/memory-systems-agentic-ai/ Wed, 12 Aug 2026 18:00:00 +0000 https://blog.sksoumik.com/artificial-intelligence/memory-systems-agentic-ai/ Large Language Models are good at understanding text and generating answers. But there is an important problem. By default, an LLM does not truly remember everything that happened before. Imagine you talk to an AI assistant today and tell it: I prefer short answers and I usually work with Python. Tomorrow, you start a new conversation. If the system has no memory, the assistant may know nothing about that previous conversation. https://blog.sksoumik.com/artificial-intelligence/how_vllm_works_for_serving_llms/ Fri, 15 May 2026 00:00:00 +0000 https://blog.sksoumik.com/artificial-intelligence/how_vllm_works_for_serving_llms/ You have downloaded an open source Large Language Model. Maybe it is Llama, Qwen, Gemma, or another model. You load it on an NVIDIA GPU and send a prompt: 1What is the capital of Bangladesh? The model answers: 1The capital of Bangladesh is Dhaka. Everything works. But then 100 users start sending requests at the same time. Suddenly things become harder. Some requests wait. GPU memory fills up. Your expensive GPU may not be used efficiently. https://blog.sksoumik.com/artificial-intelligence/how-to-make-llm-inference-faster/ Sun, 12 Apr 2026 00:00:00 +0000 https://blog.sksoumik.com/artificial-intelligence/how-to-make-llm-inference-faster/ Running a Large Language Model is easy. Running it fast, at scale, and without wasting expensive GPUs is much harder. Imagine you deploy an open source LLM on an NVIDIA A100 GPU. Your API works, but users sometimes wait several seconds before they see the first word. When many users arrive at the same time, things get even slower. Buying more GPUs is one solution. But it is often not the best first solution. https://blog.sksoumik.com/artificial-intelligence/building-multi-agent-ai-systems/ Thu, 15 Jan 2026 18:00:00 +0000 https://blog.sksoumik.com/artificial-intelligence/building-multi-agent-ai-systems/ Building one AI agent is relatively easy. You give an LLM some instructions, connect a few tools, and let it perform a task. For example: 1User 2 ↓ 3Agent 4 ↓ 5LLM 6 ↓ 7Tools 8 ↓ 9Answer But imagine the task becomes much larger. You want an AI system that can: Understand a complex request Break the request into smaller tasks Search the web Read internal documents Write code Analyze data Ask another agent for help Remember previous work Check its own results Ask a human before taking risky actions Continue working even when one step fails One agent can try to do everything. https://blog.sksoumik.com/artificial-intelligence/different-types-of-llms-architectures/ Tue, 18 Mar 2025 00:00:00 +0000 https://blog.sksoumik.com/artificial-intelligence/different-types-of-llms-architectures/ Let's first break it down: what exactly are large language models (LLMs), why do we call them 'large,' and how are they different from other types of language models? An LLM is a machine learning model trained on massive amounts of text using transformer-based architectures (or their variations). These models can generate, understand, and process human-like text, making them useful for tasks like translation, summarization, reasoning, coding etc,. How is an LLM different from other language models? https://blog.sksoumik.com/artificial-intelligence/airflow-machine-learning-pipeline/ Fri, 17 Feb 2023 00:00:00 +0000 https://blog.sksoumik.com/artificial-intelligence/airflow-machine-learning-pipeline/ Author: Sadman Kabir Soumik Let's first understand what's MLOps. What is MLOps? MLOps (Machine Learning Operations) is a set of practices and tools used to manage the entire lifecycle of machine learning models. MLOps includes everything from data preparation and model training to deployment, monitoring, and ongoing maintenance. The primary goal of MLOps is to create a streamlined and automated process for deploying and managing machine learning models at scale. https://blog.sksoumik.com/software-engineering/system-design-fundamentals-software-engineering/ Wed, 20 Jul 2022 00:00:00 +0000 https://blog.sksoumik.com/software-engineering/system-design-fundamentals-software-engineering/ What is System Design in Software Engineering? System design in software engineering is defining the structure, components, connections, and information for a system that meets specific needs. It is a vital step in software development as it outlines how the system will work and be structured. A team of developers usually performs this process with input from business analysts and end-users. Process of Designing a System To create a good software system, we need to follow these steps: https://blog.sksoumik.com/cloud-computing/automate-gcp-compute-engine-vm-instance/ Sat, 12 Feb 2022 00:00:00 +0000 https://blog.sksoumik.com/cloud-computing/automate-gcp-compute-engine-vm-instance/ Suppose you have a script in the Google Cloud’s VM Instance that needs to run every day/week/month at a particular time ( e.g., a data pipeline for Machine Learning model training, data crawling, etc.). In that case, this article will guide you through automating the task. Different ways you can automate a script on GCP Your script/program can be in any language, but let’s assume we are trying to automate a Python script for simplicity. https://blog.sksoumik.com/artificial-intelligence/designing-large-scale-high-performance-software-systems/ Fri, 12 Mar 2021 00:00:00 +0000 https://blog.sksoumik.com/artificial-intelligence/designing-large-scale-high-performance-software-systems/ Start With A Simple Architecture To begin building an app, we will start from the beginning. We will create a basic app with some users. The easiest way to do this is to put the whole app on one server. This is a common way to start. The app and any API's will run on a server like Apache or Tomcat. We will also use a database like Oracle or MySQL. https://blog.sksoumik.com/software-engineering/choosing-right-web-api-rest-grpc/ Fri, 10 Apr 2020 00:00:00 +0000 https://blog.sksoumik.com/software-engineering/choosing-right-web-api-rest-grpc/ REST (Representational State Transfer) and gRPC are two different techniques for creating web APIs (Application Programming Interfaces). What is REST API? A popular architectural design for developing web APIs is REST. It is built on the HTTP (HyperText Transfer Protocol) and operates on resources defined by URLs using common HTTP methods including GET, POST, PUT, and DELETE. REST APIs are simple to use with a variety of programming languages and tools since they exchange data using the JSON (JavaScript Object Notation) format. https://blog.sksoumik.com/software-engineering/dependency-injection-real-world-examples-advantages/ Sat, 10 Feb 2018 00:00:00 +0000 https://blog.sksoumik.com/software-engineering/dependency-injection-real-world-examples-advantages/ Dependency injection is a design pattern that is widely used in software development to promote good software design and make code more flexible and easier to test. In this blog post, we will take a closer look at what dependency injection is, how it works, and its advantages. What is Dependency Injection? Dependency injection is a technique that allows an object to receive its dependencies (other objects it needs to function) from the outside, rather than creating them itself.