{"componentChunkName":"component---src-templates-tag-page-js","path":"/tags/hpc/","result":{"data":{"site":{"siteMetadata":{"title":"M.Hassan Ahmed","author":"Hassan11196"}},"allMarkdownRemark":{"totalCount":1,"edges":[{"node":{"excerpt":"At some point the model does not fit. You try to load a 70B-parameter checkpoint onto a single 80GB card. The weights alone want around…","fields":{"slug":"/2026-09-28-tensor-vs-pipeline-parallelism-llm/"},"frontmatter":{"date":"2026-09-28T00:00:00.000Z","title":"Splitting an LLM Across GPUs: Tensor vs Pipeline","description":"A model too big for one GPU has to be split. How tensor and pipeline parallelism divide an LLM, what each costs, and when to reach for which.","tags":["LLM","GPU","Inference","HPC","DevOps"],"thumbnail":{"childImageSharp":{"fluid":{"base64":"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAABQAAAALCAIAAADwazoUAAAACXBIWXMAAAsSAAALEgHS3X78AAAB8UlEQVQozy1Ra5OaMBTlU8szCbo8DIEECBhBrYovkHVqt7t1q9ap0///WxrUmTOZe2/OOZOcqzDaPhCzdjG/bNa3xfzc1P/ms0u1+CPbqrrKYr36O5udl9VVcqKweUgUFeIHoJsko5WY7fh4U1b7bLLNxpti/pqNt3IymrWyHX5r8kkNnPghUUw7Mnu0g011GKkg1GCoo8iQLYruxbOW8w4gfPJ7VDFQCNwMeDnhiyhfJkVNxQr6ueWk0tdARJoCN0cDIW+ZWCdlHeaV5XJJUHRIoJtjsqasTdPvcbqPk31Im4BsdEQ0gOWJ/BEJaxa3PD3cCa8R3fl4oRiQWG4m+FGknyI58vgnZ29ldi74yXqhfT+3PY680SS/ZvH7KP3krCNMxS1lB8UAgeXwNHljdI+DDSabAV4m8SFL36HDMJu6pICeGPJjENYB2QZk7eMqTz8o3SnyYcDJpM14eJ6Xt3J4mopLkf3m7IeGMHJiHQXIEyL9NRbneXEr89O4I5wiUiua5cs/Q18QLsNoqWhItoYDYTpcA4MvutO5u0NJiPJtUrRMNAFfAW/YBaZZ3j1SuRUiYdih1WfgJQb92ESBAbGJyLPtUXmro1Bm/FiEopruV8ORUE1HNRzNdE07MBE2kJQFHWzyLBDWLFe9MzuJ6fwHDq9T9VH8IXwAAAAASUVORK5CYII=","aspectRatio":1.899441340782123,"src":"/static/91cf0860c00031917c8d46f337d29dee/40a76/hero.png","srcSet":"/static/91cf0860c00031917c8d46f337d29dee/c972b/hero.png 340w,\n/static/91cf0860c00031917c8d46f337d29dee/27625/hero.png 680w,\n/static/91cf0860c00031917c8d46f337d29dee/40a76/hero.png 1360w,\n/static/91cf0860c00031917c8d46f337d29dee/ed396/hero.png 2000w","sizes":"(max-width: 1360px) 100vw, 1360px"}}}}}}]}},"pageContext":{"tag":"HPC"}},"staticQueryHashes":["32046230"]}