Share a single GPU across multiple workloads using Nvidia MIG and TimeSlicing for cost-efficient deployments.
Fractional GPUs enable us to allocate multiple workloads to a single GPU which can be useful in the following scenarios:
The workloads take around 2-3 GB of VRAM - so you can allocate multiple replicas of this workload on a single GPU which has around 16GB of VRAM or more.
Each workload has sparse traffic and its not able to max out the GPU usage.
There are two ways to use fractional GPUs:
TimeSlicing: In this approach, you can slice a GPU into a few fixed number of fractional parts and then choose a fraction of GPU for this workload. For e.g, we can decide to divide a GPU into 10 slices and then request 3 slices for one workload, 5 slices for workload2 and 2 slices for the workload3. This means that worload1 will use 0.3 GPU (compute + 30% of VRAM), workload2 will use 0.5 GPU and workload3 will use 0.2 GPU. However, timeslicing is only used for scheduling the workloads on the same GPU - it doesn’t mean any actual isolation on the machine. For example, if the GPU machine has 16GB VRAM, its on the user to actually make sure that workload1 takes less than 4.8GB VRAM, workload2 takes less than 8 GB of VRAM and workload3 takes less than 3.2 GB of VRAM. If one workload starts taking more memory suddenly, it can lead to crashing of the other processes. The compute is also shared but one workload can go upto using the complete GPU if the other workloads are idle - its basically context-switching among the three workloads. You can read about this more here.
MIG (Multi-Instance GPUs): This is a feature provided by Nvidia only on the A100, H100 and newer generation GPUs - it doesn’t work on other GPUs. We can divide the GPUs into a fixed number of configurable parts as mentioned in the table below. The workloads can choose one of the slices and they will get compute and memory isolation. The instances are not exactly the complete fractions of the GPU, but more discrete units as mentioned in the table below. For e.g. let’s say we divide one A100 GPU of 40GB into 7 parts - then we can place 7 workloads each using around 1/7 GPU and 5GB VRAM. Please note that we cannot simply provide 2 slices to one workload in this case and expect it to get 2/7 GPU and 10 GB VRAM. Each workload can only get one slice in this case. Here are the different MIG profiles:
A brief comparison chart between TimeSlicing and MIG is as follows:
Feature
TimeSlicing
MIG (Multi-Instance GPU)
GPU Support
Works on most GPUs.
Only supported on NVIDIA A100 and H100 GPUs.
Isolation
No real isolation. User is responsible for memory management. Potential for crashes if one workload exceeds its allocated memory.
Strong Isolation. Compute and memory are isolated between instances. Guaranteed resource allocation.
Resource Allocation
Divides GPU into fractional parts (e.g., 0.3, 0.5, 0.2). Workloads can use these fractional parts.
Divides GPU into pre-defined, discrete instance types (as per NVIDIA’s configurations). Workloads are assigned entire instances.
VRAM Management
User-managed. VRAM allocation is not enforced by the hardware.
Hardware-enforced. Each instance has dedicated VRAM.
Compute Sharing
Compute is shared via context-switching. Workloads can potentially use the entire GPU when others are idle.
Compute is partitioned and isolated to each instance. No sharing of compute resources beyond the instance’s allocation.
Flexibility
More flexible in terms of resource allocation fractions (e.g., can request 0.3, 0.5, etc.).
Limited to NVIDIA’s pre-defined instance types. Less flexible in terms of fine-grained resource requests.
Create a nodepool and pass the mig_profile in accelerator by passing gpu_partition_size=1g.5gb[OR one of the allowed values for MIG profile values mentioned above]
Shell
gcloud container node-pools create a100-40-mig-1g5gb \ INT ✘ --project=<enter your project name> \ --region=<enter your region> \ --cluster=<enter your cluster name here> \ --machine-type=a2-highgpu-1g \ --accelerator type=nvidia-tesla-a100,count=1,gpu-partition-size=1g.5gb \ --enable-autoscaling \ --total-min-nodes 0 \ --total-max-nodes 4 \ --min-provision-nodes 0 \ --num-nodes 0
On AWS EKS you do not create managed nodegroups for MIG. TrueFoundry discovers fractional GPUs from Karpenter NodePools + NodeOverlays. Enable MIG by updating three things:
Use the latest tfy-gpu-operator and tfy-karpenter-config chart versions.1. Configure MIG profiles in tfy-gpu-operatorEdit the tfy-gpu-operator Helm application and merge the following into values (add more mig-configs entries for other profiles you need):
The config name under mig-configs (for example all-1g.24gb) must match the nvidia.com/mig.config label on the NodePool in the next step. Deploy the updated Helm application.2. Add a Karpenter NodePool + NodeOverlay via tfy-karpenter-configEdit the tfy-karpenter-config Helm application. Under Kustomize → Additional Manifests, add both resources. Example: RTX PRO 6000 on g7e.2xlarge with MIG profile 1g.24gb (4 partitions per GPU):
NodeOverlay must set truefoundry.com/fractional-gpu: mig
Capacity key must be nvidia.com/mig-<profile> (for example nvidia.com/mig-1g.24gb)
Capacity value must equal (MIG partitions per GPU) × (GPU count on the instance). g7e.2xlarge has 1 GPU and 1g.24gb has 4 partitions → 4. p5.4xlarge has 1 GPU and 1g.10gb has 7 partitions → 7. p5.48xlarge has 8 GPUs and 1g.10gb has 7 partitions → 56
NodePool must resolve to exactly one GPU instance type
nvidia.com/mig.config on the NodePool must match a MIG config name from tfy-gpu-operator
Deploy the updated tfy-karpenter-config application.3. Sync ClusterGo to Platform → Clusters and click Sync on the cluster. Sync also runs periodically; a manual sync makes the new NodePools show up sooner.
Sync Cluster
After sync, the fractional GPU NodePools appear under the cluster’s Node Pools:
Fractional GPU NodePools after sync
2
Deploy your workload on the MIG nodepool
Once you have created the nodepool, you will be able to see the MIG nodepool in the available nodepools in Resources section in deployment.
It might take a few minutes for the newly created nodepool to be visible on the TrueFoundry UI. You can force sync the nodepools by going to Platform → Clusters → Sync.
To deploy a workload that utilizes fractional GPU, start deploying your service/job on truefoundry and in the “Resources” section, select nodepool selector.
You can now see the Fractional GPUs on the UI which you can select (as shown below)
Selecting a MIG GPU in Resources
Deploying on Fractional GPUs via Timeslicing
1
Create a Nodepool with Timeslicing enabled
Azure
GCP
AWS
Create a Nodepool with device-plugin.config pointing to the correct time-slicing config with Azure CLI.
On AWS EKS you do not create managed nodegroups for timeslicing. TrueFoundry discovers fractional GPUs from Karpenter NodePools + NodeOverlays. Enable timeslicing by updating three things:
Use the latest tfy-gpu-operator and tfy-karpenter-config chart versions.1. Configure timeslicing in tfy-gpu-operatorEdit the tfy-gpu-operator Helm application and merge the following into values. This creates a device-plugin profile named time-sliced-10 (10 slices per GPU):
The profile name (time-sliced-10) must match the nvidia.com/device-plugin.config label on the NodePool in the next step. Deploy the updated Helm application.2. Add a Karpenter NodePool + NodeOverlay via tfy-karpenter-configEdit the tfy-karpenter-config Helm application. Under Kustomize → Additional Manifests, add both resources. Example: L4 on g6.xlarge with 10 timeslices:
NodeOverlay must set truefoundry.com/fractional-gpu: timeslice
Capacity key must be nvidia.com/gpu.shared
Capacity value must be divisible by the GPU count on the instance. For g6.xlarge (1 GPU) with 10 slices → 10. For a 2-GPU instance with 10 slices per GPU → 20
NodePool must set nvidia.com/device-plugin.config to a profile name from tfy-gpu-operator (for example time-sliced-10)
NodePool must resolve to exactly one GPU instance type
Deploy the updated tfy-karpenter-config application.3. Sync ClusterGo to Platform → Clusters and click Sync on the cluster. Sync also runs periodically; a manual sync makes the new NodePools show up sooner.
Sync Cluster
2
Deploy your workload on the Timeslicing nodepool
Once you have created the nodepool, you will be able to see the timesliced nodepool in the available nodepools in Resources section in deployment.
It might take a few minutes for the newly created nodepool to be visible on the TrueFoundry UI. You can force sync the nodepools by going to Platform → Clusters → Sync.
To deploy a workload that utilizes fractional GPU, start deploying your service/job on TrueFoundry and in the “Resources” section, select nodepool selector.
You can now see the timesliced GPUs on the UI which you can select (as shown below)
Selecting a timesliced GPU in Resources
Was this page helpful?
{"docsConfig":{"theme":"mint","$schema":"https://mintlify.com/docs.json","name":"TrueFoundry Docs","description":"TrueFoundry documentation — enterprise-grade AI Gateway combining LLM, MCP, and Agent Gateways to connect, monitor, and govern agentic AI applications across providers from a unified control plane.","colors":{"primary":"#4f46e5","light":"#4f46e5","dark":"#4f46e5"},"logo":{"light":"https://mintcdn.com/truefoundry/e2uUimEwX30qkWxc/logo/light.svg?fit=max\u0026auto=format\u0026n=e2uUimEwX30qkWxc\u0026q=85\u0026s=6068d83664aeda0239bec2422657dd20","dark":"https://mintcdn.com/truefoundry/e2uUimEwX30qkWxc/logo/dark.svg?fit=max\u0026auto=format\u0026n=e2uUimEwX30qkWxc\u0026q=85\u0026s=c1cb26554a546d84500b40f7057bece8"},"favicon":"https://mintcdn.com/truefoundry/6bbtTt95c1iCcyph/favicon.svg?fit=max\u0026auto=format\u0026n=6bbtTt95c1iCcyph\u0026q=85\u0026s=f55bde34214b38034126d05b08ea16c7","appearance":{"default":"light"},"background":{"decoration":"grid"},"navbar":{"links":[{"href":"https://www.truefoundry.com/book-demo","label":"Book Demo"},{"href":"https://www.truefoundry.com/blog","label":"Blog"}],"primary":{"type":"button","label":"Sign Up","href":"https://www.truefoundry.com/register"}},"footer":{"socials":{"x":"https://twitter.com/truefoundry","linkedin":"https://in.linkedin.com/company/truefoundry"}},"seo":{"metatags":{"og:type":"website","og:site_name":"TrueFoundry Docs","og:image":"https://assets.production.truefoundry.com/logo.svg","og:description":"TrueFoundry offers an enterprise-grade AI Gateway combining LLM, MCP, and Agent Gateways—empowering businesses to connect, monitor, and govern agentic AI applications across providers from a unified control plane.","description":"TrueFoundry offers an enterprise-grade AI Gateway combining LLM, MCP, and Agent Gateways—empowering businesses to connect, monitor, and govern agentic AI applications across providers from a unified control plane."}},"icons":{"library":"fontawesome"},"styling":{"codeblocks":"dark","latex":true},"contextual":{"options":["copy","view","chatgpt","claude","mcp","cursor","vscode"],"display":"header"},"navigation":{"pages":[]}},"deployment":{"deploymentMetadataValue":{"subdomain":"truefoundry","hasAudioNarration":false,"actualSubdomain":"truefoundry","customDomains":["www.truefoundry.com"],"gitSource":{"type":"github","owner":"truefoundry","repo":"docs-mintlify","deployBranch":"main","contentDirectory":"","isPrivate":true,"mountPath":"","label":""},"feedback":{"thumbs":true},"entitlements":{"AI_CHAT":{"status":"ENABLED"},"WEB_MCP":{"status":"ENABLED"},"NARRATION":{"status":"ENABLED"}},"buildId":"6b02f56c3ca2a55c00670bdc","clientVersion":"0.0.4270","searchFilterCounts":[{"language":"default","tag":"AI Gateway","count":253},{"language":"default","tag":"API Reference","count":202},{"language":"default","tag":"Platform","count":196},{"language":"default","tag":"Integrations","count":138},{"language":"default","tag":"LLM Gateway","count":118},{"language":"default","tag":"AI Deployment","count":109},{"language":"default","tag":"Integrate Models","count":51},{"language":"default","tag":"Guardrails and Security","count":48},{"language":"default","tag":"Observability","count":47},{"language":"default","tag":"MCP Servers","count":44}]},"assistantConfig":{"enableAskAiButton":true,"enableAskAiSearchBar":true,"deflection":{"enabled":true,"email":"[email protected]","showHelpButton":true}},"basePath":"","hasChatPermissions":true,"hasWebMcpPermissions":true,"toggles":[{"name":"kill-isr","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"custom-rbac-roles","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"agent-mintlify-slack","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"agent-integrations-settings","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"rbac-expansion","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"automations-pr-grouping","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"custom-prefix","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"analytics-v7","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"user-flows-analytics","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"no-publish-flow","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"workos-onboarding","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"snapshot-serving-live","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"stripTemperatureMiddleware","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"support-platform-pylon-zendesk","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"snapshot-serving","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"instant-rollback","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"granular-notifications-server","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"granular-notifications","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"automations-fable","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"fixed-ai-action-pricing","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"site-roles","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"automation-resolve-merge-conflict","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"scoped-publishing","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"mintql","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"page-level-analytics","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"editor-version-history-v2","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"billing-page-v2","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"full-page-search","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"workspace-projects","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"assistant-v5","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"fork-pr-previews","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"revalidation-jwt","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"workos-sso-scim","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"hobby-abuse-scan","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"dashboard-inbox","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"workos-signup","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"turbopuffer-dual-write","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"workos-enabled","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"approve-review-flow","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"editor-metadata","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"sso-setup-deadline-banner","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"turbopuffer-serve","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"live-agents","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"starter-pricing","enabled":true,"variant":{"name":"control","enabled":true,"payload":{"type":"string","value":"control"},"feature_enabled":true},"impressionData":false},{"name":"llms-txt-v2","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"wildcard-email-auth","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false}],"googleFontsInlined":false},"customCss":[{"filePath":"style.css","content":".line-highlight{background:rgb(var(--primary-light)/.7)!important}.tfy-cat-anchor{scroll-margin-top:6rem;margin-block:1.75rem}.line-highlight:before{background:rgb(var(--primary-light)/.7)!important}.line-highlight:after{background:rgb(var(--primary-light)/.7)!important}"}]}
{"page":{"pagePath":"docs/using-fractional-gpus","slug":"/docs/using-fractional-gpus","pageMetadata":{"title":"Using Fractional GPUs","description":"Share a single GPU across multiple workloads using Nvidia MIG and TimeSlicing for cost-efficient deployments.","href":"/docs/using-fractional-gpus"},"description":{"compiledSource":"\"use strict\";\nconst {jsx: _jsx} = arguments[0];\nconst {useMDXComponents: _provideComponents} = arguments[0];\nfunction _createMdxContent(props) {\n const _components = {\n p: \"p\",\n ..._provideComponents(),\n ...props.components\n };\n return _jsx(_components.p, {\n children: \"Share a single GPU across multiple workloads using Nvidia MIG and TimeSlicing for cost-efficient deployments.\"\n });\n}\nfunction MDXContent(props = {}) {\n const {wrapper: MDXLayout} = {\n ..._provideComponents(),\n ...props.components\n };\n return MDXLayout ? _jsx(MDXLayout, {\n ...props,\n children: _jsx(_createMdxContent, {\n ...props\n })\n }) : _createMdxContent(props);\n}\nreturn {\n default: MDXContent\n};\n","frontmatter":{},"scope":{}},"pageType":"default","mdx":{"mdxSource":{"compiledSource":"\"use strict\";const{Fragment:_Fragment,jsx:_jsx,jsxs:_jsxs}=arguments[0];const{useMDXComponents:_provideComponents}=arguments[0];function _createMdxContent(props){const _components={a:`a`,code:`code`,li:`li`,ol:`ol`,p:`p`,pre:`pre`,span:`span`,strong:`strong`,tbody:`tbody`,td:`td`,th:`th`,thead:`thead`,tr:`tr`,ul:`ul`,..._provideComponents(),...props.components},{CodeBlock,Frame,Heading,Info,OptimizedImage,Step,Steps,Tab,Table,Tabs,Warning}=_components;if(!CodeBlock)_missingMdxReference(`CodeBlock`,true);if(!Frame)_missingMdxReference(`Frame`,true);if(!Heading)_missingMdxReference(`Heading`,true);if(!Info)_missingMdxReference(`Info`,true);if(!OptimizedImage)_missingMdxReference(`OptimizedImage`,true);if(!Step)_missingMdxReference(`Step`,true);if(!Steps)_missingMdxReference(`Steps`,true);if(!Tab)_missingMdxReference(`Tab`,true);if(!Table)_missingMdxReference(`Table`,true);if(!Tabs)_missingMdxReference(`Tabs`,true);if(!Warning)_missingMdxReference(`Warning`,true);return _jsxs(_Fragment,{children:[_jsx(_components.p,{children:`Fractional GPUs enable us to allocate multiple workloads to a single GPU which can be useful in the following scenarios:`}),`\n`,_jsxs(_components.ol,{children:[`\n`,_jsx(_components.li,{children:`The workloads take around 2-3 GB of VRAM - so you can allocate multiple replicas of this workload on a single GPU which has around 16GB of VRAM or more.`}),`\n`,_jsx(_components.li,{children:`Each workload has sparse traffic and its not able to max out the GPU usage.`}),`\n`]}),`\n`,_jsx(_components.p,{children:`There are two ways to use fractional GPUs:`}),`\n`,_jsxs(_components.ol,{children:[`\n`,_jsxs(_components.li,{children:[_jsx(_components.strong,{children:`TimeSlicing`}),`: In this approach, you can slice a GPU into a few fixed number of fractional parts and then choose a fraction of GPU for this workload. For e.g, we can decide to divide a GPU into 10 slices and then request 3 slices for one workload, 5 slices for workload2 and 2 slices for the workload3. This means that worload1 will use 0.3 GPU (compute + 30% of VRAM), workload2 will use 0.5 GPU and workload3 will use 0.2 GPU. However, timeslicing is only used for scheduling the workloads on the same GPU - it doesn’t mean any actual isolation on the machine. For example, if the GPU machine has 16GB VRAM, its on the user to actually make sure that workload1 takes less than 4.8GB VRAM, workload2 takes less than 8 GB of VRAM and workload3 takes less than 3.2 GB of VRAM. If one workload starts taking more memory suddenly, it can lead to crashing of the other processes. The compute is also shared but one workload can go upto using the complete GPU if the other workloads are idle - its basically context-switching among the three workloads. You can read about this more `,_jsx(_components.a,{href:`https://docs.nvidia.com/datacenter/cloud-native/gpu-operator/latest/gpu-sharing.html#understanding-time-slicing-gpus`,children:`here.`})]}),`\n`,_jsxs(_components.li,{children:[_jsx(_components.strong,{children:`MIG (Multi-Instance GPUs)`}),`: This is a feature provided by Nvidia only on the A100, H100 and newer generation GPUs - it doesn’t work on other GPUs. We can divide the GPUs into a fixed number of configurable parts as mentioned in the table below. The workloads can choose one of the slices and they will get compute and memory isolation. The instances are not exactly the complete fractions of the GPU, but more discrete units as mentioned in the table below. For e.g. let’s say we divide one A100 GPU of 40GB into 7 parts - then we can place 7 workloads each using around 1/7 GPU and 5GB VRAM. Please note that we cannot simply provide 2 slices to one workload in this case and expect it to get 2/7 GPU and 10 GB VRAM. Each workload can only get one slice in this case. Here are the different `,_jsx(_components.strong,{children:`MIG profiles`}),`:`]}),`\n`]}),`\n`,_jsx(_components.p,{children:`A brief comparison chart between TimeSlicing and MIG is as follows:`}),`\n`,_jsxs(Table,{children:[_jsx(_components.thead,{children:_jsxs(_components.tr,{children:[_jsx(_components.th,{children:`Feature`}),_jsx(_components.th,{children:`TimeSlicing`}),_jsx(_components.th,{children:`MIG (Multi-Instance GPU)`})]})}),_jsxs(_components.tbody,{children:[_jsxs(_components.tr,{children:[_jsx(_components.td,{children:_jsx(_components.strong,{children:`GPU Support`})}),_jsx(_components.td,{children:`Works on most GPUs.`}),_jsx(_components.td,{children:`Only supported on NVIDIA A100 and H100 GPUs.`})]}),_jsxs(_components.tr,{children:[_jsx(_components.td,{children:_jsx(_components.strong,{children:`Isolation`})}),_jsxs(_components.td,{children:[_jsx(_components.strong,{children:`No real isolation.`}),` User is responsible for memory management. Potential for crashes if one workload exceeds its allocated memory.`]}),_jsxs(_components.td,{children:[_jsx(_components.strong,{children:`Strong Isolation.`}),` Compute and memory are isolated between instances. Guaranteed resource allocation.`]})]}),_jsxs(_components.tr,{children:[_jsx(_components.td,{children:_jsx(_components.strong,{children:`Resource Allocation`})}),_jsx(_components.td,{children:`Divides GPU into fractional parts (e.g., 0.3, 0.5, 0.2). Workloads can use these fractional parts.`}),_jsx(_components.td,{children:`Divides GPU into pre-defined, discrete instance types (as per NVIDIA’s configurations). Workloads are assigned entire instances.`})]}),_jsxs(_components.tr,{children:[_jsx(_components.td,{children:_jsx(_components.strong,{children:`VRAM Management`})}),_jsx(_components.td,{children:`User-managed. VRAM allocation is not enforced by the hardware.`}),_jsx(_components.td,{children:`Hardware-enforced. Each instance has dedicated VRAM.`})]}),_jsxs(_components.tr,{children:[_jsx(_components.td,{children:_jsx(_components.strong,{children:`Compute Sharing`})}),_jsx(_components.td,{children:`Compute is shared via context-switching. Workloads can potentially use the entire GPU when others are idle.`}),_jsx(_components.td,{children:`Compute is partitioned and isolated to each instance. No sharing of compute resources beyond the instance’s allocation.`})]}),_jsxs(_components.tr,{children:[_jsx(_components.td,{children:_jsx(_components.strong,{children:`Flexibility`})}),_jsx(_components.td,{children:`More flexible in terms of resource allocation fractions (e.g., can request 0.3, 0.5, etc.).`}),_jsx(_components.td,{children:`Limited to NVIDIA’s pre-defined instance types. Less flexible in terms of fine-grained resource requests.`})]})]})]}),`\n`,_jsx(Heading,{level:`2`,id:`deploying-on-fractional-gpus-via-mig`,children:`Deploying on Fractional GPUs via MIG`}),`\n`,_jsxs(Steps,{children:[_jsxs(Step,{title:`Create a Nodepool with MIG enabled`,children:[_jsxs(_components.p,{children:[`We will need to create a separate nodepool for MIG enabled GPUs. Every GPU has different MIG profiles as mentioned in this page: `,_jsx(_components.a,{href:`https://docs.nvidia.com/datacenter/tesla/mig-user-guide/index.html`,children:`https://docs.nvidia.com/datacenter/tesla/mig-user-guide/index.html`}),`. For e.g. here are the MIG profiles for A100 GPU:`]}),_jsxs(Table,{children:[_jsx(_components.thead,{children:_jsxs(_components.tr,{children:[_jsx(_components.th,{children:`GPU`}),_jsx(_components.th,{children:`GPU Compute Fraction / Instance`}),_jsx(_components.th,{children:`Number of instances per GPU`}),_jsx(_components.th,{children:`GPU Memory / Instance`}),_jsx(_components.th,{children:`Configuration Name`}),_jsx(_components.th,{children:`GPU Instance Profile (for Azure)`})]})}),_jsxs(_components.tbody,{children:[_jsxs(_components.tr,{children:[_jsx(_components.td,{children:`A100 (40GB)`}),_jsx(_components.td,{\"data-numeric\":`true`,children:`1/7`}),_jsx(_components.td,{\"data-numeric\":`true`,children:`7`}),_jsx(_components.td,{\"data-numeric\":`true`,children:`5GB`}),_jsx(_components.td,{children:`1g.5gb`}),_jsx(_components.td,{children:`MIG1g`})]}),_jsxs(_components.tr,{children:[_jsx(_components.td,{children:`A100 (40GB)`}),_jsx(_components.td,{\"data-numeric\":`true`,children:`2/7`}),_jsx(_components.td,{\"data-numeric\":`true`,children:`3`}),_jsx(_components.td,{\"data-numeric\":`true`,children:`10GB`}),_jsx(_components.td,{children:`2g.10gb`}),_jsx(_components.td,{children:`MIG2g`})]}),_jsxs(_components.tr,{children:[_jsx(_components.td,{children:`A100 (40GB)`}),_jsx(_components.td,{\"data-numeric\":`true`,children:`3/7`}),_jsx(_components.td,{\"data-numeric\":`true`,children:`2`}),_jsx(_components.td,{\"data-numeric\":`true`,children:`20GB`}),_jsx(_components.td,{children:`3g.20gb`}),_jsx(_components.td,{children:`MIG3g`})]}),_jsxs(_components.tr,{children:[_jsx(_components.td,{children:`A100 (80GB)`}),_jsx(_components.td,{\"data-numeric\":`true`,children:`1/7`}),_jsx(_components.td,{\"data-numeric\":`true`,children:`7`}),_jsx(_components.td,{\"data-numeric\":`true`,children:`10GB`}),_jsx(_components.td,{children:`1g.10gb`}),_jsx(_components.td,{children:`MIG1g`})]}),_jsxs(_components.tr,{children:[_jsx(_components.td,{children:`A100 (80GB)`}),_jsx(_components.td,{\"data-numeric\":`true`,children:`2/7`}),_jsx(_components.td,{\"data-numeric\":`true`,children:`3`}),_jsx(_components.td,{\"data-numeric\":`true`,children:`20GB`}),_jsx(_components.td,{children:`2g.20gb`}),_jsx(_components.td,{children:`MIG2g`})]}),_jsxs(_components.tr,{children:[_jsx(_components.td,{children:`A100 (80GB)`}),_jsx(_components.td,{\"data-numeric\":`true`,children:`3/7`}),_jsx(_components.td,{\"data-numeric\":`true`,children:`2`}),_jsx(_components.td,{\"data-numeric\":`true`,children:`40GB`}),_jsx(_components.td,{children:`3g.40gb`}),_jsx(_components.td,{children:`MIG3g`})]})]})]}),_jsx(_components.p,{children:`While creating the nodepool, we will need to select the MIG profile. Here are the steps to do it in different cloud providers:`}),_jsxs(Tabs,{children:[_jsxs(Tab,{title:`Azure`,id:`azure`,children:[_jsxs(_components.p,{children:[`Create a Nodepool with MIG enabled using the argument `,_jsx(_components.code,{children:`--gpu-instance-profile`}),` of Azure CLI.`]}),_jsx(CodeBlock,{filename:`Shell`,lines:`true`,numberOfLines:`18`,language:`shellscript`,children:_jsx(_components.pre,{className:`shiki shiki-themes dark-plus dark-plus`,style:{backgroundColor:`#0B0C0E`,\"--shiki-dark-bg\":`#0B0C0E`,color:`#D4D4D4`,\"--shiki-dark\":`#D4D4D4`},language:`shellscript`,children:_jsxs(_components.code,{language:`shellscript`,numberOfLines:`18`,children:[_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#DCDCAA`,\"--shiki-dark\":`#DCDCAA`},children:`az`}),_jsx(_components.span,{style:{color:`#CE9178`,\"--shiki-dark\":`#CE9178`},children:` aks`}),_jsx(_components.span,{style:{color:`#CE9178`,\"--shiki-dark\":`#CE9178`},children:` nodepool`}),_jsx(_components.span,{style:{color:`#CE9178`,\"--shiki-dark\":`#CE9178`},children:` add`}),_jsx(_components.span,{style:{color:`#D7BA7D`,\"--shiki-dark\":`#D7BA7D`},children:` \\\\`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#D4D4D4`,\"--shiki-dark\":`#D4D4D4`},children:`--cluster-name \u003c`}),_jsx(_components.span,{style:{color:`#CE9178`,\"--shiki-dark\":`#CE9178`},children:`your`}),_jsx(_components.span,{style:{color:`#CE9178`,\"--shiki-dark\":`#CE9178`},children:` cluster`}),_jsx(_components.span,{style:{color:`#CE9178`,\"--shiki-dark\":`#CE9178`},children:` nam`}),_jsx(_components.span,{style:{color:`#D4D4D4`,\"--shiki-dark\":`#D4D4D4`},children:`e> `}),_jsx(_components.span,{style:{color:`#D7BA7D`,\"--shiki-dark\":`#D7BA7D`},children:`\\\\`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#D4D4D4`,\"--shiki-dark\":`#D4D4D4`},children:`--resource-group \u003c`}),_jsx(_components.span,{style:{color:`#CE9178`,\"--shiki-dark\":`#CE9178`},children:`your`}),_jsx(_components.span,{style:{color:`#CE9178`,\"--shiki-dark\":`#CE9178`},children:` resource`}),_jsx(_components.span,{style:{color:`#CE9178`,\"--shiki-dark\":`#CE9178`},children:` grou`}),_jsx(_components.span,{style:{color:`#D4D4D4`,\"--shiki-dark\":`#D4D4D4`},children:`p> `}),_jsx(_components.span,{style:{color:`#D7BA7D`,\"--shiki-dark\":`#D7BA7D`},children:`\\\\`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#D4D4D4`,\"--shiki-dark\":`#D4D4D4`},children:`--no-wait `}),_jsx(_components.span,{style:{color:`#D7BA7D`,\"--shiki-dark\":`#D7BA7D`},children:`\\\\`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#D4D4D4`,\"--shiki-dark\":`#D4D4D4`},children:`--enable-cluster-autoscaler `}),_jsx(_components.span,{style:{color:`#D7BA7D`,\"--shiki-dark\":`#D7BA7D`},children:`\\\\`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#D4D4D4`,\"--shiki-dark\":`#D4D4D4`},children:`--eviction-policy `}),_jsx(_components.span,{style:{color:`#CE9178`,\"--shiki-dark\":`#CE9178`},children:`Delete`}),_jsx(_components.span,{style:{color:`#D7BA7D`,\"--shiki-dark\":`#D7BA7D`},children:` \\\\`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#D4D4D4`,\"--shiki-dark\":`#D4D4D4`},children:`--node-count `}),_jsx(_components.span,{style:{color:`#B5CEA8`,\"--shiki-dark\":`#B5CEA8`},children:`0`}),_jsx(_components.span,{style:{color:`#D7BA7D`,\"--shiki-dark\":`#D7BA7D`},children:` \\\\`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#D4D4D4`,\"--shiki-dark\":`#D4D4D4`},children:`--max-count `}),_jsx(_components.span,{style:{color:`#B5CEA8`,\"--shiki-dark\":`#B5CEA8`},children:`20`}),_jsx(_components.span,{style:{color:`#D7BA7D`,\"--shiki-dark\":`#D7BA7D`},children:` \\\\`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#D4D4D4`,\"--shiki-dark\":`#D4D4D4`},children:`--min-count `}),_jsx(_components.span,{style:{color:`#B5CEA8`,\"--shiki-dark\":`#B5CEA8`},children:`1`}),_jsx(_components.span,{style:{color:`#D7BA7D`,\"--shiki-dark\":`#D7BA7D`},children:` \\\\`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#D4D4D4`,\"--shiki-dark\":`#D4D4D4`},children:`--node-osdisk-size `}),_jsx(_components.span,{style:{color:`#B5CEA8`,\"--shiki-dark\":`#B5CEA8`},children:`200`}),_jsx(_components.span,{style:{color:`#D7BA7D`,\"--shiki-dark\":`#D7BA7D`},children:` \\\\`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#D4D4D4`,\"--shiki-dark\":`#D4D4D4`},children:`--scale-down-mode `}),_jsx(_components.span,{style:{color:`#CE9178`,\"--shiki-dark\":`#CE9178`},children:`Delete`}),_jsx(_components.span,{style:{color:`#D7BA7D`,\"--shiki-dark\":`#D7BA7D`},children:` \\\\`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#D4D4D4`,\"--shiki-dark\":`#D4D4D4`},children:`--os-type `}),_jsx(_components.span,{style:{color:`#CE9178`,\"--shiki-dark\":`#CE9178`},children:`Linux`}),_jsx(_components.span,{style:{color:`#D7BA7D`,\"--shiki-dark\":`#D7BA7D`},children:` \\\\`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#D4D4D4`,\"--shiki-dark\":`#D4D4D4`},children:`--node-taints `}),_jsx(_components.span,{style:{color:`#CE9178`,\"--shiki-dark\":`#CE9178`},children:`\"nvidia.com/gpu=Present:NoSchedule\"`}),_jsx(_components.span,{style:{color:`#D7BA7D`,\"--shiki-dark\":`#D7BA7D`},children:` \\\\`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#D4D4D4`,\"--shiki-dark\":`#D4D4D4`},children:`--name `}),_jsx(_components.span,{style:{color:`#CE9178`,\"--shiki-dark\":`#CE9178`},children:`a100mig7`}),_jsx(_components.span,{style:{color:`#D7BA7D`,\"--shiki-dark\":`#D7BA7D`},children:` \\\\`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#D4D4D4`,\"--shiki-dark\":`#D4D4D4`},children:`--node-vm-size `}),_jsx(_components.span,{style:{color:`#CE9178`,\"--shiki-dark\":`#CE9178`},children:`Standard_NC24ads_A100_v4`}),_jsx(_components.span,{style:{color:`#D7BA7D`,\"--shiki-dark\":`#D7BA7D`},children:` \\\\`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#D4D4D4`,\"--shiki-dark\":`#D4D4D4`},children:`--priority `}),_jsx(_components.span,{style:{color:`#CE9178`,\"--shiki-dark\":`#CE9178`},children:`Spot`}),_jsx(_components.span,{style:{color:`#D7BA7D`,\"--shiki-dark\":`#D7BA7D`},children:` \\\\`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#D4D4D4`,\"--shiki-dark\":`#D4D4D4`},children:`--os-sku `}),_jsx(_components.span,{style:{color:`#CE9178`,\"--shiki-dark\":`#CE9178`},children:`Ubuntu`}),_jsx(_components.span,{style:{color:`#D7BA7D`,\"--shiki-dark\":`#D7BA7D`},children:` \\\\`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#D4D4D4`,\"--shiki-dark\":`#D4D4D4`},children:`--gpu-instance-profile `}),_jsx(_components.span,{style:{color:`#CE9178`,\"--shiki-dark\":`#CE9178`},children:`MIG1g`})]}),`\n`]})})})]}),_jsxs(Tab,{title:`GCP`,id:`gcp`,children:[_jsxs(_components.p,{children:[`Create a nodepool and pass the mig_profile in `,_jsx(_components.code,{children:`accelerator`}),` by passing `,_jsx(_components.code,{children:`gpu_partition_size=1g.5gb`}),`[OR one of the allowed values for MIG profile values mentioned above]`]}),_jsx(CodeBlock,{filename:`Shell`,lines:`true`,numberOfLines:`11`,language:`shellscript`,children:_jsx(_components.pre,{className:`shiki shiki-themes dark-plus dark-plus`,style:{backgroundColor:`#0B0C0E`,\"--shiki-dark-bg\":`#0B0C0E`,color:`#D4D4D4`,\"--shiki-dark\":`#D4D4D4`},language:`shellscript`,children:_jsxs(_components.code,{language:`shellscript`,numberOfLines:`11`,children:[_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#DCDCAA`,\"--shiki-dark\":`#DCDCAA`},children:`gcloud`}),_jsx(_components.span,{style:{color:`#CE9178`,\"--shiki-dark\":`#CE9178`},children:` container`}),_jsx(_components.span,{style:{color:`#CE9178`,\"--shiki-dark\":`#CE9178`},children:` node-pools`}),_jsx(_components.span,{style:{color:`#CE9178`,\"--shiki-dark\":`#CE9178`},children:` create`}),_jsx(_components.span,{style:{color:`#CE9178`,\"--shiki-dark\":`#CE9178`},children:` a100-40-mig-1g5gb`}),_jsx(_components.span,{style:{color:`#D7BA7D`,\"--shiki-dark\":`#D7BA7D`},children:` \\\\ `}),_jsx(_components.span,{style:{color:`#CE9178`,\"--shiki-dark\":`#CE9178`},children:` `}),_jsx(_components.span,{style:{color:`#CE9178`,\"--shiki-dark\":`#CE9178`},children:` INT`}),_jsx(_components.span,{style:{color:`#CE9178`,\"--shiki-dark\":`#CE9178`},children:` ✘`}),_jsx(_components.span,{style:{color:`#CE9178`,\"--shiki-dark\":`#CE9178`},children:` `}),_jsx(_components.span,{style:{color:`#CE9178`,\"--shiki-dark\":`#CE9178`},children:` `})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#9CDCFE`,\"--shiki-dark\":`#9CDCFE`},children:` --project`}),_jsx(_components.span,{style:{color:`#D4D4D4`,\"--shiki-dark\":`#D4D4D4`},children:`=\u003c`}),_jsx(_components.span,{style:{color:`#CE9178`,\"--shiki-dark\":`#CE9178`},children:`enter`}),_jsx(_components.span,{style:{color:`#DCDCAA`,\"--shiki-dark\":`#DCDCAA`},children:` your`}),_jsx(_components.span,{style:{color:`#CE9178`,\"--shiki-dark\":`#CE9178`},children:` project`}),_jsx(_components.span,{style:{color:`#CE9178`,\"--shiki-dark\":`#CE9178`},children:` nam`}),_jsx(_components.span,{style:{color:`#D4D4D4`,\"--shiki-dark\":`#D4D4D4`},children:`e> `}),_jsx(_components.span,{style:{color:`#D7BA7D`,\"--shiki-dark\":`#D7BA7D`},children:`\\\\`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#569CD6`,\"--shiki-dark\":`#569CD6`},children:` --region=`}),_jsx(_components.span,{style:{color:`#D4D4D4`,\"--shiki-dark\":`#D4D4D4`},children:`\u003c`}),_jsx(_components.span,{style:{color:`#569CD6`,\"--shiki-dark\":`#569CD6`},children:`enter`}),_jsx(_components.span,{style:{color:`#CE9178`,\"--shiki-dark\":`#CE9178`},children:` your`}),_jsx(_components.span,{style:{color:`#CE9178`,\"--shiki-dark\":`#CE9178`},children:` regio`}),_jsx(_components.span,{style:{color:`#D4D4D4`,\"--shiki-dark\":`#D4D4D4`},children:`n> `}),_jsx(_components.span,{style:{color:`#D7BA7D`,\"--shiki-dark\":`#D7BA7D`},children:`\\\\`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#569CD6`,\"--shiki-dark\":`#569CD6`},children:` --cluster=`}),_jsx(_components.span,{style:{color:`#D4D4D4`,\"--shiki-dark\":`#D4D4D4`},children:`\u003c`}),_jsx(_components.span,{style:{color:`#569CD6`,\"--shiki-dark\":`#569CD6`},children:`enter`}),_jsx(_components.span,{style:{color:`#CE9178`,\"--shiki-dark\":`#CE9178`},children:` your`}),_jsx(_components.span,{style:{color:`#CE9178`,\"--shiki-dark\":`#CE9178`},children:` cluster`}),_jsx(_components.span,{style:{color:`#CE9178`,\"--shiki-dark\":`#CE9178`},children:` name`}),_jsx(_components.span,{style:{color:`#CE9178`,\"--shiki-dark\":`#CE9178`},children:` her`}),_jsx(_components.span,{style:{color:`#D4D4D4`,\"--shiki-dark\":`#D4D4D4`},children:`e> `}),_jsx(_components.span,{style:{color:`#D7BA7D`,\"--shiki-dark\":`#D7BA7D`},children:`\\\\`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#569CD6`,\"--shiki-dark\":`#569CD6`},children:` --machine-type=a2-highgpu-1g`}),_jsx(_components.span,{style:{color:`#D7BA7D`,\"--shiki-dark\":`#D7BA7D`},children:` \\\\`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#569CD6`,\"--shiki-dark\":`#569CD6`},children:` --accelerator`}),_jsx(_components.span,{style:{color:`#CE9178`,\"--shiki-dark\":`#CE9178`},children:` type=nvidia-tesla-a100,count=1,gpu-partition-size=1g.5gb`}),_jsx(_components.span,{style:{color:`#D7BA7D`,\"--shiki-dark\":`#D7BA7D`},children:` \\\\`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#569CD6`,\"--shiki-dark\":`#569CD6`},children:` --enable-autoscaling`}),_jsx(_components.span,{style:{color:`#D7BA7D`,\"--shiki-dark\":`#D7BA7D`},children:` \\\\`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#569CD6`,\"--shiki-dark\":`#569CD6`},children:` --total-min-nodes`}),_jsx(_components.span,{style:{color:`#B5CEA8`,\"--shiki-dark\":`#B5CEA8`},children:` 0`}),_jsx(_components.span,{style:{color:`#D7BA7D`,\"--shiki-dark\":`#D7BA7D`},children:` \\\\`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#569CD6`,\"--shiki-dark\":`#569CD6`},children:` --total-max-nodes`}),_jsx(_components.span,{style:{color:`#B5CEA8`,\"--shiki-dark\":`#B5CEA8`},children:` 4`}),_jsx(_components.span,{style:{color:`#D7BA7D`,\"--shiki-dark\":`#D7BA7D`},children:` \\\\`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#569CD6`,\"--shiki-dark\":`#569CD6`},children:` --min-provision-nodes`}),_jsx(_components.span,{style:{color:`#B5CEA8`,\"--shiki-dark\":`#B5CEA8`},children:` 0`}),_jsx(_components.span,{style:{color:`#D7BA7D`,\"--shiki-dark\":`#D7BA7D`},children:` \\\\`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#569CD6`,\"--shiki-dark\":`#569CD6`},children:` --num-nodes`}),_jsx(_components.span,{style:{color:`#B5CEA8`,\"--shiki-dark\":`#B5CEA8`},children:` 0`})]}),`\n`]})})})]}),_jsxs(Tab,{title:`AWS`,id:`aws`,children:[_jsxs(_components.p,{children:[`On AWS EKS you do `,_jsx(_components.strong,{children:`not`}),` create managed nodegroups for MIG. TrueFoundry discovers fractional GPUs from Karpenter `,_jsx(_components.strong,{children:`NodePools`}),` + `,_jsx(_components.strong,{children:`NodeOverlays`}),`. Enable MIG by updating three things:`]}),_jsxs(_components.ol,{children:[`\n`,_jsxs(_components.li,{children:[_jsx(_components.code,{children:`tfy-gpu-operator`}),` values (MIG configs)`]}),`\n`,_jsxs(_components.li,{children:[_jsx(_components.code,{children:`tfy-karpenter-config`}),` kustomize additions (NodePool + NodeOverlay)`]}),`\n`,_jsx(_components.li,{children:`Sync Cluster in the UI`}),`\n`]}),_jsxs(_components.p,{children:[`Use the latest `,_jsx(_components.a,{href:`https://artifacthub.io/packages/helm/truefoundry/tfy-gpu-operator`,children:`tfy-gpu-operator`}),` and `,_jsx(_components.a,{href:`https://artifacthub.io/packages/helm/truefoundry/tfy-karpenter-config`,children:`tfy-karpenter-config`}),` chart versions.`]}),_jsx(_components.p,{children:_jsxs(_components.strong,{children:[`1. Configure MIG profiles in `,_jsx(_components.code,{children:`tfy-gpu-operator`})]})}),_jsxs(_components.p,{children:[`Edit the `,_jsx(_components.code,{children:`tfy-gpu-operator`}),` Helm application and merge the following into values (add more `,_jsx(_components.code,{children:`mig-configs`}),` entries for other profiles you need):`]}),_jsx(CodeBlock,{filename:``,numberOfLines:`27`,language:`yaml`,children:_jsx(_components.pre,{className:`shiki shiki-themes dark-plus dark-plus`,style:{backgroundColor:`#0B0C0E`,\"--shiki-dark-bg\":`#0B0C0E`,color:`#D4D4D4`,\"--shiki-dark\":`#D4D4D4`},language:`yaml`,children:_jsxs(_components.code,{language:`yaml`,numberOfLines:`27`,children:[_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#569CD6`,\"--shiki-dark\":`#569CD6`},children:`clusterType`}),_jsx(_components.span,{style:{color:`#D4D4D4`,\"--shiki-dark\":`#D4D4D4`},children:`:`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#569CD6`,\"--shiki-dark\":`#569CD6`},children:` awsEks`}),_jsx(_components.span,{style:{color:`#D4D4D4`,\"--shiki-dark\":`#D4D4D4`},children:`: `}),_jsx(_components.span,{style:{color:`#569CD6`,\"--shiki-dark\":`#569CD6`},children:`true`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#569CD6`,\"--shiki-dark\":`#569CD6`},children:`aws-eks-gpu-operator`}),_jsx(_components.span,{style:{color:`#D4D4D4`,\"--shiki-dark\":`#D4D4D4`},children:`:`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#569CD6`,\"--shiki-dark\":`#569CD6`},children:` mig`}),_jsx(_components.span,{style:{color:`#D4D4D4`,\"--shiki-dark\":`#D4D4D4`},children:`:`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#569CD6`,\"--shiki-dark\":`#569CD6`},children:` strategy`}),_jsx(_components.span,{style:{color:`#D4D4D4`,\"--shiki-dark\":`#D4D4D4`},children:`: `}),_jsx(_components.span,{style:{color:`#CE9178`,\"--shiki-dark\":`#CE9178`},children:`mixed`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#569CD6`,\"--shiki-dark\":`#569CD6`},children:` migManager`}),_jsx(_components.span,{style:{color:`#D4D4D4`,\"--shiki-dark\":`#D4D4D4`},children:`:`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#569CD6`,\"--shiki-dark\":`#569CD6`},children:` enabled`}),_jsx(_components.span,{style:{color:`#D4D4D4`,\"--shiki-dark\":`#D4D4D4`},children:`: `}),_jsx(_components.span,{style:{color:`#569CD6`,\"--shiki-dark\":`#569CD6`},children:`true`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#569CD6`,\"--shiki-dark\":`#569CD6`},children:` config`}),_jsx(_components.span,{style:{color:`#D4D4D4`,\"--shiki-dark\":`#D4D4D4`},children:`:`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#569CD6`,\"--shiki-dark\":`#569CD6`},children:` name`}),_jsx(_components.span,{style:{color:`#D4D4D4`,\"--shiki-dark\":`#D4D4D4`},children:`: `}),_jsx(_components.span,{style:{color:`#CE9178`,\"--shiki-dark\":`#CE9178`},children:`nvidia-mig-parted-configs`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#569CD6`,\"--shiki-dark\":`#569CD6`},children:` create`}),_jsx(_components.span,{style:{color:`#D4D4D4`,\"--shiki-dark\":`#D4D4D4`},children:`: `}),_jsx(_components.span,{style:{color:`#569CD6`,\"--shiki-dark\":`#569CD6`},children:`true`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#569CD6`,\"--shiki-dark\":`#569CD6`},children:` data`}),_jsx(_components.span,{style:{color:`#D4D4D4`,\"--shiki-dark\":`#D4D4D4`},children:`:`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#569CD6`,\"--shiki-dark\":`#569CD6`},children:` config.yaml`}),_jsx(_components.span,{style:{color:`#D4D4D4`,\"--shiki-dark\":`#D4D4D4`},children:`: `}),_jsx(_components.span,{style:{color:`#C586C0`,\"--shiki-dark\":`#C586C0`},children:`|`}),_jsx(_components.span,{style:{color:`#569CD6`,\"--shiki-dark\":`#569CD6`},children:`-`})]}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{style:{color:`#CE9178`,\"--shiki-dark\":`#CE9178`},children:` version: v1`})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{style:{color:`#CE9178`,\"--shiki-dark\":`#CE9178`},children:` mig-configs:`})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{style:{color:`#CE9178`,\"--shiki-dark\":`#CE9178`},children:` all-disabled:`})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{style:{color:`#CE9178`,\"--shiki-dark\":`#CE9178`},children:` - devices: all`})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{style:{color:`#CE9178`,\"--shiki-dark\":`#CE9178`},children:` mig-enabled: false`})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{style:{color:`#CE9178`,\"--shiki-dark\":`#CE9178`},children:` all-1g.10gb:`})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{style:{color:`#CE9178`,\"--shiki-dark\":`#CE9178`},children:` - devices: all`})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{style:{color:`#CE9178`,\"--shiki-dark\":`#CE9178`},children:` mig-enabled: true`})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{style:{color:`#CE9178`,\"--shiki-dark\":`#CE9178`},children:` mig-devices:`})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{style:{color:`#CE9178`,\"--shiki-dark\":`#CE9178`},children:` \"1g.10gb\": 7`})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{style:{color:`#CE9178`,\"--shiki-dark\":`#CE9178`},children:` all-1g.24gb:`})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{style:{color:`#CE9178`,\"--shiki-dark\":`#CE9178`},children:` - devices: all`})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{style:{color:`#CE9178`,\"--shiki-dark\":`#CE9178`},children:` mig-enabled: true`})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{style:{color:`#CE9178`,\"--shiki-dark\":`#CE9178`},children:` mig-devices:`})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{style:{color:`#CE9178`,\"--shiki-dark\":`#CE9178`},children:` \"1g.24gb\": 4`})}),`\n`]})})}),_jsxs(_components.p,{children:[`The config name under `,_jsx(_components.code,{children:`mig-configs`}),` (for example `,_jsx(_components.code,{children:`all-1g.24gb`}),`) must match the `,_jsx(_components.code,{children:`nvidia.com/mig.config`}),` label on the NodePool in the next step. Deploy the updated Helm application.`]}),_jsx(_components.p,{children:_jsxs(_components.strong,{children:[`2. Add a Karpenter NodePool + NodeOverlay via `,_jsx(_components.code,{children:`tfy-karpenter-config`})]})}),_jsxs(_components.p,{children:[`Edit the `,_jsx(_components.code,{children:`tfy-karpenter-config`}),` Helm application. Under `,_jsx(_components.strong,{children:`Kustomize → Additional Manifests`}),`, add both resources. Example: RTX PRO 6000 on `,_jsx(_components.code,{children:`g7e.2xlarge`}),` with MIG profile `,_jsx(_components.code,{children:`1g.24gb`}),` (4 partitions per GPU):`]}),_jsx(CodeBlock,{filename:``,numberOfLines:`57`,language:`yaml`,children:_jsx(_components.pre,{className:`shiki shiki-themes dark-plus dark-plus`,style:{backgroundColor:`#0B0C0E`,\"--shiki-dark-bg\":`#0B0C0E`,color:`#D4D4D4`,\"--shiki-dark\":`#D4D4D4`},language:`yaml`,children:_jsxs(_components.code,{language:`yaml`,numberOfLines:`57`,children:[_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#569CD6`,\"--shiki-dark\":`#569CD6`},children:`apiVersion`}),_jsx(_components.span,{style:{color:`#D4D4D4`,\"--shiki-dark\":`#D4D4D4`},children:`: `}),_jsx(_components.span,{style:{color:`#CE9178`,\"--shiki-dark\":`#CE9178`},children:`karpenter.sh/v1`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#569CD6`,\"--shiki-dark\":`#569CD6`},children:`kind`}),_jsx(_components.span,{style:{color:`#D4D4D4`,\"--shiki-dark\":`#D4D4D4`},children:`: `}),_jsx(_components.span,{style:{color:`#CE9178`,\"--shiki-dark\":`#CE9178`},children:`NodePool`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#569CD6`,\"--shiki-dark\":`#569CD6`},children:`metadata`}),_jsx(_components.span,{style:{color:`#D4D4D4`,\"--shiki-dark\":`#D4D4D4`},children:`:`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#569CD6`,\"--shiki-dark\":`#569CD6`},children:` name`}),_jsx(_components.span,{style:{color:`#D4D4D4`,\"--shiki-dark\":`#D4D4D4`},children:`: `}),_jsx(_components.span,{style:{color:`#CE9178`,\"--shiki-dark\":`#CE9178`},children:`g7e-2xlarge-mig-1g-24gb`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#569CD6`,\"--shiki-dark\":`#569CD6`},children:`spec`}),_jsx(_components.span,{style:{color:`#D4D4D4`,\"--shiki-dark\":`#D4D4D4`},children:`:`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#569CD6`,\"--shiki-dark\":`#569CD6`},children:` weight`}),_jsx(_components.span,{style:{color:`#D4D4D4`,\"--shiki-dark\":`#D4D4D4`},children:`: `}),_jsx(_components.span,{style:{color:`#B5CEA8`,\"--shiki-dark\":`#B5CEA8`},children:`10`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#569CD6`,\"--shiki-dark\":`#569CD6`},children:` disruption`}),_jsx(_components.span,{style:{color:`#D4D4D4`,\"--shiki-dark\":`#D4D4D4`},children:`:`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#569CD6`,\"--shiki-dark\":`#569CD6`},children:` consolidationPolicy`}),_jsx(_components.span,{style:{color:`#D4D4D4`,\"--shiki-dark\":`#D4D4D4`},children:`: `}),_jsx(_components.span,{style:{color:`#CE9178`,\"--shiki-dark\":`#CE9178`},children:`WhenEmptyOrUnderutilized`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#569CD6`,\"--shiki-dark\":`#569CD6`},children:` consolidateAfter`}),_jsx(_components.span,{style:{color:`#D4D4D4`,\"--shiki-dark\":`#D4D4D4`},children:`: `}),_jsx(_components.span,{style:{color:`#CE9178`,\"--shiki-dark\":`#CE9178`},children:`5m`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#569CD6`,\"--shiki-dark\":`#569CD6`},children:` budgets`}),_jsx(_components.span,{style:{color:`#D4D4D4`,\"--shiki-dark\":`#D4D4D4`},children:`:`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#D4D4D4`,\"--shiki-dark\":`#D4D4D4`},children:` - `}),_jsx(_components.span,{style:{color:`#569CD6`,\"--shiki-dark\":`#569CD6`},children:`nodes`}),_jsx(_components.span,{style:{color:`#D4D4D4`,\"--shiki-dark\":`#D4D4D4`},children:`: `}),_jsx(_components.span,{style:{color:`#CE9178`,\"--shiki-dark\":`#CE9178`},children:`10%`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#569CD6`,\"--shiki-dark\":`#569CD6`},children:` reasons`}),_jsx(_components.span,{style:{color:`#D4D4D4`,\"--shiki-dark\":`#D4D4D4`},children:`: [`}),_jsx(_components.span,{style:{color:`#CE9178`,\"--shiki-dark\":`#CE9178`},children:`Underutilized`}),_jsx(_components.span,{style:{color:`#D4D4D4`,\"--shiki-dark\":`#D4D4D4`},children:`, `}),_jsx(_components.span,{style:{color:`#CE9178`,\"--shiki-dark\":`#CE9178`},children:`Empty`}),_jsx(_components.span,{style:{color:`#D4D4D4`,\"--shiki-dark\":`#D4D4D4`},children:`]`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#D4D4D4`,\"--shiki-dark\":`#D4D4D4`},children:` - `}),_jsx(_components.span,{style:{color:`#569CD6`,\"--shiki-dark\":`#569CD6`},children:`nodes`}),_jsx(_components.span,{style:{color:`#D4D4D4`,\"--shiki-dark\":`#D4D4D4`},children:`: `}),_jsx(_components.span,{style:{color:`#CE9178`,\"--shiki-dark\":`#CE9178`},children:`\"1\"`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#569CD6`,\"--shiki-dark\":`#569CD6`},children:` reasons`}),_jsx(_components.span,{style:{color:`#D4D4D4`,\"--shiki-dark\":`#D4D4D4`},children:`: [`}),_jsx(_components.span,{style:{color:`#CE9178`,\"--shiki-dark\":`#CE9178`},children:`Drifted`}),_jsx(_components.span,{style:{color:`#D4D4D4`,\"--shiki-dark\":`#D4D4D4`},children:`]`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#569CD6`,\"--shiki-dark\":`#569CD6`},children:` template`}),_jsx(_components.span,{style:{color:`#D4D4D4`,\"--shiki-dark\":`#D4D4D4`},children:`:`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#569CD6`,\"--shiki-dark\":`#569CD6`},children:` metadata`}),_jsx(_components.span,{style:{color:`#D4D4D4`,\"--shiki-dark\":`#D4D4D4`},children:`:`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#569CD6`,\"--shiki-dark\":`#569CD6`},children:` labels`}),_jsx(_components.span,{style:{color:`#D4D4D4`,\"--shiki-dark\":`#D4D4D4`},children:`:`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#569CD6`,\"--shiki-dark\":`#569CD6`},children:` eks.amazonaws.com/nodegroup`}),_jsx(_components.span,{style:{color:`#D4D4D4`,\"--shiki-dark\":`#D4D4D4`},children:`: `}),_jsx(_components.span,{style:{color:`#CE9178`,\"--shiki-dark\":`#CE9178`},children:`g7e-2xlarge-mig-1g-24gb`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#569CD6`,\"--shiki-dark\":`#569CD6`},children:` nvidia.com/gpu.deploy.operands`}),_jsx(_components.span,{style:{color:`#D4D4D4`,\"--shiki-dark\":`#D4D4D4`},children:`: `}),_jsx(_components.span,{style:{color:`#CE9178`,\"--shiki-dark\":`#CE9178`},children:`\"true\"`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#569CD6`,\"--shiki-dark\":`#569CD6`},children:` nvidia.com/mig.config`}),_jsx(_components.span,{style:{color:`#D4D4D4`,\"--shiki-dark\":`#D4D4D4`},children:`: `}),_jsx(_components.span,{style:{color:`#CE9178`,\"--shiki-dark\":`#CE9178`},children:`all-1g.24gb`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#569CD6`,\"--shiki-dark\":`#569CD6`},children:` spec`}),_jsx(_components.span,{style:{color:`#D4D4D4`,\"--shiki-dark\":`#D4D4D4`},children:`:`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#569CD6`,\"--shiki-dark\":`#569CD6`},children:` expireAfter`}),_jsx(_components.span,{style:{color:`#D4D4D4`,\"--shiki-dark\":`#D4D4D4`},children:`: `}),_jsx(_components.span,{style:{color:`#CE9178`,\"--shiki-dark\":`#CE9178`},children:`720h`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#569CD6`,\"--shiki-dark\":`#569CD6`},children:` nodeClassRef`}),_jsx(_components.span,{style:{color:`#D4D4D4`,\"--shiki-dark\":`#D4D4D4`},children:`:`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#569CD6`,\"--shiki-dark\":`#569CD6`},children:` group`}),_jsx(_components.span,{style:{color:`#D4D4D4`,\"--shiki-dark\":`#D4D4D4`},children:`: `}),_jsx(_components.span,{style:{color:`#CE9178`,\"--shiki-dark\":`#CE9178`},children:`karpenter.k8s.aws`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#569CD6`,\"--shiki-dark\":`#569CD6`},children:` kind`}),_jsx(_components.span,{style:{color:`#D4D4D4`,\"--shiki-dark\":`#D4D4D4`},children:`: `}),_jsx(_components.span,{style:{color:`#CE9178`,\"--shiki-dark\":`#CE9178`},children:`EC2NodeClass`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#569CD6`,\"--shiki-dark\":`#569CD6`},children:` name`}),_jsx(_components.span,{style:{color:`#D4D4D4`,\"--shiki-dark\":`#D4D4D4`},children:`: `}),_jsx(_components.span,{style:{color:`#CE9178`,\"--shiki-dark\":`#CE9178`},children:`gpu-default`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#569CD6`,\"--shiki-dark\":`#569CD6`},children:` taints`}),_jsx(_components.span,{style:{color:`#D4D4D4`,\"--shiki-dark\":`#D4D4D4`},children:`:`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#D4D4D4`,\"--shiki-dark\":`#D4D4D4`},children:` - `}),_jsx(_components.span,{style:{color:`#569CD6`,\"--shiki-dark\":`#569CD6`},children:`key`}),_jsx(_components.span,{style:{color:`#D4D4D4`,\"--shiki-dark\":`#D4D4D4`},children:`: `}),_jsx(_components.span,{style:{color:`#CE9178`,\"--shiki-dark\":`#CE9178`},children:`nvidia.com/gpu`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#569CD6`,\"--shiki-dark\":`#569CD6`},children:` effect`}),_jsx(_components.span,{style:{color:`#D4D4D4`,\"--shiki-dark\":`#D4D4D4`},children:`: `}),_jsx(_components.span,{style:{color:`#CE9178`,\"--shiki-dark\":`#CE9178`},children:`NoSchedule`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#569CD6`,\"--shiki-dark\":`#569CD6`},children:` requirements`}),_jsx(_components.span,{style:{color:`#D4D4D4`,\"--shiki-dark\":`#D4D4D4`},children:`:`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#D4D4D4`,\"--shiki-dark\":`#D4D4D4`},children:` - `}),_jsx(_components.span,{style:{color:`#569CD6`,\"--shiki-dark\":`#569CD6`},children:`key`}),_jsx(_components.span,{style:{color:`#D4D4D4`,\"--shiki-dark\":`#D4D4D4`},children:`: `}),_jsx(_components.span,{style:{color:`#CE9178`,\"--shiki-dark\":`#CE9178`},children:`karpenter.sh/capacity-type`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#569CD6`,\"--shiki-dark\":`#569CD6`},children:` operator`}),_jsx(_components.span,{style:{color:`#D4D4D4`,\"--shiki-dark\":`#D4D4D4`},children:`: `}),_jsx(_components.span,{style:{color:`#CE9178`,\"--shiki-dark\":`#CE9178`},children:`In`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#569CD6`,\"--shiki-dark\":`#569CD6`},children:` values`}),_jsx(_components.span,{style:{color:`#D4D4D4`,\"--shiki-dark\":`#D4D4D4`},children:`: [`}),_jsx(_components.span,{style:{color:`#CE9178`,\"--shiki-dark\":`#CE9178`},children:`spot`}),_jsx(_components.span,{style:{color:`#D4D4D4`,\"--shiki-dark\":`#D4D4D4`},children:`, `}),_jsx(_components.span,{style:{color:`#CE9178`,\"--shiki-dark\":`#CE9178`},children:`on-demand`}),_jsx(_components.span,{style:{color:`#D4D4D4`,\"--shiki-dark\":`#D4D4D4`},children:`]`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#D4D4D4`,\"--shiki-dark\":`#D4D4D4`},children:` - `}),_jsx(_components.span,{style:{color:`#569CD6`,\"--shiki-dark\":`#569CD6`},children:`key`}),_jsx(_components.span,{style:{color:`#D4D4D4`,\"--shiki-dark\":`#D4D4D4`},children:`: `}),_jsx(_components.span,{style:{color:`#CE9178`,\"--shiki-dark\":`#CE9178`},children:`kubernetes.io/arch`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#569CD6`,\"--shiki-dark\":`#569CD6`},children:` operator`}),_jsx(_components.span,{style:{color:`#D4D4D4`,\"--shiki-dark\":`#D4D4D4`},children:`: `}),_jsx(_components.span,{style:{color:`#CE9178`,\"--shiki-dark\":`#CE9178`},children:`In`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#569CD6`,\"--shiki-dark\":`#569CD6`},children:` values`}),_jsx(_components.span,{style:{color:`#D4D4D4`,\"--shiki-dark\":`#D4D4D4`},children:`: [`}),_jsx(_components.span,{style:{color:`#CE9178`,\"--shiki-dark\":`#CE9178`},children:`amd64`}),_jsx(_components.span,{style:{color:`#D4D4D4`,\"--shiki-dark\":`#D4D4D4`},children:`]`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#D4D4D4`,\"--shiki-dark\":`#D4D4D4`},children:` - `}),_jsx(_components.span,{style:{color:`#569CD6`,\"--shiki-dark\":`#569CD6`},children:`key`}),_jsx(_components.span,{style:{color:`#D4D4D4`,\"--shiki-dark\":`#D4D4D4`},children:`: `}),_jsx(_components.span,{style:{color:`#CE9178`,\"--shiki-dark\":`#CE9178`},children:`node.kubernetes.io/instance-type`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#569CD6`,\"--shiki-dark\":`#569CD6`},children:` operator`}),_jsx(_components.span,{style:{color:`#D4D4D4`,\"--shiki-dark\":`#D4D4D4`},children:`: `}),_jsx(_components.span,{style:{color:`#CE9178`,\"--shiki-dark\":`#CE9178`},children:`In`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#569CD6`,\"--shiki-dark\":`#569CD6`},children:` values`}),_jsx(_components.span,{style:{color:`#D4D4D4`,\"--shiki-dark\":`#D4D4D4`},children:`: [`}),_jsx(_components.span,{style:{color:`#CE9178`,\"--shiki-dark\":`#CE9178`},children:`g7e.2xlarge`}),_jsx(_components.span,{style:{color:`#D4D4D4`,\"--shiki-dark\":`#D4D4D4`},children:`]`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#D4D4D4`,\"--shiki-dark\":`#D4D4D4`},children:` - `}),_jsx(_components.span,{style:{color:`#569CD6`,\"--shiki-dark\":`#569CD6`},children:`key`}),_jsx(_components.span,{style:{color:`#D4D4D4`,\"--shiki-dark\":`#D4D4D4`},children:`: `}),_jsx(_components.span,{style:{color:`#CE9178`,\"--shiki-dark\":`#CE9178`},children:`topology.kubernetes.io/zone`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#569CD6`,\"--shiki-dark\":`#569CD6`},children:` operator`}),_jsx(_components.span,{style:{color:`#D4D4D4`,\"--shiki-dark\":`#D4D4D4`},children:`: `}),_jsx(_components.span,{style:{color:`#CE9178`,\"--shiki-dark\":`#CE9178`},children:`In`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#569CD6`,\"--shiki-dark\":`#569CD6`},children:` values`}),_jsx(_components.span,{style:{color:`#D4D4D4`,\"--shiki-dark\":`#D4D4D4`},children:`: [`}),_jsx(_components.span,{style:{color:`#CE9178`,\"--shiki-dark\":`#CE9178`},children:`us-east-1a`}),_jsx(_components.span,{style:{color:`#D4D4D4`,\"--shiki-dark\":`#D4D4D4`},children:`, `}),_jsx(_components.span,{style:{color:`#CE9178`,\"--shiki-dark\":`#CE9178`},children:`us-east-1b`}),_jsx(_components.span,{style:{color:`#D4D4D4`,\"--shiki-dark\":`#D4D4D4`},children:`, `}),_jsx(_components.span,{style:{color:`#CE9178`,\"--shiki-dark\":`#CE9178`},children:`us-east-1c`}),_jsx(_components.span,{style:{color:`#D4D4D4`,\"--shiki-dark\":`#D4D4D4`},children:`, `}),_jsx(_components.span,{style:{color:`#CE9178`,\"--shiki-dark\":`#CE9178`},children:`us-east-1d`}),_jsx(_components.span,{style:{color:`#D4D4D4`,\"--shiki-dark\":`#D4D4D4`},children:`]`})]}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{style:{color:`#D4D4D4`,\"--shiki-dark\":`#D4D4D4`},children:`---`})}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#569CD6`,\"--shiki-dark\":`#569CD6`},children:`apiVersion`}),_jsx(_components.span,{style:{color:`#D4D4D4`,\"--shiki-dark\":`#D4D4D4`},children:`: `}),_jsx(_components.span,{style:{color:`#CE9178`,\"--shiki-dark\":`#CE9178`},children:`karpenter.sh/v1alpha1`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#569CD6`,\"--shiki-dark\":`#569CD6`},children:`kind`}),_jsx(_components.span,{style:{color:`#D4D4D4`,\"--shiki-dark\":`#D4D4D4`},children:`: `}),_jsx(_components.span,{style:{color:`#CE9178`,\"--shiki-dark\":`#CE9178`},children:`NodeOverlay`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#569CD6`,\"--shiki-dark\":`#569CD6`},children:`metadata`}),_jsx(_components.span,{style:{color:`#D4D4D4`,\"--shiki-dark\":`#D4D4D4`},children:`:`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#569CD6`,\"--shiki-dark\":`#569CD6`},children:` name`}),_jsx(_components.span,{style:{color:`#D4D4D4`,\"--shiki-dark\":`#D4D4D4`},children:`: `}),_jsx(_components.span,{style:{color:`#CE9178`,\"--shiki-dark\":`#CE9178`},children:`g7e-2xlarge-mig-1g-24gb`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#569CD6`,\"--shiki-dark\":`#569CD6`},children:` labels`}),_jsx(_components.span,{style:{color:`#D4D4D4`,\"--shiki-dark\":`#D4D4D4`},children:`:`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#569CD6`,\"--shiki-dark\":`#569CD6`},children:` truefoundry.com/fractional-gpu`}),_jsx(_components.span,{style:{color:`#D4D4D4`,\"--shiki-dark\":`#D4D4D4`},children:`: `}),_jsx(_components.span,{style:{color:`#CE9178`,\"--shiki-dark\":`#CE9178`},children:`mig`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#569CD6`,\"--shiki-dark\":`#569CD6`},children:`spec`}),_jsx(_components.span,{style:{color:`#D4D4D4`,\"--shiki-dark\":`#D4D4D4`},children:`:`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#569CD6`,\"--shiki-dark\":`#569CD6`},children:` weight`}),_jsx(_components.span,{style:{color:`#D4D4D4`,\"--shiki-dark\":`#D4D4D4`},children:`: `}),_jsx(_components.span,{style:{color:`#B5CEA8`,\"--shiki-dark\":`#B5CEA8`},children:`10`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#569CD6`,\"--shiki-dark\":`#569CD6`},children:` capacity`}),_jsx(_components.span,{style:{color:`#D4D4D4`,\"--shiki-dark\":`#D4D4D4`},children:`:`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#569CD6`,\"--shiki-dark\":`#569CD6`},children:` nvidia.com/mig-1g.24gb`}),_jsx(_components.span,{style:{color:`#D4D4D4`,\"--shiki-dark\":`#D4D4D4`},children:`: `}),_jsx(_components.span,{style:{color:`#B5CEA8`,\"--shiki-dark\":`#B5CEA8`},children:`4`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#569CD6`,\"--shiki-dark\":`#569CD6`},children:` requirements`}),_jsx(_components.span,{style:{color:`#D4D4D4`,\"--shiki-dark\":`#D4D4D4`},children:`:`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#D4D4D4`,\"--shiki-dark\":`#D4D4D4`},children:` - `}),_jsx(_components.span,{style:{color:`#569CD6`,\"--shiki-dark\":`#569CD6`},children:`key`}),_jsx(_components.span,{style:{color:`#D4D4D4`,\"--shiki-dark\":`#D4D4D4`},children:`: `}),_jsx(_components.span,{style:{color:`#CE9178`,\"--shiki-dark\":`#CE9178`},children:`karpenter.sh/nodepool`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#569CD6`,\"--shiki-dark\":`#569CD6`},children:` operator`}),_jsx(_components.span,{style:{color:`#D4D4D4`,\"--shiki-dark\":`#D4D4D4`},children:`: `}),_jsx(_components.span,{style:{color:`#CE9178`,\"--shiki-dark\":`#CE9178`},children:`In`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#569CD6`,\"--shiki-dark\":`#569CD6`},children:` values`}),_jsx(_components.span,{style:{color:`#D4D4D4`,\"--shiki-dark\":`#D4D4D4`},children:`: [`}),_jsx(_components.span,{style:{color:`#CE9178`,\"--shiki-dark\":`#CE9178`},children:`g7e-2xlarge-mig-1g-24gb`}),_jsx(_components.span,{style:{color:`#D4D4D4`,\"--shiki-dark\":`#D4D4D4`},children:`]`})]}),`\n`]})})}),_jsxs(_components.p,{children:[`Another common example is H100 80GB on `,_jsx(_components.code,{children:`p5.4xlarge`}),` with `,_jsx(_components.code,{children:`1g.10gb`}),` (7 partitions):`]}),_jsx(CodeBlock,{filename:``,numberOfLines:`57`,language:`yaml`,children:_jsx(_components.pre,{className:`shiki shiki-themes dark-plus dark-plus`,style:{backgroundColor:`#0B0C0E`,\"--shiki-dark-bg\":`#0B0C0E`,color:`#D4D4D4`,\"--shiki-dark\":`#D4D4D4`},language:`yaml`,children:_jsxs(_components.code,{language:`yaml`,numberOfLines:`57`,children:[_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#569CD6`,\"--shiki-dark\":`#569CD6`},children:`apiVersion`}),_jsx(_components.span,{style:{color:`#D4D4D4`,\"--shiki-dark\":`#D4D4D4`},children:`: `}),_jsx(_components.span,{style:{color:`#CE9178`,\"--shiki-dark\":`#CE9178`},children:`karpenter.sh/v1`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#569CD6`,\"--shiki-dark\":`#569CD6`},children:`kind`}),_jsx(_components.span,{style:{color:`#D4D4D4`,\"--shiki-dark\":`#D4D4D4`},children:`: `}),_jsx(_components.span,{style:{color:`#CE9178`,\"--shiki-dark\":`#CE9178`},children:`NodePool`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#569CD6`,\"--shiki-dark\":`#569CD6`},children:`metadata`}),_jsx(_components.span,{style:{color:`#D4D4D4`,\"--shiki-dark\":`#D4D4D4`},children:`:`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#569CD6`,\"--shiki-dark\":`#569CD6`},children:` name`}),_jsx(_components.span,{style:{color:`#D4D4D4`,\"--shiki-dark\":`#D4D4D4`},children:`: `}),_jsx(_components.span,{style:{color:`#CE9178`,\"--shiki-dark\":`#CE9178`},children:`p5-4xlarge-mig-1g-10gb`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#569CD6`,\"--shiki-dark\":`#569CD6`},children:`spec`}),_jsx(_components.span,{style:{color:`#D4D4D4`,\"--shiki-dark\":`#D4D4D4`},children:`:`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#569CD6`,\"--shiki-dark\":`#569CD6`},children:` weight`}),_jsx(_components.span,{style:{color:`#D4D4D4`,\"--shiki-dark\":`#D4D4D4`},children:`: `}),_jsx(_components.span,{style:{color:`#B5CEA8`,\"--shiki-dark\":`#B5CEA8`},children:`10`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#569CD6`,\"--shiki-dark\":`#569CD6`},children:` disruption`}),_jsx(_components.span,{style:{color:`#D4D4D4`,\"--shiki-dark\":`#D4D4D4`},children:`:`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#569CD6`,\"--shiki-dark\":`#569CD6`},children:` consolidationPolicy`}),_jsx(_components.span,{style:{color:`#D4D4D4`,\"--shiki-dark\":`#D4D4D4`},children:`: `}),_jsx(_components.span,{style:{color:`#CE9178`,\"--shiki-dark\":`#CE9178`},children:`WhenEmptyOrUnderutilized`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#569CD6`,\"--shiki-dark\":`#569CD6`},children:` consolidateAfter`}),_jsx(_components.span,{style:{color:`#D4D4D4`,\"--shiki-dark\":`#D4D4D4`},children:`: `}),_jsx(_components.span,{style:{color:`#CE9178`,\"--shiki-dark\":`#CE9178`},children:`5m`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#569CD6`,\"--shiki-dark\":`#569CD6`},children:` budgets`}),_jsx(_components.span,{style:{color:`#D4D4D4`,\"--shiki-dark\":`#D4D4D4`},children:`:`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#D4D4D4`,\"--shiki-dark\":`#D4D4D4`},children:` - `}),_jsx(_components.span,{style:{color:`#569CD6`,\"--shiki-dark\":`#569CD6`},children:`nodes`}),_jsx(_components.span,{style:{color:`#D4D4D4`,\"--shiki-dark\":`#D4D4D4`},children:`: `}),_jsx(_components.span,{style:{color:`#CE9178`,\"--shiki-dark\":`#CE9178`},children:`10%`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#569CD6`,\"--shiki-dark\":`#569CD6`},children:` reasons`}),_jsx(_components.span,{style:{color:`#D4D4D4`,\"--shiki-dark\":`#D4D4D4`},children:`: [`}),_jsx(_components.span,{style:{color:`#CE9178`,\"--shiki-dark\":`#CE9178`},children:`Underutilized`}),_jsx(_components.span,{style:{color:`#D4D4D4`,\"--shiki-dark\":`#D4D4D4`},children:`, `}),_jsx(_components.span,{style:{color:`#CE9178`,\"--shiki-dark\":`#CE9178`},children:`Empty`}),_jsx(_components.span,{style:{color:`#D4D4D4`,\"--shiki-dark\":`#D4D4D4`},children:`]`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#D4D4D4`,\"--shiki-dark\":`#D4D4D4`},children:` - `}),_jsx(_components.span,{style:{color:`#569CD6`,\"--shiki-dark\":`#569CD6`},children:`nodes`}),_jsx(_components.span,{style:{color:`#D4D4D4`,\"--shiki-dark\":`#D4D4D4`},children:`: `}),_jsx(_components.span,{style:{color:`#CE9178`,\"--shiki-dark\":`#CE9178`},children:`\"1\"`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#569CD6`,\"--shiki-dark\":`#569CD6`},children:` reasons`}),_jsx(_components.span,{style:{color:`#D4D4D4`,\"--shiki-dark\":`#D4D4D4`},children:`: [`}),_jsx(_components.span,{style:{color:`#CE9178`,\"--shiki-dark\":`#CE9178`},children:`Drifted`}),_jsx(_components.span,{style:{color:`#D4D4D4`,\"--shiki-dark\":`#D4D4D4`},children:`]`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#569CD6`,\"--shiki-dark\":`#569CD6`},children:` template`}),_jsx(_components.span,{style:{color:`#D4D4D4`,\"--shiki-dark\":`#D4D4D4`},children:`:`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#569CD6`,\"--shiki-dark\":`#569CD6`},children:` metadata`}),_jsx(_components.span,{style:{color:`#D4D4D4`,\"--shiki-dark\":`#D4D4D4`},children:`:`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#569CD6`,\"--shiki-dark\":`#569CD6`},children:` labels`}),_jsx(_components.span,{style:{color:`#D4D4D4`,\"--shiki-dark\":`#D4D4D4`},children:`:`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#569CD6`,\"--shiki-dark\":`#569CD6`},children:` eks.amazonaws.com/nodegroup`}),_jsx(_components.span,{style:{color:`#D4D4D4`,\"--shiki-dark\":`#D4D4D4`},children:`: `}),_jsx(_components.span,{style:{color:`#CE9178`,\"--shiki-dark\":`#CE9178`},children:`p5-4xlarge-mig-1g-10gb`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#569CD6`,\"--shiki-dark\":`#569CD6`},children:` nvidia.com/gpu.deploy.operands`}),_jsx(_components.span,{style:{color:`#D4D4D4`,\"--shiki-dark\":`#D4D4D4`},children:`: `}),_jsx(_components.span,{style:{color:`#CE9178`,\"--shiki-dark\":`#CE9178`},children:`\"true\"`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#569CD6`,\"--shiki-dark\":`#569CD6`},children:` nvidia.com/mig.config`}),_jsx(_components.span,{style:{color:`#D4D4D4`,\"--shiki-dark\":`#D4D4D4`},children:`: `}),_jsx(_components.span,{style:{color:`#CE9178`,\"--shiki-dark\":`#CE9178`},children:`all-1g.10gb`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#569CD6`,\"--shiki-dark\":`#569CD6`},children:` spec`}),_jsx(_components.span,{style:{color:`#D4D4D4`,\"--shiki-dark\":`#D4D4D4`},children:`:`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#569CD6`,\"--shiki-dark\":`#569CD6`},children:` expireAfter`}),_jsx(_components.span,{style:{color:`#D4D4D4`,\"--shiki-dark\":`#D4D4D4`},children:`: `}),_jsx(_components.span,{style:{color:`#CE9178`,\"--shiki-dark\":`#CE9178`},children:`720h`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#569CD6`,\"--shiki-dark\":`#569CD6`},children:` nodeClassRef`}),_jsx(_components.span,{style:{color:`#D4D4D4`,\"--shiki-dark\":`#D4D4D4`},children:`:`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#569CD6`,\"--shiki-dark\":`#569CD6`},children:` group`}),_jsx(_components.span,{style:{color:`#D4D4D4`,\"--shiki-dark\":`#D4D4D4`},children:`: `}),_jsx(_components.span,{style:{color:`#CE9178`,\"--shiki-dark\":`#CE9178`},children:`karpenter.k8s.aws`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#569CD6`,\"--shiki-dark\":`#569CD6`},children:` kind`}),_jsx(_components.span,{style:{color:`#D4D4D4`,\"--shiki-dark\":`#D4D4D4`},children:`: `}),_jsx(_components.span,{style:{color:`#CE9178`,\"--shiki-dark\":`#CE9178`},children:`EC2NodeClass`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#569CD6`,\"--shiki-dark\":`#569CD6`},children:` name`}),_jsx(_components.span,{style:{color:`#D4D4D4`,\"--shiki-dark\":`#D4D4D4`},children:`: `}),_jsx(_components.span,{style:{color:`#CE9178`,\"--shiki-dark\":`#CE9178`},children:`gpu-default`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#569CD6`,\"--shiki-dark\":`#569CD6`},children:` taints`}),_jsx(_components.span,{style:{color:`#D4D4D4`,\"--shiki-dark\":`#D4D4D4`},children:`:`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#D4D4D4`,\"--shiki-dark\":`#D4D4D4`},children:` - `}),_jsx(_components.span,{style:{color:`#569CD6`,\"--shiki-dark\":`#569CD6`},children:`key`}),_jsx(_components.span,{style:{color:`#D4D4D4`,\"--shiki-dark\":`#D4D4D4`},children:`: `}),_jsx(_components.span,{style:{color:`#CE9178`,\"--shiki-dark\":`#CE9178`},children:`nvidia.com/gpu`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#569CD6`,\"--shiki-dark\":`#569CD6`},children:` effect`}),_jsx(_components.span,{style:{color:`#D4D4D4`,\"--shiki-dark\":`#D4D4D4`},children:`: `}),_jsx(_components.span,{style:{color:`#CE9178`,\"--shiki-dark\":`#CE9178`},children:`NoSchedule`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#569CD6`,\"--shiki-dark\":`#569CD6`},children:` requirements`}),_jsx(_components.span,{style:{color:`#D4D4D4`,\"--shiki-dark\":`#D4D4D4`},children:`:`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#D4D4D4`,\"--shiki-dark\":`#D4D4D4`},children:` - `}),_jsx(_components.span,{style:{color:`#569CD6`,\"--shiki-dark\":`#569CD6`},children:`key`}),_jsx(_components.span,{style:{color:`#D4D4D4`,\"--shiki-dark\":`#D4D4D4`},children:`: `}),_jsx(_components.span,{style:{color:`#CE9178`,\"--shiki-dark\":`#CE9178`},children:`karpenter.sh/capacity-type`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#569CD6`,\"--shiki-dark\":`#569CD6`},children:` operator`}),_jsx(_components.span,{style:{color:`#D4D4D4`,\"--shiki-dark\":`#D4D4D4`},children:`: `}),_jsx(_components.span,{style:{color:`#CE9178`,\"--shiki-dark\":`#CE9178`},children:`In`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#569CD6`,\"--shiki-dark\":`#569CD6`},children:` values`}),_jsx(_components.span,{style:{color:`#D4D4D4`,\"--shiki-dark\":`#D4D4D4`},children:`: [`}),_jsx(_components.span,{style:{color:`#CE9178`,\"--shiki-dark\":`#CE9178`},children:`spot`}),_jsx(_components.span,{style:{color:`#D4D4D4`,\"--shiki-dark\":`#D4D4D4`},children:`, `}),_jsx(_components.span,{style:{color:`#CE9178`,\"--shiki-dark\":`#CE9178`},children:`on-demand`}),_jsx(_components.span,{style:{color:`#D4D4D4`,\"--shiki-dark\":`#D4D4D4`},children:`]`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#D4D4D4`,\"--shiki-dark\":`#D4D4D4`},children:` - `}),_jsx(_components.span,{style:{color:`#569CD6`,\"--shiki-dark\":`#569CD6`},children:`key`}),_jsx(_components.span,{style:{color:`#D4D4D4`,\"--shiki-dark\":`#D4D4D4`},children:`: `}),_jsx(_components.span,{style:{color:`#CE9178`,\"--shiki-dark\":`#CE9178`},children:`kubernetes.io/arch`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#569CD6`,\"--shiki-dark\":`#569CD6`},children:` operator`}),_jsx(_components.span,{style:{color:`#D4D4D4`,\"--shiki-dark\":`#D4D4D4`},children:`: `}),_jsx(_components.span,{style:{color:`#CE9178`,\"--shiki-dark\":`#CE9178`},children:`In`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#569CD6`,\"--shiki-dark\":`#569CD6`},children:` values`}),_jsx(_components.span,{style:{color:`#D4D4D4`,\"--shiki-dark\":`#D4D4D4`},children:`: [`}),_jsx(_components.span,{style:{color:`#CE9178`,\"--shiki-dark\":`#CE9178`},children:`amd64`}),_jsx(_components.span,{style:{color:`#D4D4D4`,\"--shiki-dark\":`#D4D4D4`},children:`]`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#D4D4D4`,\"--shiki-dark\":`#D4D4D4`},children:` - `}),_jsx(_components.span,{style:{color:`#569CD6`,\"--shiki-dark\":`#569CD6`},children:`key`}),_jsx(_components.span,{style:{color:`#D4D4D4`,\"--shiki-dark\":`#D4D4D4`},children:`: `}),_jsx(_components.span,{style:{color:`#CE9178`,\"--shiki-dark\":`#CE9178`},children:`node.kubernetes.io/instance-type`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#569CD6`,\"--shiki-dark\":`#569CD6`},children:` operator`}),_jsx(_components.span,{style:{color:`#D4D4D4`,\"--shiki-dark\":`#D4D4D4`},children:`: `}),_jsx(_components.span,{style:{color:`#CE9178`,\"--shiki-dark\":`#CE9178`},children:`In`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#569CD6`,\"--shiki-dark\":`#569CD6`},children:` values`}),_jsx(_components.span,{style:{color:`#D4D4D4`,\"--shiki-dark\":`#D4D4D4`},children:`: [`}),_jsx(_components.span,{style:{color:`#CE9178`,\"--shiki-dark\":`#CE9178`},children:`p5.4xlarge`}),_jsx(_components.span,{style:{color:`#D4D4D4`,\"--shiki-dark\":`#D4D4D4`},children:`]`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#D4D4D4`,\"--shiki-dark\":`#D4D4D4`},children:` - `}),_jsx(_components.span,{style:{color:`#569CD6`,\"--shiki-dark\":`#569CD6`},children:`key`}),_jsx(_components.span,{style:{color:`#D4D4D4`,\"--shiki-dark\":`#D4D4D4`},children:`: `}),_jsx(_components.span,{style:{color:`#CE9178`,\"--shiki-dark\":`#CE9178`},children:`topology.kubernetes.io/zone`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#569CD6`,\"--shiki-dark\":`#569CD6`},children:` operator`}),_jsx(_components.span,{style:{color:`#D4D4D4`,\"--shiki-dark\":`#D4D4D4`},children:`: `}),_jsx(_components.span,{style:{color:`#CE9178`,\"--shiki-dark\":`#CE9178`},children:`In`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#569CD6`,\"--shiki-dark\":`#569CD6`},children:` values`}),_jsx(_components.span,{style:{color:`#D4D4D4`,\"--shiki-dark\":`#D4D4D4`},children:`: [`}),_jsx(_components.span,{style:{color:`#CE9178`,\"--shiki-dark\":`#CE9178`},children:`us-east-1a`}),_jsx(_components.span,{style:{color:`#D4D4D4`,\"--shiki-dark\":`#D4D4D4`},children:`, `}),_jsx(_components.span,{style:{color:`#CE9178`,\"--shiki-dark\":`#CE9178`},children:`us-east-1b`}),_jsx(_components.span,{style:{color:`#D4D4D4`,\"--shiki-dark\":`#D4D4D4`},children:`, `}),_jsx(_components.span,{style:{color:`#CE9178`,\"--shiki-dark\":`#CE9178`},children:`us-east-1c`}),_jsx(_components.span,{style:{color:`#D4D4D4`,\"--shiki-dark\":`#D4D4D4`},children:`, `}),_jsx(_components.span,{style:{color:`#CE9178`,\"--shiki-dark\":`#CE9178`},children:`us-east-1d`}),_jsx(_components.span,{style:{color:`#D4D4D4`,\"--shiki-dark\":`#D4D4D4`},children:`]`})]}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{style:{color:`#D4D4D4`,\"--shiki-dark\":`#D4D4D4`},children:`---`})}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#569CD6`,\"--shiki-dark\":`#569CD6`},children:`apiVersion`}),_jsx(_components.span,{style:{color:`#D4D4D4`,\"--shiki-dark\":`#D4D4D4`},children:`: `}),_jsx(_components.span,{style:{color:`#CE9178`,\"--shiki-dark\":`#CE9178`},children:`karpenter.sh/v1alpha1`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#569CD6`,\"--shiki-dark\":`#569CD6`},children:`kind`}),_jsx(_components.span,{style:{color:`#D4D4D4`,\"--shiki-dark\":`#D4D4D4`},children:`: `}),_jsx(_components.span,{style:{color:`#CE9178`,\"--shiki-dark\":`#CE9178`},children:`NodeOverlay`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#569CD6`,\"--shiki-dark\":`#569CD6`},children:`metadata`}),_jsx(_components.span,{style:{color:`#D4D4D4`,\"--shiki-dark\":`#D4D4D4`},children:`:`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#569CD6`,\"--shiki-dark\":`#569CD6`},children:` name`}),_jsx(_components.span,{style:{color:`#D4D4D4`,\"--shiki-dark\":`#D4D4D4`},children:`: `}),_jsx(_components.span,{style:{color:`#CE9178`,\"--shiki-dark\":`#CE9178`},children:`p5-4xlarge-mig-1g-10gb`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#569CD6`,\"--shiki-dark\":`#569CD6`},children:` labels`}),_jsx(_components.span,{style:{color:`#D4D4D4`,\"--shiki-dark\":`#D4D4D4`},children:`:`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#569CD6`,\"--shiki-dark\":`#569CD6`},children:` truefoundry.com/fractional-gpu`}),_jsx(_components.span,{style:{color:`#D4D4D4`,\"--shiki-dark\":`#D4D4D4`},children:`: `}),_jsx(_components.span,{style:{color:`#CE9178`,\"--shiki-dark\":`#CE9178`},children:`mig`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#569CD6`,\"--shiki-dark\":`#569CD6`},children:`spec`}),_jsx(_components.span,{style:{color:`#D4D4D4`,\"--shiki-dark\":`#D4D4D4`},children:`:`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#569CD6`,\"--shiki-dark\":`#569CD6`},children:` weight`}),_jsx(_components.span,{style:{color:`#D4D4D4`,\"--shiki-dark\":`#D4D4D4`},children:`: `}),_jsx(_components.span,{style:{color:`#B5CEA8`,\"--shiki-dark\":`#B5CEA8`},children:`10`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#569CD6`,\"--shiki-dark\":`#569CD6`},children:` capacity`}),_jsx(_components.span,{style:{color:`#D4D4D4`,\"--shiki-dark\":`#D4D4D4`},children:`:`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#569CD6`,\"--shiki-dark\":`#569CD6`},children:` nvidia.com/mig-1g.10gb`}),_jsx(_components.span,{style:{color:`#D4D4D4`,\"--shiki-dark\":`#D4D4D4`},children:`: `}),_jsx(_components.span,{style:{color:`#B5CEA8`,\"--shiki-dark\":`#B5CEA8`},children:`7`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#569CD6`,\"--shiki-dark\":`#569CD6`},children:` requirements`}),_jsx(_components.span,{style:{color:`#D4D4D4`,\"--shiki-dark\":`#D4D4D4`},children:`:`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#D4D4D4`,\"--shiki-dark\":`#D4D4D4`},children:` - `}),_jsx(_components.span,{style:{color:`#569CD6`,\"--shiki-dark\":`#569CD6`},children:`key`}),_jsx(_components.span,{style:{color:`#D4D4D4`,\"--shiki-dark\":`#D4D4D4`},children:`: `}),_jsx(_components.span,{style:{color:`#CE9178`,\"--shiki-dark\":`#CE9178`},children:`karpenter.sh/nodepool`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#569CD6`,\"--shiki-dark\":`#569CD6`},children:` operator`}),_jsx(_components.span,{style:{color:`#D4D4D4`,\"--shiki-dark\":`#D4D4D4`},children:`: `}),_jsx(_components.span,{style:{color:`#CE9178`,\"--shiki-dark\":`#CE9178`},children:`In`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#569CD6`,\"--shiki-dark\":`#569CD6`},children:` values`}),_jsx(_components.span,{style:{color:`#D4D4D4`,\"--shiki-dark\":`#D4D4D4`},children:`: [`}),_jsx(_components.span,{style:{color:`#CE9178`,\"--shiki-dark\":`#CE9178`},children:`p5-4xlarge-mig-1g-10gb`}),_jsx(_components.span,{style:{color:`#D4D4D4`,\"--shiki-dark\":`#D4D4D4`},children:`]`})]}),`\n`]})})}),_jsxs(Info,{children:[_jsx(_components.p,{children:`Requirements TrueFoundry enforces when syncing:`}),_jsxs(_components.ul,{children:[`\n`,_jsxs(_components.li,{children:[`NodeOverlay must set `,_jsx(_components.code,{children:`truefoundry.com/fractional-gpu: mig`})]}),`\n`,_jsxs(_components.li,{children:[`Capacity key must be `,_jsx(_components.code,{children:`nvidia.com/mig-\u003cprofile>`}),` (for example `,_jsx(_components.code,{children:`nvidia.com/mig-1g.24gb`}),`)`]}),`\n`,_jsxs(_components.li,{children:[`Capacity value must equal `,_jsx(_components.strong,{children:`(MIG partitions per GPU) × (GPU count on the instance)`}),`. `,_jsx(_components.code,{children:`g7e.2xlarge`}),` has 1 GPU and `,_jsx(_components.code,{children:`1g.24gb`}),` has 4 partitions → `,_jsx(_components.code,{children:`4`}),`. `,_jsx(_components.code,{children:`p5.4xlarge`}),` has 1 GPU and `,_jsx(_components.code,{children:`1g.10gb`}),` has 7 partitions → `,_jsx(_components.code,{children:`7`}),`. `,_jsx(_components.code,{children:`p5.48xlarge`}),` has 8 GPUs and `,_jsx(_components.code,{children:`1g.10gb`}),` has 7 partitions → `,_jsx(_components.code,{children:`56`})]}),`\n`,_jsx(_components.li,{children:`NodePool must resolve to exactly one GPU instance type`}),`\n`,_jsxs(_components.li,{children:[_jsx(_components.code,{children:`nvidia.com/mig.config`}),` on the NodePool must match a MIG config name from `,_jsx(_components.code,{children:`tfy-gpu-operator`})]}),`\n`]})]}),_jsxs(_components.p,{children:[`Deploy the updated `,_jsx(_components.code,{children:`tfy-karpenter-config`}),` application.`]}),_jsx(_components.p,{children:_jsx(_components.strong,{children:`3. Sync Cluster`})}),_jsxs(_components.p,{children:[`Go to `,_jsx(_components.strong,{children:`Platform → Clusters`}),` and click `,_jsx(_components.strong,{children:`Sync`}),` on the cluster. Sync also runs periodically; a manual sync makes the new NodePools show up sooner.`]}),_jsx(Frame,{caption:`Sync Cluster`,children:_jsx(OptimizedImage,{src:`https://mintcdn.com/truefoundry/fDI2fd3aQvGCJmLJ/images/docs/using-fractional-gpus-sync-cluster.png?fit=max\u0026auto=format\u0026n=fDI2fd3aQvGCJmLJ\u0026q=85\u0026s=5ce3b17c6ae0bd88b649670a5a23ae5e`,alt:`Clusters page with the Sync Cluster button highlighted`,\"data-og-width\":`3020`,width:`3020`,\"data-og-height\":`1084`,height:`1084`,\"data-path\":`images/docs/using-fractional-gpus-sync-cluster.png`,\"data-optimize\":`true`,\"data-opv\":`3`,srcset:`https://mintcdn.com/truefoundry/fDI2fd3aQvGCJmLJ/images/docs/using-fractional-gpus-sync-cluster.png?w=280\u0026fit=max\u0026auto=format\u0026n=fDI2fd3aQvGCJmLJ\u0026q=85\u0026s=e5e1b538b57a7e17560000c06e1a9b25 280w, https://mintcdn.com/truefoundry/fDI2fd3aQvGCJmLJ/images/docs/using-fractional-gpus-sync-cluster.png?w=560\u0026fit=max\u0026auto=format\u0026n=fDI2fd3aQvGCJmLJ\u0026q=85\u0026s=872ea43c7cd712a2f9c83d4d364b1b06 560w, https://mintcdn.com/truefoundry/fDI2fd3aQvGCJmLJ/images/docs/using-fractional-gpus-sync-cluster.png?w=840\u0026fit=max\u0026auto=format\u0026n=fDI2fd3aQvGCJmLJ\u0026q=85\u0026s=c41bb83e1396b3a9018ef53e23565e08 840w, https://mintcdn.com/truefoundry/fDI2fd3aQvGCJmLJ/images/docs/using-fractional-gpus-sync-cluster.png?w=1100\u0026fit=max\u0026auto=format\u0026n=fDI2fd3aQvGCJmLJ\u0026q=85\u0026s=e8efe86ed95b282b7123f1b21fb17392 1100w, https://mintcdn.com/truefoundry/fDI2fd3aQvGCJmLJ/images/docs/using-fractional-gpus-sync-cluster.png?w=1650\u0026fit=max\u0026auto=format\u0026n=fDI2fd3aQvGCJmLJ\u0026q=85\u0026s=6aef7db9299c8111a0e4d1f2a65e5dce 1650w, https://mintcdn.com/truefoundry/fDI2fd3aQvGCJmLJ/images/docs/using-fractional-gpus-sync-cluster.png?w=2500\u0026fit=max\u0026auto=format\u0026n=fDI2fd3aQvGCJmLJ\u0026q=85\u0026s=54f0f79a1436a563ae0265d9a17b091b 2500w`})}),_jsx(_components.p,{children:`After sync, the fractional GPU NodePools appear under the cluster’s Node Pools:`}),_jsx(Frame,{caption:`Fractional GPU NodePools after sync`,children:_jsx(OptimizedImage,{src:`https://mintcdn.com/truefoundry/b3o-7tzF9YXHx_sQ/images/docs/using-fractional-gpus-aws-nodepools.png?fit=max\u0026auto=format\u0026n=b3o-7tzF9YXHx_sQ\u0026q=85\u0026s=edb62e9c645448a9ecd01b7bfd90774f`,alt:`Node Pools tab showing timesliced and MIG NodePools`,\"data-og-width\":`3018`,width:`3018`,\"data-og-height\":`934`,height:`934`,\"data-path\":`images/docs/using-fractional-gpus-aws-nodepools.png`,\"data-optimize\":`true`,\"data-opv\":`3`,srcset:`https://mintcdn.com/truefoundry/b3o-7tzF9YXHx_sQ/images/docs/using-fractional-gpus-aws-nodepools.png?w=280\u0026fit=max\u0026auto=format\u0026n=b3o-7tzF9YXHx_sQ\u0026q=85\u0026s=7b9813ac1120451a13aac9238ee8d7bf 280w, https://mintcdn.com/truefoundry/b3o-7tzF9YXHx_sQ/images/docs/using-fractional-gpus-aws-nodepools.png?w=560\u0026fit=max\u0026auto=format\u0026n=b3o-7tzF9YXHx_sQ\u0026q=85\u0026s=efec322fb073869852fd71a65a641acf 560w, https://mintcdn.com/truefoundry/b3o-7tzF9YXHx_sQ/images/docs/using-fractional-gpus-aws-nodepools.png?w=840\u0026fit=max\u0026auto=format\u0026n=b3o-7tzF9YXHx_sQ\u0026q=85\u0026s=d9377e5adb1b18377ae52a9a2a2c0fd2 840w, https://mintcdn.com/truefoundry/b3o-7tzF9YXHx_sQ/images/docs/using-fractional-gpus-aws-nodepools.png?w=1100\u0026fit=max\u0026auto=format\u0026n=b3o-7tzF9YXHx_sQ\u0026q=85\u0026s=91aa31a242be7e0267bdda5b8430606e 1100w, https://mintcdn.com/truefoundry/b3o-7tzF9YXHx_sQ/images/docs/using-fractional-gpus-aws-nodepools.png?w=1650\u0026fit=max\u0026auto=format\u0026n=b3o-7tzF9YXHx_sQ\u0026q=85\u0026s=602eebfd17573bbbe2d81be4f2ec49b8 1650w, https://mintcdn.com/truefoundry/b3o-7tzF9YXHx_sQ/images/docs/using-fractional-gpus-aws-nodepools.png?w=2500\u0026fit=max\u0026auto=format\u0026n=b3o-7tzF9YXHx_sQ\u0026q=85\u0026s=4108af704ebed9de099a8ea44691d4da 2500w`})})]})]})]}),_jsxs(Step,{title:`Deploy your workload on the MIG nodepool`,children:[_jsxs(_components.p,{children:[`Once you have created the nodepool, you will be able to see the MIG nodepool in the available nodepools in `,_jsx(_components.code,{children:`Resources`}),` section in deployment.`]}),_jsx(Warning,{children:_jsxs(_components.p,{children:[`It might take a few minutes for the newly created nodepool to be visible on the TrueFoundry UI. You can force sync the nodepools by going to `,_jsx(_components.strong,{children:`Platform → Clusters → Sync`}),`.`]})}),_jsx(_components.p,{children:`To deploy a workload that utilizes fractional GPU, start deploying your service/job on truefoundry and in the “Resources” section, select nodepool selector.\nYou can now see the Fractional GPUs on the UI which you can select (as shown below)`}),_jsx(Frame,{caption:`Selecting a MIG GPU in Resources`,children:_jsx(OptimizedImage,{src:`https://mintcdn.com/truefoundry/fDI2fd3aQvGCJmLJ/images/docs/using-fractional-gpus-resources-mig.png?fit=max\u0026auto=format\u0026n=fDI2fd3aQvGCJmLJ\u0026q=85\u0026s=5cb7070b9a201516296891aca30ed0ad`,alt:`Resources section with RTX PRO 6000 MIG 1g.24gb selected`,\"data-og-width\":`1770`,width:`1770`,\"data-og-height\":`1068`,height:`1068`,\"data-path\":`images/docs/using-fractional-gpus-resources-mig.png`,\"data-optimize\":`true`,\"data-opv\":`3`,srcset:`https://mintcdn.com/truefoundry/fDI2fd3aQvGCJmLJ/images/docs/using-fractional-gpus-resources-mig.png?w=280\u0026fit=max\u0026auto=format\u0026n=fDI2fd3aQvGCJmLJ\u0026q=85\u0026s=8600034392c4739dc17c708cf56a0abd 280w, https://mintcdn.com/truefoundry/fDI2fd3aQvGCJmLJ/images/docs/using-fractional-gpus-resources-mig.png?w=560\u0026fit=max\u0026auto=format\u0026n=fDI2fd3aQvGCJmLJ\u0026q=85\u0026s=17f579b9cd75415817544282d7c74aa7 560w, https://mintcdn.com/truefoundry/fDI2fd3aQvGCJmLJ/images/docs/using-fractional-gpus-resources-mig.png?w=840\u0026fit=max\u0026auto=format\u0026n=fDI2fd3aQvGCJmLJ\u0026q=85\u0026s=802872cd0e7ee029955e7175bba9f4de 840w, https://mintcdn.com/truefoundry/fDI2fd3aQvGCJmLJ/images/docs/using-fractional-gpus-resources-mig.png?w=1100\u0026fit=max\u0026auto=format\u0026n=fDI2fd3aQvGCJmLJ\u0026q=85\u0026s=42f60e014606af61c30e9835303a89e5 1100w, https://mintcdn.com/truefoundry/fDI2fd3aQvGCJmLJ/images/docs/using-fractional-gpus-resources-mig.png?w=1650\u0026fit=max\u0026auto=format\u0026n=fDI2fd3aQvGCJmLJ\u0026q=85\u0026s=75ea91e781beb8009ea62711da2ac201 1650w, https://mintcdn.com/truefoundry/fDI2fd3aQvGCJmLJ/images/docs/using-fractional-gpus-resources-mig.png?w=2500\u0026fit=max\u0026auto=format\u0026n=fDI2fd3aQvGCJmLJ\u0026q=85\u0026s=fb704a1adb76ca875e15028b522c6d87 2500w`})})]})]}),`\n`,_jsx(Heading,{level:`2`,id:`deploying-on-fractional-gpus-via-timeslicing`,children:`Deploying on Fractional GPUs via Timeslicing`}),`\n`,_jsxs(Steps,{children:[_jsx(Step,{title:`Create a Nodepool with Timeslicing enabled`,children:_jsxs(Tabs,{children:[_jsxs(Tab,{title:`Azure`,id:`azure-2`,children:[_jsxs(_components.p,{children:[`Create a Nodepool with `,_jsx(_components.code,{children:`device-plugin.config`}),` pointing to the correct time-slicing config with Azure CLI.`]}),_jsx(CodeBlock,{filename:`Shell`,lines:`true`,numberOfLines:`18`,language:`shellscript`,children:_jsx(_components.pre,{className:`shiki shiki-themes dark-plus dark-plus`,style:{backgroundColor:`#0B0C0E`,\"--shiki-dark-bg\":`#0B0C0E`,color:`#D4D4D4`,\"--shiki-dark\":`#D4D4D4`},language:`shellscript`,children:_jsxs(_components.code,{language:`shellscript`,numberOfLines:`18`,children:[_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#DCDCAA`,\"--shiki-dark\":`#DCDCAA`},children:`az`}),_jsx(_components.span,{style:{color:`#CE9178`,\"--shiki-dark\":`#CE9178`},children:` aks`}),_jsx(_components.span,{style:{color:`#CE9178`,\"--shiki-dark\":`#CE9178`},children:` nodepool`}),_jsx(_components.span,{style:{color:`#CE9178`,\"--shiki-dark\":`#CE9178`},children:` add`}),_jsx(_components.span,{style:{color:`#D7BA7D`,\"--shiki-dark\":`#D7BA7D`},children:` \\\\`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#D4D4D4`,\"--shiki-dark\":`#D4D4D4`},children:`--cluster-name \u003c`}),_jsx(_components.span,{style:{color:`#CE9178`,\"--shiki-dark\":`#CE9178`},children:`your`}),_jsx(_components.span,{style:{color:`#CE9178`,\"--shiki-dark\":`#CE9178`},children:` cluster`}),_jsx(_components.span,{style:{color:`#CE9178`,\"--shiki-dark\":`#CE9178`},children:` nam`}),_jsx(_components.span,{style:{color:`#D4D4D4`,\"--shiki-dark\":`#D4D4D4`},children:`e> `}),_jsx(_components.span,{style:{color:`#D7BA7D`,\"--shiki-dark\":`#D7BA7D`},children:`\\\\`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#D4D4D4`,\"--shiki-dark\":`#D4D4D4`},children:`--resource-group \u003c`}),_jsx(_components.span,{style:{color:`#CE9178`,\"--shiki-dark\":`#CE9178`},children:`your`}),_jsx(_components.span,{style:{color:`#CE9178`,\"--shiki-dark\":`#CE9178`},children:` resource`}),_jsx(_components.span,{style:{color:`#CE9178`,\"--shiki-dark\":`#CE9178`},children:` grou`}),_jsx(_components.span,{style:{color:`#D4D4D4`,\"--shiki-dark\":`#D4D4D4`},children:`p> `}),_jsx(_components.span,{style:{color:`#D7BA7D`,\"--shiki-dark\":`#D7BA7D`},children:`\\\\`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#D4D4D4`,\"--shiki-dark\":`#D4D4D4`},children:`--no-wait `}),_jsx(_components.span,{style:{color:`#D7BA7D`,\"--shiki-dark\":`#D7BA7D`},children:`\\\\`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#D4D4D4`,\"--shiki-dark\":`#D4D4D4`},children:`--enable-cluster-autoscaler `}),_jsx(_components.span,{style:{color:`#D7BA7D`,\"--shiki-dark\":`#D7BA7D`},children:`\\\\`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#D4D4D4`,\"--shiki-dark\":`#D4D4D4`},children:`--eviction-policy `}),_jsx(_components.span,{style:{color:`#CE9178`,\"--shiki-dark\":`#CE9178`},children:`Delete`}),_jsx(_components.span,{style:{color:`#D7BA7D`,\"--shiki-dark\":`#D7BA7D`},children:` \\\\`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#D4D4D4`,\"--shiki-dark\":`#D4D4D4`},children:`--node-count `}),_jsx(_components.span,{style:{color:`#B5CEA8`,\"--shiki-dark\":`#B5CEA8`},children:`0`}),_jsx(_components.span,{style:{color:`#D7BA7D`,\"--shiki-dark\":`#D7BA7D`},children:` \\\\`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#D4D4D4`,\"--shiki-dark\":`#D4D4D4`},children:`--max-count `}),_jsx(_components.span,{style:{color:`#B5CEA8`,\"--shiki-dark\":`#B5CEA8`},children:`20`}),_jsx(_components.span,{style:{color:`#D7BA7D`,\"--shiki-dark\":`#D7BA7D`},children:` \\\\`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#D4D4D4`,\"--shiki-dark\":`#D4D4D4`},children:`--min-count `}),_jsx(_components.span,{style:{color:`#B5CEA8`,\"--shiki-dark\":`#B5CEA8`},children:`0`}),_jsx(_components.span,{style:{color:`#D7BA7D`,\"--shiki-dark\":`#D7BA7D`},children:` \\\\`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#D4D4D4`,\"--shiki-dark\":`#D4D4D4`},children:`--node-osdisk-size `}),_jsx(_components.span,{style:{color:`#B5CEA8`,\"--shiki-dark\":`#B5CEA8`},children:`200`}),_jsx(_components.span,{style:{color:`#D7BA7D`,\"--shiki-dark\":`#D7BA7D`},children:` \\\\`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#D4D4D4`,\"--shiki-dark\":`#D4D4D4`},children:`--scale-down-mode `}),_jsx(_components.span,{style:{color:`#CE9178`,\"--shiki-dark\":`#CE9178`},children:`Delete`}),_jsx(_components.span,{style:{color:`#D7BA7D`,\"--shiki-dark\":`#D7BA7D`},children:` \\\\`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#D4D4D4`,\"--shiki-dark\":`#D4D4D4`},children:`--os-type `}),_jsx(_components.span,{style:{color:`#CE9178`,\"--shiki-dark\":`#CE9178`},children:`Linux`}),_jsx(_components.span,{style:{color:`#D7BA7D`,\"--shiki-dark\":`#D7BA7D`},children:` \\\\`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#D4D4D4`,\"--shiki-dark\":`#D4D4D4`},children:`--node-taints `}),_jsx(_components.span,{style:{color:`#CE9178`,\"--shiki-dark\":`#CE9178`},children:`\"nvidia.com/gpu=Present:NoSchedule\"`}),_jsx(_components.span,{style:{color:`#D7BA7D`,\"--shiki-dark\":`#D7BA7D`},children:` \\\\`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#D4D4D4`,\"--shiki-dark\":`#D4D4D4`},children:`--name `}),_jsx(_components.span,{style:{color:`#CE9178`,\"--shiki-dark\":`#CE9178`},children:`a100mig7`}),_jsx(_components.span,{style:{color:`#D7BA7D`,\"--shiki-dark\":`#D7BA7D`},children:` \\\\`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#D4D4D4`,\"--shiki-dark\":`#D4D4D4`},children:`--node-vm-size `}),_jsx(_components.span,{style:{color:`#CE9178`,\"--shiki-dark\":`#CE9178`},children:`Standard_NC24ads_A100_v4`}),_jsx(_components.span,{style:{color:`#D7BA7D`,\"--shiki-dark\":`#D7BA7D`},children:` \\\\`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#D4D4D4`,\"--shiki-dark\":`#D4D4D4`},children:`--priority `}),_jsx(_components.span,{style:{color:`#CE9178`,\"--shiki-dark\":`#CE9178`},children:`Spot`}),_jsx(_components.span,{style:{color:`#D7BA7D`,\"--shiki-dark\":`#D7BA7D`},children:` \\\\`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#D4D4D4`,\"--shiki-dark\":`#D4D4D4`},children:`--os-sku `}),_jsx(_components.span,{style:{color:`#CE9178`,\"--shiki-dark\":`#CE9178`},children:`Ubuntu`}),_jsx(_components.span,{style:{color:`#D7BA7D`,\"--shiki-dark\":`#D7BA7D`},children:` \\\\`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#D4D4D4`,\"--shiki-dark\":`#D4D4D4`},children:`--labels `}),_jsx(_components.span,{style:{color:`#CE9178`,\"--shiki-dark\":`#CE9178`},children:`nvidia.com/device-plugin.config=time-sliced-10`})]}),`\n`]})})})]}),_jsxs(Tab,{title:`GCP`,id:`gcp-2`,children:[_jsx(_components.p,{children:`Here’s the command to create a nodepool with timeslicing enabled on GCP.`}),_jsx(CodeBlock,{filename:``,lines:`true`,numberOfLines:`11`,language:`shellscript`,children:_jsx(_components.pre,{className:`shiki shiki-themes dark-plus dark-plus`,style:{backgroundColor:`#0B0C0E`,\"--shiki-dark-bg\":`#0B0C0E`,color:`#D4D4D4`,\"--shiki-dark\":`#D4D4D4`},language:`shellscript`,children:_jsxs(_components.code,{language:`shellscript`,numberOfLines:`11`,children:[_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#DCDCAA`,\"--shiki-dark\":`#DCDCAA`},children:`gcloud`}),_jsx(_components.span,{style:{color:`#CE9178`,\"--shiki-dark\":`#CE9178`},children:` container`}),_jsx(_components.span,{style:{color:`#CE9178`,\"--shiki-dark\":`#CE9178`},children:` node-pools`}),_jsx(_components.span,{style:{color:`#CE9178`,\"--shiki-dark\":`#CE9178`},children:` create`}),_jsx(_components.span,{style:{color:`#CE9178`,\"--shiki-dark\":`#CE9178`},children:` a100-40-frac-10`}),_jsx(_components.span,{style:{color:`#D7BA7D`,\"--shiki-dark\":`#D7BA7D`},children:` \\\\ `}),_jsx(_components.span,{style:{color:`#CE9178`,\"--shiki-dark\":`#CE9178`},children:` `}),_jsx(_components.span,{style:{color:`#CE9178`,\"--shiki-dark\":`#CE9178`},children:` ✔`}),_jsx(_components.span,{style:{color:`#CE9178`,\"--shiki-dark\":`#CE9178`},children:` `}),_jsx(_components.span,{style:{color:`#CE9178`,\"--shiki-dark\":`#CE9178`},children:` `})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#9CDCFE`,\"--shiki-dark\":`#9CDCFE`},children:` --project`}),_jsx(_components.span,{style:{color:`#D4D4D4`,\"--shiki-dark\":`#D4D4D4`},children:`=`}),_jsx(_components.span,{style:{color:`#CE9178`,\"--shiki-dark\":`#CE9178`},children:`tfy-devtest`}),_jsx(_components.span,{style:{color:`#DCDCAA`,\"--shiki-dark\":`#DCDCAA`},children:` \\\\`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#569CD6`,\"--shiki-dark\":`#569CD6`},children:` --region=us-central1`}),_jsx(_components.span,{style:{color:`#D7BA7D`,\"--shiki-dark\":`#D7BA7D`},children:` \\\\`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#569CD6`,\"--shiki-dark\":`#569CD6`},children:` --cluster=tfy-gtl-b-us-central-1`}),_jsx(_components.span,{style:{color:`#D7BA7D`,\"--shiki-dark\":`#D7BA7D`},children:` \\\\`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#569CD6`,\"--shiki-dark\":`#569CD6`},children:` --machine-type=a2-highgpu-1g`}),_jsx(_components.span,{style:{color:`#D7BA7D`,\"--shiki-dark\":`#D7BA7D`},children:` \\\\`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#569CD6`,\"--shiki-dark\":`#569CD6`},children:` --accelerator`}),_jsx(_components.span,{style:{color:`#CE9178`,\"--shiki-dark\":`#CE9178`},children:` type=nvidia-tesla-a100,count=1,gpu-sharing-strategy=time-sharing,max-shared-clients-per-gpu=`}),_jsx(_components.span,{style:{color:`#B5CEA8`,\"--shiki-dark\":`#B5CEA8`},children:`10`}),_jsx(_components.span,{style:{color:`#D7BA7D`,\"--shiki-dark\":`#D7BA7D`},children:` \\\\`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#569CD6`,\"--shiki-dark\":`#569CD6`},children:` --enable-autoscaling`}),_jsx(_components.span,{style:{color:`#D7BA7D`,\"--shiki-dark\":`#D7BA7D`},children:` \\\\`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#569CD6`,\"--shiki-dark\":`#569CD6`},children:` --total-min-nodes`}),_jsx(_components.span,{style:{color:`#B5CEA8`,\"--shiki-dark\":`#B5CEA8`},children:` 0`}),_jsx(_components.span,{style:{color:`#D7BA7D`,\"--shiki-dark\":`#D7BA7D`},children:` \\\\`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#569CD6`,\"--shiki-dark\":`#569CD6`},children:` --total-max-nodes`}),_jsx(_components.span,{style:{color:`#B5CEA8`,\"--shiki-dark\":`#B5CEA8`},children:` 4`}),_jsx(_components.span,{style:{color:`#D7BA7D`,\"--shiki-dark\":`#D7BA7D`},children:` \\\\`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#569CD6`,\"--shiki-dark\":`#569CD6`},children:` --min-provision-nodes`}),_jsx(_components.span,{style:{color:`#B5CEA8`,\"--shiki-dark\":`#B5CEA8`},children:` 0`}),_jsx(_components.span,{style:{color:`#D7BA7D`,\"--shiki-dark\":`#D7BA7D`},children:` \\\\`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#569CD6`,\"--shiki-dark\":`#569CD6`},children:` --num-nodes`}),_jsx(_components.span,{style:{color:`#B5CEA8`,\"--shiki-dark\":`#B5CEA8`},children:` 0`})]}),`\n`]})})})]}),_jsxs(Tab,{title:`AWS`,id:`aws-2`,children:[_jsxs(_components.p,{children:[`On AWS EKS you do `,_jsx(_components.strong,{children:`not`}),` create managed nodegroups for timeslicing. TrueFoundry discovers fractional GPUs from Karpenter `,_jsx(_components.strong,{children:`NodePools`}),` + `,_jsx(_components.strong,{children:`NodeOverlays`}),`. Enable timeslicing by updating three things:`]}),_jsxs(_components.ol,{children:[`\n`,_jsxs(_components.li,{children:[_jsx(_components.code,{children:`tfy-gpu-operator`}),` values (device-plugin timeslice configs)`]}),`\n`,_jsxs(_components.li,{children:[_jsx(_components.code,{children:`tfy-karpenter-config`}),` kustomize additions (NodePool + NodeOverlay)`]}),`\n`,_jsx(_components.li,{children:`Sync Cluster in the UI`}),`\n`]}),_jsxs(_components.p,{children:[`Use the latest `,_jsx(_components.a,{href:`https://artifacthub.io/packages/helm/truefoundry/tfy-gpu-operator`,children:`tfy-gpu-operator`}),` and `,_jsx(_components.a,{href:`https://artifacthub.io/packages/helm/truefoundry/tfy-karpenter-config`,children:`tfy-karpenter-config`}),` chart versions.`]}),_jsx(_components.p,{children:_jsxs(_components.strong,{children:[`1. Configure timeslicing in `,_jsx(_components.code,{children:`tfy-gpu-operator`})]})}),_jsxs(_components.p,{children:[`Edit the `,_jsx(_components.code,{children:`tfy-gpu-operator`}),` Helm application and merge the following into values. This creates a device-plugin profile named `,_jsx(_components.code,{children:`time-sliced-10`}),` (10 slices per GPU):`]}),_jsx(CodeBlock,{filename:``,numberOfLines:`20`,language:`yaml`,children:_jsx(_components.pre,{className:`shiki shiki-themes dark-plus dark-plus`,style:{backgroundColor:`#0B0C0E`,\"--shiki-dark-bg\":`#0B0C0E`,color:`#D4D4D4`,\"--shiki-dark\":`#D4D4D4`},language:`yaml`,children:_jsxs(_components.code,{language:`yaml`,numberOfLines:`20`,children:[_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#569CD6`,\"--shiki-dark\":`#569CD6`},children:`clusterType`}),_jsx(_components.span,{style:{color:`#D4D4D4`,\"--shiki-dark\":`#D4D4D4`},children:`:`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#569CD6`,\"--shiki-dark\":`#569CD6`},children:` awsEks`}),_jsx(_components.span,{style:{color:`#D4D4D4`,\"--shiki-dark\":`#D4D4D4`},children:`: `}),_jsx(_components.span,{style:{color:`#569CD6`,\"--shiki-dark\":`#569CD6`},children:`true`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#569CD6`,\"--shiki-dark\":`#569CD6`},children:`aws-eks-gpu-operator`}),_jsx(_components.span,{style:{color:`#D4D4D4`,\"--shiki-dark\":`#D4D4D4`},children:`:`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#569CD6`,\"--shiki-dark\":`#569CD6`},children:` devicePlugin`}),_jsx(_components.span,{style:{color:`#D4D4D4`,\"--shiki-dark\":`#D4D4D4`},children:`:`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#569CD6`,\"--shiki-dark\":`#569CD6`},children:` config`}),_jsx(_components.span,{style:{color:`#D4D4D4`,\"--shiki-dark\":`#D4D4D4`},children:`:`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#569CD6`,\"--shiki-dark\":`#569CD6`},children:` name`}),_jsx(_components.span,{style:{color:`#D4D4D4`,\"--shiki-dark\":`#D4D4D4`},children:`: `}),_jsx(_components.span,{style:{color:`#CE9178`,\"--shiki-dark\":`#CE9178`},children:`nvidia-device-plugin-configs`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#569CD6`,\"--shiki-dark\":`#569CD6`},children:` create`}),_jsx(_components.span,{style:{color:`#D4D4D4`,\"--shiki-dark\":`#D4D4D4`},children:`: `}),_jsx(_components.span,{style:{color:`#569CD6`,\"--shiki-dark\":`#569CD6`},children:`true`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#569CD6`,\"--shiki-dark\":`#569CD6`},children:` default`}),_jsx(_components.span,{style:{color:`#D4D4D4`,\"--shiki-dark\":`#D4D4D4`},children:`: `}),_jsx(_components.span,{style:{color:`#CE9178`,\"--shiki-dark\":`#CE9178`},children:`all`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#569CD6`,\"--shiki-dark\":`#569CD6`},children:` data`}),_jsx(_components.span,{style:{color:`#D4D4D4`,\"--shiki-dark\":`#D4D4D4`},children:`:`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#569CD6`,\"--shiki-dark\":`#569CD6`},children:` all`}),_jsx(_components.span,{style:{color:`#D4D4D4`,\"--shiki-dark\":`#D4D4D4`},children:`: `}),_jsx(_components.span,{style:{color:`#CE9178`,\"--shiki-dark\":`#CE9178`},children:`\"\"`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#569CD6`,\"--shiki-dark\":`#569CD6`},children:` time-sliced-10`}),_jsx(_components.span,{style:{color:`#D4D4D4`,\"--shiki-dark\":`#D4D4D4`},children:`: `}),_jsx(_components.span,{style:{color:`#C586C0`,\"--shiki-dark\":`#C586C0`},children:`|`}),_jsx(_components.span,{style:{color:`#569CD6`,\"--shiki-dark\":`#569CD6`},children:`-`})]}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{style:{color:`#CE9178`,\"--shiki-dark\":`#CE9178`},children:` version: v1`})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{style:{color:`#CE9178`,\"--shiki-dark\":`#CE9178`},children:` sharing:`})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{style:{color:`#CE9178`,\"--shiki-dark\":`#CE9178`},children:` flags:`})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{style:{color:`#CE9178`,\"--shiki-dark\":`#CE9178`},children:` migStrategy: none`})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{style:{color:`#CE9178`,\"--shiki-dark\":`#CE9178`},children:` timeSlicing:`})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{style:{color:`#CE9178`,\"--shiki-dark\":`#CE9178`},children:` renameByDefault: true`})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{style:{color:`#CE9178`,\"--shiki-dark\":`#CE9178`},children:` resources:`})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{style:{color:`#CE9178`,\"--shiki-dark\":`#CE9178`},children:` - name: nvidia.com/gpu`})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{style:{color:`#CE9178`,\"--shiki-dark\":`#CE9178`},children:` replicas: 10`})}),`\n`]})})}),_jsxs(_components.p,{children:[`The profile name (`,_jsx(_components.code,{children:`time-sliced-10`}),`) must match the `,_jsx(_components.code,{children:`nvidia.com/device-plugin.config`}),` label on the NodePool in the next step. Deploy the updated Helm application.`]}),_jsx(_components.p,{children:_jsxs(_components.strong,{children:[`2. Add a Karpenter NodePool + NodeOverlay via `,_jsx(_components.code,{children:`tfy-karpenter-config`})]})}),_jsxs(_components.p,{children:[`Edit the `,_jsx(_components.code,{children:`tfy-karpenter-config`}),` Helm application. Under `,_jsx(_components.strong,{children:`Kustomize → Additional Manifests`}),`, add both resources. Example: L4 on `,_jsx(_components.code,{children:`g6.xlarge`}),` with 10 timeslices:`]}),_jsx(CodeBlock,{filename:``,numberOfLines:`57`,language:`yaml`,children:_jsx(_components.pre,{className:`shiki shiki-themes dark-plus dark-plus`,style:{backgroundColor:`#0B0C0E`,\"--shiki-dark-bg\":`#0B0C0E`,color:`#D4D4D4`,\"--shiki-dark\":`#D4D4D4`},language:`yaml`,children:_jsxs(_components.code,{language:`yaml`,numberOfLines:`57`,children:[_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#569CD6`,\"--shiki-dark\":`#569CD6`},children:`apiVersion`}),_jsx(_components.span,{style:{color:`#D4D4D4`,\"--shiki-dark\":`#D4D4D4`},children:`: `}),_jsx(_components.span,{style:{color:`#CE9178`,\"--shiki-dark\":`#CE9178`},children:`karpenter.sh/v1`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#569CD6`,\"--shiki-dark\":`#569CD6`},children:`kind`}),_jsx(_components.span,{style:{color:`#D4D4D4`,\"--shiki-dark\":`#D4D4D4`},children:`: `}),_jsx(_components.span,{style:{color:`#CE9178`,\"--shiki-dark\":`#CE9178`},children:`NodePool`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#569CD6`,\"--shiki-dark\":`#569CD6`},children:`metadata`}),_jsx(_components.span,{style:{color:`#D4D4D4`,\"--shiki-dark\":`#D4D4D4`},children:`:`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#569CD6`,\"--shiki-dark\":`#569CD6`},children:` name`}),_jsx(_components.span,{style:{color:`#D4D4D4`,\"--shiki-dark\":`#D4D4D4`},children:`: `}),_jsx(_components.span,{style:{color:`#CE9178`,\"--shiki-dark\":`#CE9178`},children:`g6-time-sliced-10`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#569CD6`,\"--shiki-dark\":`#569CD6`},children:`spec`}),_jsx(_components.span,{style:{color:`#D4D4D4`,\"--shiki-dark\":`#D4D4D4`},children:`:`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#569CD6`,\"--shiki-dark\":`#569CD6`},children:` weight`}),_jsx(_components.span,{style:{color:`#D4D4D4`,\"--shiki-dark\":`#D4D4D4`},children:`: `}),_jsx(_components.span,{style:{color:`#B5CEA8`,\"--shiki-dark\":`#B5CEA8`},children:`10`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#569CD6`,\"--shiki-dark\":`#569CD6`},children:` disruption`}),_jsx(_components.span,{style:{color:`#D4D4D4`,\"--shiki-dark\":`#D4D4D4`},children:`:`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#569CD6`,\"--shiki-dark\":`#569CD6`},children:` consolidationPolicy`}),_jsx(_components.span,{style:{color:`#D4D4D4`,\"--shiki-dark\":`#D4D4D4`},children:`: `}),_jsx(_components.span,{style:{color:`#CE9178`,\"--shiki-dark\":`#CE9178`},children:`WhenEmptyOrUnderutilized`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#569CD6`,\"--shiki-dark\":`#569CD6`},children:` consolidateAfter`}),_jsx(_components.span,{style:{color:`#D4D4D4`,\"--shiki-dark\":`#D4D4D4`},children:`: `}),_jsx(_components.span,{style:{color:`#CE9178`,\"--shiki-dark\":`#CE9178`},children:`5m`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#569CD6`,\"--shiki-dark\":`#569CD6`},children:` budgets`}),_jsx(_components.span,{style:{color:`#D4D4D4`,\"--shiki-dark\":`#D4D4D4`},children:`:`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#D4D4D4`,\"--shiki-dark\":`#D4D4D4`},children:` - `}),_jsx(_components.span,{style:{color:`#569CD6`,\"--shiki-dark\":`#569CD6`},children:`nodes`}),_jsx(_components.span,{style:{color:`#D4D4D4`,\"--shiki-dark\":`#D4D4D4`},children:`: `}),_jsx(_components.span,{style:{color:`#CE9178`,\"--shiki-dark\":`#CE9178`},children:`10%`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#569CD6`,\"--shiki-dark\":`#569CD6`},children:` reasons`}),_jsx(_components.span,{style:{color:`#D4D4D4`,\"--shiki-dark\":`#D4D4D4`},children:`: [`}),_jsx(_components.span,{style:{color:`#CE9178`,\"--shiki-dark\":`#CE9178`},children:`Underutilized`}),_jsx(_components.span,{style:{color:`#D4D4D4`,\"--shiki-dark\":`#D4D4D4`},children:`, `}),_jsx(_components.span,{style:{color:`#CE9178`,\"--shiki-dark\":`#CE9178`},children:`Empty`}),_jsx(_components.span,{style:{color:`#D4D4D4`,\"--shiki-dark\":`#D4D4D4`},children:`]`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#D4D4D4`,\"--shiki-dark\":`#D4D4D4`},children:` - `}),_jsx(_components.span,{style:{color:`#569CD6`,\"--shiki-dark\":`#569CD6`},children:`nodes`}),_jsx(_components.span,{style:{color:`#D4D4D4`,\"--shiki-dark\":`#D4D4D4`},children:`: `}),_jsx(_components.span,{style:{color:`#CE9178`,\"--shiki-dark\":`#CE9178`},children:`\"1\"`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#569CD6`,\"--shiki-dark\":`#569CD6`},children:` reasons`}),_jsx(_components.span,{style:{color:`#D4D4D4`,\"--shiki-dark\":`#D4D4D4`},children:`: [`}),_jsx(_components.span,{style:{color:`#CE9178`,\"--shiki-dark\":`#CE9178`},children:`Drifted`}),_jsx(_components.span,{style:{color:`#D4D4D4`,\"--shiki-dark\":`#D4D4D4`},children:`]`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#569CD6`,\"--shiki-dark\":`#569CD6`},children:` template`}),_jsx(_components.span,{style:{color:`#D4D4D4`,\"--shiki-dark\":`#D4D4D4`},children:`:`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#569CD6`,\"--shiki-dark\":`#569CD6`},children:` metadata`}),_jsx(_components.span,{style:{color:`#D4D4D4`,\"--shiki-dark\":`#D4D4D4`},children:`:`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#569CD6`,\"--shiki-dark\":`#569CD6`},children:` labels`}),_jsx(_components.span,{style:{color:`#D4D4D4`,\"--shiki-dark\":`#D4D4D4`},children:`:`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#569CD6`,\"--shiki-dark\":`#569CD6`},children:` eks.amazonaws.com/nodegroup`}),_jsx(_components.span,{style:{color:`#D4D4D4`,\"--shiki-dark\":`#D4D4D4`},children:`: `}),_jsx(_components.span,{style:{color:`#CE9178`,\"--shiki-dark\":`#CE9178`},children:`g6-time-sliced-10`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#569CD6`,\"--shiki-dark\":`#569CD6`},children:` nvidia.com/gpu.deploy.operands`}),_jsx(_components.span,{style:{color:`#D4D4D4`,\"--shiki-dark\":`#D4D4D4`},children:`: `}),_jsx(_components.span,{style:{color:`#CE9178`,\"--shiki-dark\":`#CE9178`},children:`\"true\"`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#569CD6`,\"--shiki-dark\":`#569CD6`},children:` nvidia.com/device-plugin.config`}),_jsx(_components.span,{style:{color:`#D4D4D4`,\"--shiki-dark\":`#D4D4D4`},children:`: `}),_jsx(_components.span,{style:{color:`#CE9178`,\"--shiki-dark\":`#CE9178`},children:`time-sliced-10`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#569CD6`,\"--shiki-dark\":`#569CD6`},children:` spec`}),_jsx(_components.span,{style:{color:`#D4D4D4`,\"--shiki-dark\":`#D4D4D4`},children:`:`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#569CD6`,\"--shiki-dark\":`#569CD6`},children:` expireAfter`}),_jsx(_components.span,{style:{color:`#D4D4D4`,\"--shiki-dark\":`#D4D4D4`},children:`: `}),_jsx(_components.span,{style:{color:`#CE9178`,\"--shiki-dark\":`#CE9178`},children:`720h`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#569CD6`,\"--shiki-dark\":`#569CD6`},children:` nodeClassRef`}),_jsx(_components.span,{style:{color:`#D4D4D4`,\"--shiki-dark\":`#D4D4D4`},children:`:`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#569CD6`,\"--shiki-dark\":`#569CD6`},children:` group`}),_jsx(_components.span,{style:{color:`#D4D4D4`,\"--shiki-dark\":`#D4D4D4`},children:`: `}),_jsx(_components.span,{style:{color:`#CE9178`,\"--shiki-dark\":`#CE9178`},children:`karpenter.k8s.aws`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#569CD6`,\"--shiki-dark\":`#569CD6`},children:` kind`}),_jsx(_components.span,{style:{color:`#D4D4D4`,\"--shiki-dark\":`#D4D4D4`},children:`: `}),_jsx(_components.span,{style:{color:`#CE9178`,\"--shiki-dark\":`#CE9178`},children:`EC2NodeClass`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#569CD6`,\"--shiki-dark\":`#569CD6`},children:` name`}),_jsx(_components.span,{style:{color:`#D4D4D4`,\"--shiki-dark\":`#D4D4D4`},children:`: `}),_jsx(_components.span,{style:{color:`#CE9178`,\"--shiki-dark\":`#CE9178`},children:`gpu-default`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#569CD6`,\"--shiki-dark\":`#569CD6`},children:` taints`}),_jsx(_components.span,{style:{color:`#D4D4D4`,\"--shiki-dark\":`#D4D4D4`},children:`:`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#D4D4D4`,\"--shiki-dark\":`#D4D4D4`},children:` - `}),_jsx(_components.span,{style:{color:`#569CD6`,\"--shiki-dark\":`#569CD6`},children:`key`}),_jsx(_components.span,{style:{color:`#D4D4D4`,\"--shiki-dark\":`#D4D4D4`},children:`: `}),_jsx(_components.span,{style:{color:`#CE9178`,\"--shiki-dark\":`#CE9178`},children:`nvidia.com/gpu`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#569CD6`,\"--shiki-dark\":`#569CD6`},children:` effect`}),_jsx(_components.span,{style:{color:`#D4D4D4`,\"--shiki-dark\":`#D4D4D4`},children:`: `}),_jsx(_components.span,{style:{color:`#CE9178`,\"--shiki-dark\":`#CE9178`},children:`NoSchedule`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#569CD6`,\"--shiki-dark\":`#569CD6`},children:` requirements`}),_jsx(_components.span,{style:{color:`#D4D4D4`,\"--shiki-dark\":`#D4D4D4`},children:`:`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#D4D4D4`,\"--shiki-dark\":`#D4D4D4`},children:` - `}),_jsx(_components.span,{style:{color:`#569CD6`,\"--shiki-dark\":`#569CD6`},children:`key`}),_jsx(_components.span,{style:{color:`#D4D4D4`,\"--shiki-dark\":`#D4D4D4`},children:`: `}),_jsx(_components.span,{style:{color:`#CE9178`,\"--shiki-dark\":`#CE9178`},children:`karpenter.sh/capacity-type`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#569CD6`,\"--shiki-dark\":`#569CD6`},children:` operator`}),_jsx(_components.span,{style:{color:`#D4D4D4`,\"--shiki-dark\":`#D4D4D4`},children:`: `}),_jsx(_components.span,{style:{color:`#CE9178`,\"--shiki-dark\":`#CE9178`},children:`In`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#569CD6`,\"--shiki-dark\":`#569CD6`},children:` values`}),_jsx(_components.span,{style:{color:`#D4D4D4`,\"--shiki-dark\":`#D4D4D4`},children:`: [`}),_jsx(_components.span,{style:{color:`#CE9178`,\"--shiki-dark\":`#CE9178`},children:`spot`}),_jsx(_components.span,{style:{color:`#D4D4D4`,\"--shiki-dark\":`#D4D4D4`},children:`, `}),_jsx(_components.span,{style:{color:`#CE9178`,\"--shiki-dark\":`#CE9178`},children:`on-demand`}),_jsx(_components.span,{style:{color:`#D4D4D4`,\"--shiki-dark\":`#D4D4D4`},children:`]`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#D4D4D4`,\"--shiki-dark\":`#D4D4D4`},children:` - `}),_jsx(_components.span,{style:{color:`#569CD6`,\"--shiki-dark\":`#569CD6`},children:`key`}),_jsx(_components.span,{style:{color:`#D4D4D4`,\"--shiki-dark\":`#D4D4D4`},children:`: `}),_jsx(_components.span,{style:{color:`#CE9178`,\"--shiki-dark\":`#CE9178`},children:`kubernetes.io/arch`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#569CD6`,\"--shiki-dark\":`#569CD6`},children:` operator`}),_jsx(_components.span,{style:{color:`#D4D4D4`,\"--shiki-dark\":`#D4D4D4`},children:`: `}),_jsx(_components.span,{style:{color:`#CE9178`,\"--shiki-dark\":`#CE9178`},children:`In`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#569CD6`,\"--shiki-dark\":`#569CD6`},children:` values`}),_jsx(_components.span,{style:{color:`#D4D4D4`,\"--shiki-dark\":`#D4D4D4`},children:`: [`}),_jsx(_components.span,{style:{color:`#CE9178`,\"--shiki-dark\":`#CE9178`},children:`amd64`}),_jsx(_components.span,{style:{color:`#D4D4D4`,\"--shiki-dark\":`#D4D4D4`},children:`]`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#D4D4D4`,\"--shiki-dark\":`#D4D4D4`},children:` - `}),_jsx(_components.span,{style:{color:`#569CD6`,\"--shiki-dark\":`#569CD6`},children:`key`}),_jsx(_components.span,{style:{color:`#D4D4D4`,\"--shiki-dark\":`#D4D4D4`},children:`: `}),_jsx(_components.span,{style:{color:`#CE9178`,\"--shiki-dark\":`#CE9178`},children:`node.kubernetes.io/instance-type`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#569CD6`,\"--shiki-dark\":`#569CD6`},children:` operator`}),_jsx(_components.span,{style:{color:`#D4D4D4`,\"--shiki-dark\":`#D4D4D4`},children:`: `}),_jsx(_components.span,{style:{color:`#CE9178`,\"--shiki-dark\":`#CE9178`},children:`In`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#569CD6`,\"--shiki-dark\":`#569CD6`},children:` values`}),_jsx(_components.span,{style:{color:`#D4D4D4`,\"--shiki-dark\":`#D4D4D4`},children:`: [`}),_jsx(_components.span,{style:{color:`#CE9178`,\"--shiki-dark\":`#CE9178`},children:`g6.xlarge`}),_jsx(_components.span,{style:{color:`#D4D4D4`,\"--shiki-dark\":`#D4D4D4`},children:`]`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#D4D4D4`,\"--shiki-dark\":`#D4D4D4`},children:` - `}),_jsx(_components.span,{style:{color:`#569CD6`,\"--shiki-dark\":`#569CD6`},children:`key`}),_jsx(_components.span,{style:{color:`#D4D4D4`,\"--shiki-dark\":`#D4D4D4`},children:`: `}),_jsx(_components.span,{style:{color:`#CE9178`,\"--shiki-dark\":`#CE9178`},children:`topology.kubernetes.io/zone`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#569CD6`,\"--shiki-dark\":`#569CD6`},children:` operator`}),_jsx(_components.span,{style:{color:`#D4D4D4`,\"--shiki-dark\":`#D4D4D4`},children:`: `}),_jsx(_components.span,{style:{color:`#CE9178`,\"--shiki-dark\":`#CE9178`},children:`In`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#569CD6`,\"--shiki-dark\":`#569CD6`},children:` values`}),_jsx(_components.span,{style:{color:`#D4D4D4`,\"--shiki-dark\":`#D4D4D4`},children:`: [`}),_jsx(_components.span,{style:{color:`#CE9178`,\"--shiki-dark\":`#CE9178`},children:`us-east-1a`}),_jsx(_components.span,{style:{color:`#D4D4D4`,\"--shiki-dark\":`#D4D4D4`},children:`, `}),_jsx(_components.span,{style:{color:`#CE9178`,\"--shiki-dark\":`#CE9178`},children:`us-east-1b`}),_jsx(_components.span,{style:{color:`#D4D4D4`,\"--shiki-dark\":`#D4D4D4`},children:`, `}),_jsx(_components.span,{style:{color:`#CE9178`,\"--shiki-dark\":`#CE9178`},children:`us-east-1c`}),_jsx(_components.span,{style:{color:`#D4D4D4`,\"--shiki-dark\":`#D4D4D4`},children:`, `}),_jsx(_components.span,{style:{color:`#CE9178`,\"--shiki-dark\":`#CE9178`},children:`us-east-1d`}),_jsx(_components.span,{style:{color:`#D4D4D4`,\"--shiki-dark\":`#D4D4D4`},children:`]`})]}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{style:{color:`#D4D4D4`,\"--shiki-dark\":`#D4D4D4`},children:`---`})}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#569CD6`,\"--shiki-dark\":`#569CD6`},children:`apiVersion`}),_jsx(_components.span,{style:{color:`#D4D4D4`,\"--shiki-dark\":`#D4D4D4`},children:`: `}),_jsx(_components.span,{style:{color:`#CE9178`,\"--shiki-dark\":`#CE9178`},children:`karpenter.sh/v1alpha1`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#569CD6`,\"--shiki-dark\":`#569CD6`},children:`kind`}),_jsx(_components.span,{style:{color:`#D4D4D4`,\"--shiki-dark\":`#D4D4D4`},children:`: `}),_jsx(_components.span,{style:{color:`#CE9178`,\"--shiki-dark\":`#CE9178`},children:`NodeOverlay`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#569CD6`,\"--shiki-dark\":`#569CD6`},children:`metadata`}),_jsx(_components.span,{style:{color:`#D4D4D4`,\"--shiki-dark\":`#D4D4D4`},children:`:`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#569CD6`,\"--shiki-dark\":`#569CD6`},children:` name`}),_jsx(_components.span,{style:{color:`#D4D4D4`,\"--shiki-dark\":`#D4D4D4`},children:`: `}),_jsx(_components.span,{style:{color:`#CE9178`,\"--shiki-dark\":`#CE9178`},children:`g6-time-sliced-10`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#569CD6`,\"--shiki-dark\":`#569CD6`},children:` labels`}),_jsx(_components.span,{style:{color:`#D4D4D4`,\"--shiki-dark\":`#D4D4D4`},children:`:`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#569CD6`,\"--shiki-dark\":`#569CD6`},children:` truefoundry.com/fractional-gpu`}),_jsx(_components.span,{style:{color:`#D4D4D4`,\"--shiki-dark\":`#D4D4D4`},children:`: `}),_jsx(_components.span,{style:{color:`#CE9178`,\"--shiki-dark\":`#CE9178`},children:`timeslice`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#569CD6`,\"--shiki-dark\":`#569CD6`},children:`spec`}),_jsx(_components.span,{style:{color:`#D4D4D4`,\"--shiki-dark\":`#D4D4D4`},children:`:`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#569CD6`,\"--shiki-dark\":`#569CD6`},children:` weight`}),_jsx(_components.span,{style:{color:`#D4D4D4`,\"--shiki-dark\":`#D4D4D4`},children:`: `}),_jsx(_components.span,{style:{color:`#B5CEA8`,\"--shiki-dark\":`#B5CEA8`},children:`10`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#569CD6`,\"--shiki-dark\":`#569CD6`},children:` capacity`}),_jsx(_components.span,{style:{color:`#D4D4D4`,\"--shiki-dark\":`#D4D4D4`},children:`:`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#569CD6`,\"--shiki-dark\":`#569CD6`},children:` nvidia.com/gpu.shared`}),_jsx(_components.span,{style:{color:`#D4D4D4`,\"--shiki-dark\":`#D4D4D4`},children:`: `}),_jsx(_components.span,{style:{color:`#B5CEA8`,\"--shiki-dark\":`#B5CEA8`},children:`10`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#569CD6`,\"--shiki-dark\":`#569CD6`},children:` requirements`}),_jsx(_components.span,{style:{color:`#D4D4D4`,\"--shiki-dark\":`#D4D4D4`},children:`:`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#D4D4D4`,\"--shiki-dark\":`#D4D4D4`},children:` - `}),_jsx(_components.span,{style:{color:`#569CD6`,\"--shiki-dark\":`#569CD6`},children:`key`}),_jsx(_components.span,{style:{color:`#D4D4D4`,\"--shiki-dark\":`#D4D4D4`},children:`: `}),_jsx(_components.span,{style:{color:`#CE9178`,\"--shiki-dark\":`#CE9178`},children:`karpenter.sh/nodepool`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#569CD6`,\"--shiki-dark\":`#569CD6`},children:` operator`}),_jsx(_components.span,{style:{color:`#D4D4D4`,\"--shiki-dark\":`#D4D4D4`},children:`: `}),_jsx(_components.span,{style:{color:`#CE9178`,\"--shiki-dark\":`#CE9178`},children:`In`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#569CD6`,\"--shiki-dark\":`#569CD6`},children:` values`}),_jsx(_components.span,{style:{color:`#D4D4D4`,\"--shiki-dark\":`#D4D4D4`},children:`: [`}),_jsx(_components.span,{style:{color:`#CE9178`,\"--shiki-dark\":`#CE9178`},children:`g6-time-sliced-10`}),_jsx(_components.span,{style:{color:`#D4D4D4`,\"--shiki-dark\":`#D4D4D4`},children:`]`})]}),`\n`]})})}),_jsxs(Info,{children:[_jsx(_components.p,{children:`Requirements TrueFoundry enforces when syncing:`}),_jsxs(_components.ul,{children:[`\n`,_jsxs(_components.li,{children:[`NodeOverlay must set `,_jsx(_components.code,{children:`truefoundry.com/fractional-gpu: timeslice`})]}),`\n`,_jsxs(_components.li,{children:[`Capacity key must be `,_jsx(_components.code,{children:`nvidia.com/gpu.shared`})]}),`\n`,_jsxs(_components.li,{children:[`Capacity value must be divisible by the GPU count on the instance. For `,_jsx(_components.code,{children:`g6.xlarge`}),` (1 GPU) with 10 slices → `,_jsx(_components.code,{children:`10`}),`. For a 2-GPU instance with 10 slices per GPU → `,_jsx(_components.code,{children:`20`})]}),`\n`,_jsxs(_components.li,{children:[`NodePool must set `,_jsx(_components.code,{children:`nvidia.com/device-plugin.config`}),` to a profile name from `,_jsx(_components.code,{children:`tfy-gpu-operator`}),` (for example `,_jsx(_components.code,{children:`time-sliced-10`}),`)`]}),`\n`,_jsx(_components.li,{children:`NodePool must resolve to exactly one GPU instance type`}),`\n`]})]}),_jsxs(_components.p,{children:[`Deploy the updated `,_jsx(_components.code,{children:`tfy-karpenter-config`}),` application.`]}),_jsx(_components.p,{children:_jsx(_components.strong,{children:`3. Sync Cluster`})}),_jsxs(_components.p,{children:[`Go to `,_jsx(_components.strong,{children:`Platform → Clusters`}),` and click `,_jsx(_components.strong,{children:`Sync`}),` on the cluster. Sync also runs periodically; a manual sync makes the new NodePools show up sooner.`]}),_jsx(Frame,{caption:`Sync Cluster`,children:_jsx(OptimizedImage,{src:`https://mintcdn.com/truefoundry/fDI2fd3aQvGCJmLJ/images/docs/using-fractional-gpus-sync-cluster.png?fit=max\u0026auto=format\u0026n=fDI2fd3aQvGCJmLJ\u0026q=85\u0026s=5ce3b17c6ae0bd88b649670a5a23ae5e`,alt:`Clusters page with the Sync Cluster button highlighted`,\"data-og-width\":`3020`,width:`3020`,\"data-og-height\":`1084`,height:`1084`,\"data-path\":`images/docs/using-fractional-gpus-sync-cluster.png`,\"data-optimize\":`true`,\"data-opv\":`3`,srcset:`https://mintcdn.com/truefoundry/fDI2fd3aQvGCJmLJ/images/docs/using-fractional-gpus-sync-cluster.png?w=280\u0026fit=max\u0026auto=format\u0026n=fDI2fd3aQvGCJmLJ\u0026q=85\u0026s=e5e1b538b57a7e17560000c06e1a9b25 280w, https://mintcdn.com/truefoundry/fDI2fd3aQvGCJmLJ/images/docs/using-fractional-gpus-sync-cluster.png?w=560\u0026fit=max\u0026auto=format\u0026n=fDI2fd3aQvGCJmLJ\u0026q=85\u0026s=872ea43c7cd712a2f9c83d4d364b1b06 560w, https://mintcdn.com/truefoundry/fDI2fd3aQvGCJmLJ/images/docs/using-fractional-gpus-sync-cluster.png?w=840\u0026fit=max\u0026auto=format\u0026n=fDI2fd3aQvGCJmLJ\u0026q=85\u0026s=c41bb83e1396b3a9018ef53e23565e08 840w, https://mintcdn.com/truefoundry/fDI2fd3aQvGCJmLJ/images/docs/using-fractional-gpus-sync-cluster.png?w=1100\u0026fit=max\u0026auto=format\u0026n=fDI2fd3aQvGCJmLJ\u0026q=85\u0026s=e8efe86ed95b282b7123f1b21fb17392 1100w, https://mintcdn.com/truefoundry/fDI2fd3aQvGCJmLJ/images/docs/using-fractional-gpus-sync-cluster.png?w=1650\u0026fit=max\u0026auto=format\u0026n=fDI2fd3aQvGCJmLJ\u0026q=85\u0026s=6aef7db9299c8111a0e4d1f2a65e5dce 1650w, https://mintcdn.com/truefoundry/fDI2fd3aQvGCJmLJ/images/docs/using-fractional-gpus-sync-cluster.png?w=2500\u0026fit=max\u0026auto=format\u0026n=fDI2fd3aQvGCJmLJ\u0026q=85\u0026s=54f0f79a1436a563ae0265d9a17b091b 2500w`})})]})]})}),_jsxs(Step,{title:`Deploy your workload on the Timeslicing nodepool`,children:[_jsxs(_components.p,{children:[`Once you have created the nodepool, you will be able to see the timesliced nodepool in the available nodepools in `,_jsx(_components.code,{children:`Resources`}),` section in deployment.`]}),_jsx(Warning,{children:_jsxs(_components.p,{children:[`It might take a few minutes for the newly created nodepool to be visible on the TrueFoundry UI. You can force sync the nodepools by going to `,_jsx(_components.strong,{children:`Platform → Clusters → Sync`}),`.`]})}),_jsx(_components.p,{children:`To deploy a workload that utilizes fractional GPU, start deploying your service/job on TrueFoundry and in the “Resources” section, select nodepool selector.\nYou can now see the timesliced GPUs on the UI which you can select (as shown below)`}),_jsx(Frame,{caption:`Selecting a timesliced GPU in Resources`,children:_jsx(OptimizedImage,{src:`https://mintcdn.com/truefoundry/fDI2fd3aQvGCJmLJ/images/docs/using-fractional-gpus-resources-timeslicing.png?fit=max\u0026auto=format\u0026n=fDI2fd3aQvGCJmLJ\u0026q=85\u0026s=191edb17631a8727af982e0bb7caeacb`,alt:`Resources section with L4 Timeslicing selected and slice memory dropdown open`,\"data-og-width\":`1760`,width:`1760`,\"data-og-height\":`1320`,height:`1320`,\"data-path\":`images/docs/using-fractional-gpus-resources-timeslicing.png`,\"data-optimize\":`true`,\"data-opv\":`3`,srcset:`https://mintcdn.com/truefoundry/fDI2fd3aQvGCJmLJ/images/docs/using-fractional-gpus-resources-timeslicing.png?w=280\u0026fit=max\u0026auto=format\u0026n=fDI2fd3aQvGCJmLJ\u0026q=85\u0026s=9313abb416605e82d976ddebc5d483f2 280w, https://mintcdn.com/truefoundry/fDI2fd3aQvGCJmLJ/images/docs/using-fractional-gpus-resources-timeslicing.png?w=560\u0026fit=max\u0026auto=format\u0026n=fDI2fd3aQvGCJmLJ\u0026q=85\u0026s=f1833e904f6250f0f0d4db5d1eb78454 560w, https://mintcdn.com/truefoundry/fDI2fd3aQvGCJmLJ/images/docs/using-fractional-gpus-resources-timeslicing.png?w=840\u0026fit=max\u0026auto=format\u0026n=fDI2fd3aQvGCJmLJ\u0026q=85\u0026s=0955f81b4549744fd16d624d158c9a18 840w, https://mintcdn.com/truefoundry/fDI2fd3aQvGCJmLJ/images/docs/using-fractional-gpus-resources-timeslicing.png?w=1100\u0026fit=max\u0026auto=format\u0026n=fDI2fd3aQvGCJmLJ\u0026q=85\u0026s=bfb50f0fdd575796f47ee7bf38120ad7 1100w, https://mintcdn.com/truefoundry/fDI2fd3aQvGCJmLJ/images/docs/using-fractional-gpus-resources-timeslicing.png?w=1650\u0026fit=max\u0026auto=format\u0026n=fDI2fd3aQvGCJmLJ\u0026q=85\u0026s=c4ed6efe75f51d54e3f26a262bc09a68 1650w, https://mintcdn.com/truefoundry/fDI2fd3aQvGCJmLJ/images/docs/using-fractional-gpus-resources-timeslicing.png?w=2500\u0026fit=max\u0026auto=format\u0026n=fDI2fd3aQvGCJmLJ\u0026q=85\u0026s=88bbd3b0b51f236143d77f07ed7efb35 2500w`})})]})]})]})}function MDXContent(props={}){const{wrapper:MDXLayout}={..._provideComponents(),...props.components};return MDXLayout?_jsx(MDXLayout,{...props,children:_jsx(_createMdxContent,{...props})}):_createMdxContent(props)}return{default:MDXContent};function _missingMdxReference(id,component){throw new Error(`Expected `+(component?`component`:`object`)+\" `\"+id+\"` to be defined: you likely forgot to import, pass, or provide it.\")}","frontmatter":{},"scope":{"codeStyling":"dark","config":{},"pageMetadata":{"title":"Using Fractional GPUs","description":"Share a single GPU across multiple workloads using Nvidia MIG and TimeSlicing for cost-efficient deployments.","href":"/docs/using-fractional-gpus"}}},"mdxSourceWithNoJs":null},"mdxExtracts":{"tableOfContents":[{"title":"Deploying on Fractional GPUs via MIG","slug":"deploying-on-fractional-gpus-via-mig","depth":2,"children":[]},{"title":"Deploying on Fractional GPUs via Timeslicing","slug":"deploying-on-fractional-gpus-via-timeslicing","depth":2,"children":[]}],"codeExamples":{}},"jsLinks":["/_mintlify/custom/truefoundry/9e0085f7ed271180/scripts.js"],"subdomain":"truefoundry","siteName":"TrueFoundry Docs","title":"Using Fractional GPUs - TrueFoundry Docs","navMaps":{"firstHrefInVersion":{},"firstHrefInLanguage":{}},"lastModified":"2026-11-13T07:27:51.532Z","originalFileLocation":"docs/using-fractional-gpus.mdx"},"headHtml":"\u003cscript data-mint-head>(function(){var d=document.documentElement,p=null;if(!p){try{p=localStorage.getItem(\"isDarkMode\")}catch(e){}if(p==='true')p='dark';else if(p==='false')p='light';if(p!=='light'\u0026\u0026p!=='dark'\u0026\u0026p!=='system')p=\"light\"}var r=p==='system'?(matchMedia(\"(prefers-color-scheme: dark)\").matches?'dark':'light'):p;d.classList.remove(r==='dark'?'light':'dark');d.classList.add(r);d.setAttribute(\"data-theme-preference\",p)})();\u003c/script>\u003cmeta data-mint-head name=\"description\" content=\"Share a single GPU across multiple workloads using Nvidia MIG and TimeSlicing for cost-efficient deployments.\">\u003cmeta data-mint-head name=\"application-name\" content=\"TrueFoundry Docs\">\u003cmeta data-mint-head name=\"generator\" content=\"Mintlify\">\u003clink data-mint-head rel=\"icon\" type=\"image/png\" sizes=\"192x192\" href=\"/mintlify-assets/_mintlify/favicons/truefoundry/DMuq4kzeTciVEGUc/_generated/favicon/android-chrome-192x192.png\" media=\"(prefers-color-scheme: light)\">\u003clink data-mint-head rel=\"apple-touch-icon\" type=\"image/png\" sizes=\"180x180\" href=\"/mintlify-assets/_mintlify/favicons/truefoundry/DMuq4kzeTciVEGUc/_generated/favicon/apple-touch-icon.png\">\u003clink data-mint-head rel=\"icon\" type=\"image/png\" sizes=\"16x16\" href=\"/mintlify-assets/_mintlify/favicons/truefoundry/DMuq4kzeTciVEGUc/_generated/favicon/favicon-16x16.png\" media=\"(prefers-color-scheme: light)\">\u003clink data-mint-head rel=\"icon\" type=\"image/png\" sizes=\"32x32\" href=\"/mintlify-assets/_mintlify/favicons/truefoundry/DMuq4kzeTciVEGUc/_generated/favicon/favicon-32x32.png\" media=\"(prefers-color-scheme: light)\">\u003clink data-mint-head rel=\"shortcut icon\" type=\"image/x-icon\" href=\"/mintlify-assets/_mintlify/favicons/truefoundry/DMuq4kzeTciVEGUc/_generated/favicon/favicon.ico\" media=\"(prefers-color-scheme: light)\">\u003clink data-mint-head rel=\"icon\" type=\"image/png\" sizes=\"192x192\" href=\"/mintlify-assets/_mintlify/favicons/truefoundry/DMuq4kzeTciVEGUc/_generated/favicon-dark/android-chrome-192x192.png\" media=\"(prefers-color-scheme: dark)\">\u003clink data-mint-head rel=\"icon\" type=\"image/png\" sizes=\"16x16\" href=\"/mintlify-assets/_mintlify/favicons/truefoundry/DMuq4kzeTciVEGUc/_generated/favicon-dark/favicon-16x16.png\" media=\"(prefers-color-scheme: dark)\">\u003clink data-mint-head rel=\"icon\" type=\"image/png\" sizes=\"32x32\" href=\"/mintlify-assets/_mintlify/favicons/truefoundry/DMuq4kzeTciVEGUc/_generated/favicon-dark/favicon-32x32.png\" media=\"(prefers-color-scheme: dark)\">\u003clink data-mint-head rel=\"shortcut icon\" type=\"image/x-icon\" href=\"/mintlify-assets/_mintlify/favicons/truefoundry/DMuq4kzeTciVEGUc/_generated/favicon-dark/favicon.ico\" media=\"(prefers-color-scheme: dark)\">\u003cmeta data-mint-head property=\"og:title\" content=\"Using Fractional GPUs - TrueFoundry Docs\">\u003cmeta data-mint-head property=\"og:description\" content=\"TrueFoundry offers an enterprise-grade AI Gateway combining LLM, MCP, and Agent Gateways—empowering businesses to connect, monitor, and govern agentic AI applications across providers from a unified control plane.\">\u003cmeta data-mint-head property=\"og:site_name\" content=\"TrueFoundry Docs\">\u003cmeta data-mint-head property=\"og:url\" content=\"https://www.truefoundry.com/docs/using-fractional-gpus\">\u003cmeta data-mint-head property=\"og:type\" content=\"website\">\u003cmeta data-mint-head property=\"og:image\" content=\"https://assets.production.truefoundry.com/logo.svg\">\u003cmeta data-mint-head property=\"og:image:width\" content=\"1200\">\u003cmeta data-mint-head property=\"og:image:height\" content=\"630\">\u003cmeta data-mint-head name=\"twitter:card\" content=\"summary_large_image\">\u003cmeta data-mint-head name=\"twitter:title\" content=\"Using Fractional GPUs - TrueFoundry Docs\">\u003cmeta data-mint-head name=\"twitter:description\" content=\"TrueFoundry offers an enterprise-grade AI Gateway combining LLM, MCP, and Agent Gateways—empowering businesses to connect, monitor, and govern agentic AI applications across providers from a unified control plane.\">\u003cmeta data-mint-head name=\"twitter:image\" content=\"https://assets.production.truefoundry.com/logo.svg\">\u003cmeta data-mint-head name=\"twitter:image:width\" content=\"1200\">\u003cmeta data-mint-head name=\"twitter:image:height\" content=\"630\">\u003cmeta data-mint-head name=\"msapplication-config\" content=\"/mintlify-assets/_mintlify/favicons/truefoundry/DMuq4kzeTciVEGUc/_generated/favicon/browserconfig.xml\">\u003cmeta data-mint-head name=\"apple-mobile-web-app-title\" content=\"TrueFoundry Docs\">\u003cmeta data-mint-head name=\"msapplication-TileColor\" content=\"#4f46e5\">\u003clink data-mint-head rel=\"canonical\" href=\"https://www.truefoundry.com/docs/using-fractional-gpus\">\u003clink data-mint-head rel=\"alternate\" type=\"application/xml\" href=\"/sitemap.xml\">\u003clink data-mint-head rel=\"alternate\" type=\"text/markdown\" href=\"/docs/using-fractional-gpus.md\">\u003cscript data-mint-head type=\"application/ld+json\">{\"@context\":\"https://schema.org\",\"@graph\":[{\"@type\":\"Organization\",\"@id\":\"https://www.truefoundry.com/#organization\",\"name\":\"TrueFoundry Docs\",\"url\":\"https://www.truefoundry.com\",\"logo\":{\"@type\":\"ImageObject\",\"url\":\"https://mintcdn.com/truefoundry/e2uUimEwX30qkWxc/logo/light.svg?fit=max\u0026auto=format\u0026n=e2uUimEwX30qkWxc\u0026q=85\u0026s=6068d83664aeda0239bec2422657dd20\"}},{\"@type\":\"WebSite\",\"@id\":\"https://www.truefoundry.com#website\",\"name\":\"TrueFoundry Docs\",\"url\":\"https://www.truefoundry.com\",\"publisher\":{\"@id\":\"https://www.truefoundry.com/#organization\"}},{\"@type\":\"WebPage\",\"@id\":\"https://www.truefoundry.com/docs/using-fractional-gpus#webpage\",\"url\":\"https://www.truefoundry.com/docs/using-fractional-gpus\",\"name\":\"Using Fractional GPUs\",\"description\":\"Share a single GPU across multiple workloads using Nvidia MIG and TimeSlicing for cost-efficient deployments.\",\"dateModified\":\"2026-11-13T07:27:51.532Z\",\"isPartOf\":{\"@id\":\"https://www.truefoundry.com#website\"},\"breadcrumb\":{\"@id\":\"https://www.truefoundry.com/docs/using-fractional-gpus#breadcrumb\"}},{\"@type\":\"BreadcrumbList\",\"@id\":\"https://www.truefoundry.com/docs/using-fractional-gpus#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"Service Deployment\",\"item\":\"https://www.truefoundry.com/docs/introduction-to-a-service\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"Configuring your service\",\"item\":\"https://www.truefoundry.com/docs/dockerize-code\"},{\"@type\":\"ListItem\",\"position\":3,\"name\":\"Resources\",\"item\":\"https://www.truefoundry.com/docs/resources-cpu-memory-storage\"},{\"@type\":\"ListItem\",\"position\":4,\"name\":\"Using Fractional GPUs\",\"item\":\"https://www.truefoundry.com/docs/using-fractional-gpus\"}]},{\"@type\":[\"Article\",\"TechArticle\"],\"@id\":\"https://www.truefoundry.com/docs/using-fractional-gpus#article\",\"headline\":\"Using Fractional GPUs\",\"name\":\"Using Fractional GPUs\",\"description\":\"Share a single GPU across multiple workloads using Nvidia MIG and TimeSlicing for cost-efficient deployments.\",\"url\":\"https://www.truefoundry.com/docs/using-fractional-gpus\",\"mainEntityOfPage\":{\"@id\":\"https://www.truefoundry.com/docs/using-fractional-gpus#webpage\"},\"image\":\"https://assets.production.truefoundry.com/logo.svg\",\"dateModified\":\"2026-11-13T07:27:51.532Z\",\"publisher\":{\"@id\":\"https://www.truefoundry.com/#organization\"},\"isPartOf\":{\"@id\":\"https://www.truefoundry.com#website\"}}]}\u003c/script>\u003cscript data-mint-head type=\"application/ld+json\">{\"@context\":\"https://schema.org\",\"@type\":\"WebSite\",\"name\":\"TrueFoundry Docs\",\"publisher\":{\"@type\":\"Organization\",\"name\":\"TrueFoundry Docs\"},\"provider\":{\"@type\":\"Organization\",\"name\":\"Mintlify\",\"url\":\"https://mintlify.com\"}}\u003c/script>","agentPointerHref":"/docs/using-fractional-gpus.md","siteName":"TrueFoundry Docs","nav":{"global":null,"tabs":[{"tab":"AI Gateway","pages":[{"title":"AI Gateway","href":"/docs/ai-gateway/intro-to-llm-gateway"}]},{"tab":"Agent Platform","pages":[{"title":"Agent Platform","href":"/docs/agent-platform/agent-harness/overview"}]},{"tab":"Platform","pages":[{"title":"Platform","href":"/docs/platform/overview"}]},{"tab":"AI Deployment","groups":[{"group":"Introduction","pages":[{"title":"Introduction to AI Deployment","sidebarTitle":"Overview","href":"/docs/ai-deployment-overview"},{"title":"Key Concepts","href":"/docs/key-concepts"}]},{"group":"Train and Deploy Models","pages":[{"title":"Model Deployment Introduction","sidebarTitle":"Introduction","href":"/docs/model-deployment-introduction"},{"group":"Develop and Iterate","pages":[{"title":"Launch Jupyter Notebook","href":"/docs/launch-notebooks"},{"title":"Launch a SSH Server","href":"/docs/launch-an-ssh-server"},{"title":"Launch Notebook with Spark","href":"/docs/launch-notebook-with-spark"}]},{"title":"Model Registry","href":"/docs/model-registry"},{"group":"Deploy Model As API","pages":[{"title":"Model Deployment Overview","sidebarTitle":"Overview","href":"/docs/model-deployment/overview"},{"title":"HuggingFace","href":"/docs/model-deployment/deploy-model-from-huggingface"},{"title":"Scikit Learn / XGBoost","href":"/docs/model-deployment/deploy-sklearn-xgboost"},{"title":"FastAPI","href":"/docs/model-deployment/deploy-model-fastapi"},{"title":"LitServe","href":"/docs/model-deployment/deploy-litserve"},{"title":"AWS Multi-Model Server","href":"/docs/model-deployment/deploy-model-mms"},{"title":"TorchServe","href":"/docs/model-deployment/deploy-torchserve"},{"title":"TensorFlow Serve (TFServe)","href":"/docs/model-deployment/deploy-tfserve"},{"title":"MlFlow","href":"/docs/model-deployment/deploy-model-mlflow"},{"title":"Migrate Sagemaker Pytorch Endpoint","href":"/docs/model-deployment/migrate-from-sagemaker-pytorch-endpoint"}]}]},{"group":"Service Deployment","pages":[{"title":"Introduction","href":"/docs/introduction-to-a-service"},{"title":"Deploy Your First Service","sidebarTitle":"Getting Started","href":"/docs/deploy-first-service"},{"group":"Configuring your service","pages":[{"title":"Dockerize Your Code","href":"/docs/dockerize-code"},{"title":"Define Ports And Domains","href":"/docs/define-ports-and-domains"},{"title":"Add authentication to your Endpoints","sidebarTitle":"Endpoint Authentication","href":"/docs/endpoint-authentication"},{"group":"Resources","pages":[{"title":"Set Resources","description":"Configure CPU, memory, ephemeral storage, GPU, and shared memory resources for your TrueFoundry deployments.","href":"/docs/resources-cpu-memory-storage"},{"title":"Using Fractional GPUs","href":"/docs/using-fractional-gpus"},{"title":"Using TPUs","description":"Deploy apps on Single Host TPU v4, v5e, v5p slices.","href":"/docs/using-tpus"}]},{"title":"Environment Variables \u0026 Secrets","sidebarTitle":"Environment Variables And Secrets","href":"/docs/environment-variables-and-secrets"},{"title":"Autoscaling Overview","sidebarTitle":"Autoscaling","href":"/docs/autoscaling-overview"},{"title":"Pause / Resume Service","href":"/docs/pause-resume-service"},{"title":"Scale Service to 0","href":"/docs/scale-service-to-0"},{"title":"Liveness/Readiness Probe","href":"/docs/liveness-readiness-probe"},{"title":"Rollout Strategy","href":"/docs/rollout-strategy"},{"title":"Access Cloud Services Like S3","href":"/docs/access-data-from-s3-or-other-clouds-services"},{"title":"Download Models/Artifacts","href":"/docs/download-and-cache-models"},{"title":"Redirect And Mirror Traffic","href":"/docs/intercepts"},{"title":"Kustomize Support","href":"/docs/kustomize"},{"title":"Sticky Routing","href":"/docs/sticky-routing"},{"title":"Mounting Volumes/Files","href":"/docs/mounting-volumes-service"},{"title":"Deploy a Service Programmatically","sidebarTitle":"Deploy Programatically","href":"/docs/deploy-service-programatically"},{"title":"Set Up CI/CD","href":"/docs/setting-up-cicd-for-your-service"},{"title":"Update, Rollback, Promote","href":"/docs/update-rollback-promote-your-service"},{"title":"View Logs, Metrics And Events","href":"/docs/monitor-your-service"},{"title":"Log \u0026 Monitor Custom Metrics","href":"/docs/log-monitor-custom-metrics"},{"title":"Setup Alerts","href":"/docs/setup-alerts"},{"title":"Docker Build Secrets","href":"/docs/docker-build-secrets"}]}]},{"group":"Job Deployment","pages":[{"title":"Jobs Introduction","sidebarTitle":"Introduction","href":"/docs/introduction-to-a-job"},{"title":"Deploy Job from GitHub","sidebarTitle":"Getting Started","href":"/docs/deploy-job-from-a-public-github-repository"},{"group":"Configuring your Job","pages":[{"title":"Parameterize A Job","href":"/docs/deploy-a-job-with-additional-parameters"},{"title":"Env Variables \u0026 Secrets for Jobs","sidebarTitle":"Environment Variables And Secrets","href":"/docs/environment-variables-and-secrets-jobs"},{"title":"Mounting Volumes","href":"/docs/mounting-volumes-job"},{"title":"Set Retries And Timeout","href":"/docs/retries-and-timeout"},{"title":"Set Concurrency Limit","href":"/docs/concurrency-limits"},{"title":"Deploy a Job Programmatically","sidebarTitle":"Deploy Programatically","href":"/docs/deploy-job-programatically"},{"title":"Monitor Your Job","href":"/docs/monitor-your-job"},{"title":"Access Data From S3 Or Other Clouds","href":"/docs/access-data-from-s3-or-other-clouds-jobs"},{"title":"Add Alerts To Your Job","href":"/docs/add-alerts-to-your-job"},{"title":"Logging Data in Job Runs","href":"/docs/job-runs-and-logging"}]},{"group":"Configuring your Spark Job","pages":[{"title":"Use blob storage with your Spark job","href":"/docs/deploy-a-spark-job-with-blob-storage"},{"title":"JVM metrics for Spark jobs","href":"/docs/spark-job-jvm-metrics"}]}]},{"group":"LLM Deployment","pages":[{"title":"Deploying LLMs","href":"/docs/deploying-an-llm-model-from-the-model-catalogue"},{"title":"Deploying NVIDIA NIM Models","href":"/docs/deploying-nvidia-nims"},{"title":"Benchmarking LLMs","href":"/docs/benchmarking-llms"}]},{"group":"LLM Finetuning","pages":[{"title":"Finetuning LLMs","href":"/docs/finetuning-a-model-from-the-model-catalogue"}]},{"group":"MCP Server Deployment","pages":[{"title":"MCP Server Deployment Overview","sidebarTitle":"Overview","href":"/docs/mcp-server-deployment/mcp-server-deployment-overview"},{"title":"Deploy MCP Server From Code","sidebarTitle":"Deploy From Code","href":"/docs/mcp-server-deployment/deploy-mcp-server-from-code"},{"title":"Deploy MCP Server from npx/uvx","sidebarTitle":"Deploy from npx/uvx","href":"/docs/mcp-server-deployment/deploy-mcp-server-from-npx-uvx"}]},{"group":"Workflow Deployment","pages":[{"title":"Introduction To Workflow","href":"/docs/introduction-to-workflow"},{"title":"Creating Your First Workflow","href":"/docs/creating-your-first-workflow"},{"title":"Interacting With Workflow","href":"/docs/interacting-with-workflow"},{"title":"Workflow Concepts","href":"/docs/workflow-concepts"},{"group":"Guides","pages":[{"title":"Creating A Workflow With Different Container Images","href":"/docs/creating-a-workflow-with-gpu-and-non-gpu-image"},{"title":"Using Dockerfile For Python Function Task","href":"/docs/python-function-tasks-with-dockefile"},{"title":"Using Raw Container Task","href":"/docs/container-task"},{"title":"Adding Environment Variable","href":"/docs/adding-environment-variable"},{"title":"Configuring Resources","href":"/docs/configuring-resources"},{"title":"Creating A Conditional Task","href":"/docs/conditional-task"},{"title":"Creating A Map Task","href":"/docs/creating-a-map-task"},{"title":"Creating A Cron Workflow","href":"/docs/cron-workflow"},{"title":"Creating A Spark Task","href":"/docs/spark-task"},{"title":"Creating A Databricks Task","href":"/docs/databricks-task"},{"title":"Creating A Snowflake Task","href":"/docs/snowflake-task"},{"title":"Task Config","href":"/docs/task-config"},{"title":"Attaching Mounts","href":"/docs/attaching-mounts"},{"title":"Passing Files/Artifacts Between Tasks","href":"/docs/passing-filesartifacts-between-tasks"},{"title":"Example Of Task Config With Different Parameters","href":"/docs/example-of-task-config-with-different-parameters"},{"title":"Adding Retries And Handling Failures","href":"/docs/adding-retries-and-handling-failures"},{"title":"Running Workflow Locally","href":"/docs/running-workflow-locally"},{"title":"Adding Alerts For Workflow","href":"/docs/adding-alerts-for-workflow"},{"title":"Handling Failure using Failure Task/Node","href":"/docs/handling-workflow-failure"},{"title":"Recovering a Failed Workflow Execution","href":"/docs/recovering-workflow-execution"}]}]},{"group":"Async Service Deployment","pages":[{"title":"Introduction to Async Service","href":"/docs/introduction-to-async-service"},{"title":"Deploy your first Async Service","href":"/docs/deploy-your-first-asyncservice"},{"group":"Configure async service","pages":[{"title":"Configure Ports","href":"/docs/configure-ports"},{"title":"Configure Queue","href":"/docs/configure-queue"},{"title":"Autoscaling","href":"/docs/autoscaling"},{"title":"Configure Sidecar","href":"/docs/configure-sidecar"}]},{"group":"Queue Integrations","pages":[{"title":"AWS SQS","href":"/docs/aws-sqs"}]},{"title":"Monitor Your Async Service","href":"/docs/monitoring-your-async-service"}]},{"group":"Volumes","pages":[{"title":"Introduction to Volume","href":"/docs/introduction-to-volume"},{"title":"Creating And Using Volumes","href":"/docs/creating-a-volume"}]},{"group":"Experiment Tracking","pages":[{"title":"ML Repo: Getting Started","sidebarTitle":"Getting Started","href":"/docs/ml-repo-quickstart"},{"title":"Create Runs and Log Data","href":"/docs/create-run"},{"title":"Artifacts and Artifact Versions","href":"/docs/log-artifacts"},{"title":"Models and Model Versions","href":"/docs/log-models"},{"title":"Huggingface Trainer Callback","href":"/docs/huggingface-trainer-callback"}]},{"group":"Advanced Features","pages":[{"title":"Secret Management","href":"/docs/manage-secrets"},{"title":"Deployment Guardrails and Policies","href":"/docs/applying-custom-policies"},{"title":"Deploy Helm Charts","href":"/docs/deploy-helm-charts"},{"title":"Deploy Kubernetes Manifests","href":"/docs/deploy-kubernetes-manifests"},{"title":"Ingress Controller Configuration","href":"/docs/ingress-controller-configuration"}]}]},{"tab":"API Reference","pages":[{"title":"API Reference","href":"/docs/truefoundry_sdk"}]},{"tab":"Integrations","pages":[{"title":"Integrations","href":"/docs/ai-gateway/ecosystem"}]},{"tab":"Changelog","groups":[{"group":" ","pages":[{"title":"Changelog","href":"/docs/changelog"},{"group":"Announcements","pages":[{"title":"Deprecation of Rate Limiting V1 — 2nd November, 2026","href":"/docs/change-announcements/deprecation-of-rate-limiting-v1"},{"title":"Move to Named Guardrail Policies — v0.179","href":"/docs/change-announcements/named-guardrails-config-v0.179"},{"title":"Perplexity Models Move to the Agent API","href":"/docs/change-announcements/perplexity-agent-api-migration"},{"title":"Plan-Based Feature Gating — v0.174.0","href":"/docs/change-announcements/plan-based-feature-gating-v0.174.0"},{"title":"Agent Builder Moving to Trueforge \u0026 Agent APIs Deprecated — v0.172","href":"/docs/change-announcements/agents-moving-to-trueforge"},{"title":"Removing Collaborators from Resources — v0.172.0","href":"/docs/change-announcements/removing-collaborators-from-resources"},{"title":"Deprecation of MCP tool_settings — 26th October, 2026","href":"/docs/change-announcements/deprecation-of-mcp-tool-settings"},{"title":"Deprecation of truefoundry \u003c 0.17.0 and truefoundry-sdk \u003c 0.4.0 packages","sidebarTitle":"Deprecation of truefoundry \u003c 0.17.0 and truefoundry-sdk \u003c 0.4.0 packages - 19th August 2026","href":"/docs/change-announcements/deprecation-of-truefoundry-below-0.17-and-api-ml"},{"title":"Minimum Python Version Raised to 3.10 — truefoundry v0.17.0","href":"/docs/change-announcements/python-3.10-minimum-truefoundry-cli-v0.17.0"},{"title":"Budget Limiting V2 — v0.158.0","href":"/docs/change-announcements/budget-limiting-v2-v0.158.0"},{"title":"Deprecation of MCP Servers APIs — 30th September, 2026","href":"/docs/change-announcements/deprecation-of-mcp-servers-v1-apis"},{"title":"Deprecation of Images and Plots in Job Runs — v0.156.0","href":"/docs/change-announcements/deprecation-of-images-and-plots-in-job-runs-v0.156.0"},{"title":"Moving to External Secrets Operator for Secrets Management — v0.151.5","href":"/docs/change-announcements/moving-to-eso-for-secret-management-v0.151.5"},{"title":"MCP DCR client_id \u0026 Redirect URL Validation — v0.148","href":"/docs/change-announcements/mcp-dcr-client-id-validation-v0.148"},{"title":"Deprecation of Routing Config","href":"/docs/change-announcements/deprecation-of-routing-config"},{"title":"Identity and Access Revamp — v0.143","href":"/docs/change-announcements/identity-and-access-revamp-v0.143"},{"title":"AI Gateway Metrics Rename — v0.136","href":"/docs/change-announcements/refactoring-of-ai-gateway-metrics"},{"title":"Deprecation of MCP Servers in Prompts","href":"/docs/change-announcements/deprecation-of-mcp-servers-in-prompts"},{"title":"Deprecation of Common Tools in Playground and Settings — v0.135","href":"/docs/change-announcements/deprecation-of-common-tools-v0-135-0"},{"title":"x-tfy-routing-config Header Removal — v0.133","href":"/docs/change-announcements/routing-config-header-removal-v0.133"},{"title":"Simplified GitOps CI/CD with tfy apply — v0.132","href":"/docs/change-announcements/simplified-gitops-cicd-v0.132"},{"title":"Legacy MCP OAuth Routes Removal — v0.134","href":"/docs/change-announcements/legacy-mcp-oauth-routes-removal"},{"title":"MCP Gateway URL and Transport Changes — v0.130","href":"/docs/change-announcements/mcp-gateway-url-transport-v0.130"},{"title":"Gemini CLI Model Registration Requirement — v0.118","href":"/docs/change-announcements/gemini-cli-model-registration-v0.118"},{"title":"Guardrails YAML Schema Change — v0.116","href":"/docs/change-announcements/guardrails-yaml-schema-change-v0.116"},{"title":"MCP Server Groups Removal — v0.112","href":"/docs/change-announcements/mcp-server-groups-removal-v0.112"}]}]}]}]},"basePath":"","title":"Using Fractional GPUs - TrueFoundry Docs","deploymentId":"dpl_DyXGnKeCjZyA3sUdfMJ7FjkdBefg","buildId":"dpl_DyXGnKeCjZyA3sUdfMJ7FjkdBefg","dynamics":["07c022d89b","a74be98ba7"],"chunks":["Frame-D0FT0qQs.js","DynamicMarkdown-CQM9Bhbv.js","zoom-image-lightbox-BQ4tbj_9.js","lightbox-handle-r17bVUz3.js"]}