Training · Containers and Kubernetes
The analogy. Shipping containers. The box is a standard size, the crane does not care what is inside, and the port decides where each box goes. A Service is the berth number painted on the dock.
From image to replica: the lecture behind this lab covers the mental model, the commands you are about to use, and the four things that usually go wrong.
Read the lecture before you startAI Platform: Request a GPU for Inference
Schedule an inference workload that asks the cluster for an accelerator, runs the vLLM image, and is reachable on 8000. The sandbox simulates the scheduler and grades the manifest - no GPU hardware is provisioned.
Activities
01Apply the llm-inference Deployment
A Deployment named llm-inference must exist in the default namespace.
02Run the vLLM server image
The Deployment must run an image from the vllm/ repository, for example vllm/vllm-openai:v0.6.3.
03Request one accelerator
resources.limits must request nvidia.com/gpu: 1. That is the scheduling request the GPU operator would act on.
04Expose the server on 8000
A Service named llm must route port 8000 to the workload.