▪ claudecodedaily

The task is fairly straightforward. Codex has completely bombed it so far by focusing too much on writing a script to sweep through the executions vs focusing on the actual work. We'll see if Codex does better on the 2nd run. Claude Code and Opus 4.5 are happily chugging along on the other DGX Spark. Goal: Determine under what runtime settings vLLM will load the `openai/gpt-oss-120b` model. If the model loads successfully use `nvidia-smi` to extract the system memory utilization after. Create a .csv file of executions, vLLM version, arguments, whether or not vLLM successfully loaded the model, and memory utilization. Method: Cycle through various runtime parameters 1. Max model sequence lengths: 16K, 32K, 64K, and 128K 2. GPU memory utilization: 70%, 85%, 90% 3. Max batch sequences: 16, 8, 4, 2, 1 4. Without and with quantized KV cache via `--kv-cache-dtype fp8` (you may hit a bug with FP8/FlashInfer and if it happens, use `VLLM_ATTENTION_BACKEND=FLASH_ATTN` and `VLLM_FLASH_ATTN_VERSION=2` to force FlashAttention v2. Notes: - `--trust-remote-code is fine` - Flush system cache before vLLM executions via `sudo sh -c 'sync; echo 3 > /proc/sys/vm/drop_caches’` Use the NVIDIA vLLM container image, e.g.: docker run --rm --gpus all --ipc=host --ulimit memlock=-1 --ulimit stack=67108864 -v ~/.cache/huggingface:/root/.cache/huggingface -p 8000:8000 -it https://t.co/Ut014o1mzT vllm serve openai/gpt-oss-120b --gpu-memory-utilization 0.70 --max-model-len 32768 --max-num-seqs 16 --trust-remote-code