https://modelscope.cn/models/Qwen/Qwen3-32B

部署时，您可以使用 sglang>=0.4.6.post1 或 vllm>=0.8.4 或创建一个与 OpenAI 兼容的 API 端点：

SGLANG_USE_MODELSCOPE=1 python -m sglang.launch_server --model-path Qwen/Qwen3-32B --reasoning-parser qwen3

VLLM_USE_MODELSCOPE=1 vllm serve Qwen/Qwen3-32B --enable-reasoning --reasoning-parser deepseek_r1

我爱用VLLM+Docker+不要reasoning：

# 下载镜像
docker pull vllm/vllm-openai:latest

# 下载模型
modelscope download Qwen/Qwen3-32B --local-dir ./Qwen/Qwen3-32B

# 启动服务
docker run -d --gpus '"device=4,5,6,7"' \
    -v /data/xiedong/Qwen/Qwen3-32B:/model \
    -p 8028:8000 \
    vllm/vllm-openai:latest \
    --model /model --gpu_memory_utilization=0.9 --tensor-parallel-size 4 --pipeline-parallel-size 1 --max-model-len 10000 --served-model-name gpt

要reasoning的话，run 要加入--enable-reasoning --reasoning-parser deepseek_r1

请求：

curl -X POST "http://10.136.19.27:8028/v1/chat/completions" \
  -H "Authorization: Bearer YOUR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "gpt",
    "messages": [
      {
        "role": "user",
        "content": "你是谁？"
      }
    ],
    "temperature": 0.7,
    "top_p": 0.8,
    "top_k": 20,
    "max_tokens": 4096,
    "presence_penalty": 1.5
  }'

考虑效率，不需要思考就加上"chat_template_kwargs": {"enable_thinking": false}

curl -X POST "http://10.136.19.27:8028/v1/chat/completions" \
  -H "Authorization: Bearer YOUR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "gpt",
    "messages": [
      {
        "role": "user",
        "content": "你是谁？"
      }
    ],
    "temperature": 0.7,
    "top_p": 0.8,
    "top_k": 20,
    "max_tokens": 4096,
    "presence_penalty": 1.5,
    "chat_template_kwargs": {"enable_thinking": false}
  }'