1.部署model scope

魔搭社区

pip install -i Simple Index modelscope

python升级

部署pytorch

pip install  -i https://pypi.doubanio.com/simple  torch torchvision torchaudio

安装rust

官网下载安装

升级pip到最新版本

pip install --upgrade pip -i https://pypi.tuna.tsinghua.edu.cn/simple

部署transformer

​pip install -i https://pypi.doubanio.com/simple  transformers

调整visio studio python运行版本

如何在Visual Studio代码中更改python版本？ - 腾讯云开发者社区 - 腾讯云

运行modelscope样例脚本

from modelscope import AutoModelForCausalLM, AutoTokenizer

model_name = "qwen/Qwen2.5-7B-Instruct"

model = AutoModelForCausalLM.from_pretrained(

    model_name,

    torch_dtype="auto",

    device_map="auto"

)

tokenizer = AutoTokenizer.from_pretrained(model_name)

prompt = "Give me a short introduction to large language model."

messages = [

    {"role": "system", "content": "You are Qwen, created by Alibaba Cloud. You are a helpful assistant."},

    {"role": "user", "content": prompt}

]

text = tokenizer.apply_chat_template(

    messages,

    tokenize=False,

    add_generation_prompt=True

)

model_inputs = tokenizer([text], return_tensors="pt").to(model.device)

generated_ids = model.generate(

    **model_inputs,

    max_new_tokens=512

)

generated_ids = [

    output_ids[len(input_ids):] for input_ids, output_ids in zip(model_inputs.input_ids, generated_ids)

]

response = tokenizer.batch_decode(generated_ids, skip_special_tokens=True)[0]

print(response)