单位网站建设费用账务处理,策划公司有哪些,品牌线上推广渠道有哪些,网站怎么在工信部备案信息1 部署
1.1 申请权限
在huggingface的gemma界面#xff0c;点击“term”以申请gemma访问权限
https://huggingface.co/google/gemma-7b 然后接受条款
1.2 添加hugging对应的token
如果直接用gemma提供的代码#xff0c;会出现如下问题#xff1a;
from transformers i…1 部署
1.1 申请权限
在huggingface的gemma界面点击“term”以申请gemma访问权限
https://huggingface.co/google/gemma-7b 然后接受条款
1.2 添加hugging对应的token
如果直接用gemma提供的代码会出现如下问题
from transformers import AutoTokenizer, AutoModelForCausalLMtokenizer AutoTokenizer.from_pretrained(google/gemma-7b)
model AutoModelForCausalLM.from_pretrained(google/gemma-7b)input_text Write me a poem about Machine Learning.
input_ids tokenizer(input_text, return_tensorspt)outputs model.generate(**input_ids)
print(tokenizer.decode(outputs[0]))这时候就需要添加自己hugging的token了
import os
os.environ[HF_TOKEN] ....
token的位置在 2 gemma 模型官方样例
2.0 gemma介绍
Gemma是Google推出的一系列轻量级、最先进的开放模型基于创建Gemini模型的相同研究和技术构建。它们是文本到文本的、仅解码器的大型语言模型提供英语版本具有开放的权重、预训练的变体和指令调优的变体。Gemma模型非常适合执行各种文本生成任务包括问答、摘要和推理。它们相对较小的尺寸使得可以在资源有限的环境中部署例如笔记本电脑、桌面电脑或您自己的云基础设施使每个人都能获得最先进的AI模型促进创新。
2.1 文本生成
2.1.1 CPU上执行
from transformers import AutoTokenizer, AutoModelForCausalLMAutoTokenizer用于加载预训练的分词器
AutoModelForCausalLM则用于加载预训练的因果语言模型Causal Language Model这种模型通常用于文本生成任务
tokenizer AutoTokenizer.from_pretrained(google/gemma-2b,token。。。)
#加载gemma-2b的预训练分词器
model AutoModelForCausalLM.from_pretrained(google/gemma-2b,token。。。)
#加载gemma-2b的预训练语言生成模型使用其他几个进行文本续写其他的地方是一样的就这里加载的预训练模型不同
google/gemma-2b-it
google/gemma-7b
google/gemma-7b-it
input_text Write me a poem about Machine Learning.
#定义了要生成文本的初始输入
input_ids tokenizer(input_text, return_tensorspt)
#使用前面加载的分词器将input_text转换为模型可理解的数字表示【token id】
#return_tensorspt表明返回的是PyTorch张量格式。outputs model.generate(**input_ids)
#使用模型和转换后的输入input_ids来生成文本print(tokenizer.decode(outputs[0]))
#将生成的文本令牌解码为人类可读的文本并打印出来 2.1.2 GPU上执行
多GPU 前面的一样model AutoModelForCausalLM.from_pretrained(google/gemma-2b, device_mapauto)input_text Write me a poem about Machine Learning.
input_ids tokenizer(input_text, return_tensorspt).to(model.device)
后面的一样指定单GPU 前面的一样model AutoModelForCausalLM.from_pretrained(google/gemma-2b, device_mapcuda:0)input_text Write me a poem about Machine Learning.
input_ids tokenizer(input_text, return_tensorspt).to(model.device)
后面的一样2.1.3 设置生成文本的长度
其他的不变和2.1.1比只修改outputs这一行
outputs model.generate(**input_ids,max_length100) 2.2 使用chat格式
目前gemma我没试出来同时放n个不同的chat怎么搞目前只放了一个
2.2.1 模型部分
和文本生成相同从预训练模型中导入一个分词器一个CausalLM
# pip install accelerate
from transformers import AutoTokenizer, AutoModelForCausalLMtokenizer AutoTokenizer.from_pretrained(google/gemma-2b-it)
model AutoModelForCausalLM.from_pretrained(google/gemma-2b-it, device_mapcuda:0)
2.2.2 获取prompt
chat[{role: user, content: I am going to Paris, what should I see?},{role: assistant,content: \
Paris, the capital of France, is known for its stunning architecture, art museums, historical landmarks, and romantic atmosphere. Here are some of the top attractions to see in Paris:
1. The Eiffel Tower: The iconic Eiffel Tower is one of the most recognizable landmarks in the world and offers breathtaking views of the city.
2. The Louvre Museum: The Louvre is one of the worlds largest and most famous museums, housing an impressive collection of art and artifacts, including the Mona Lisa.
3. Notre-Dame Cathedral: This beautiful cathedral is one of the most famous landmarks in Paris and is known for its Gothic architecture and stunning stained glass windows.
These are just a few of the many attractions that Paris has to offer. With so much to see and do, its no wonder that Paris is one of the most popular tourist destinations in the world.,},{role: user, content: What is so great about #1?},]prompt tokenizer.apply_chat_template(chat, tokenizeFalse,add_generation_promptTrue)
#tokenizeFalse这个参数控制是否在应用模板之后对文本进行分词处理。False表示不进行分词处理#add_generation_promptTrue这个参数控制是否在处理后的文本中添加生成提示。
#True意味着会添加一个提示这个提示通常用于指导模型进行下一步的文本生成
#添加的提示是start_of_turnmodelprint(prompt)bosstart_of_turnuser
I am going to Paris, what should I see?end_of_turn
start_of_turnmodel
Paris, the capital of France, is known for its stunning architecture, art museums, historical landmarks, and romantic atmosphere. Here are some of the top attractions to see in Paris:
1. The Eiffel Tower: The iconic Eiffel Tower is one of the most recognizable landmarks in the world and offers breathtaking views of the city.
2. The Louvre Museum: The Louvre is one of the worlds largest and most famous museums, housing an impressive collection of art and artifacts, including the Mona Lisa.
3. Notre-Dame Cathedral: This beautiful cathedral is one of the most famous landmarks in Paris and is known for its Gothic architecture and stunning stained glass windows.
These are just a few of the many attractions that Paris has to offer. With so much to see and do, its no wonder that Paris is one of the most popular tourist destinations in the world.end_of_turn
start_of_turnuser
What is so great about #1?end_of_turn
start_of_turnmodel2.2.3 分词
inputs tokenizer.encode(prompt, add_special_tokensFalse, return_tensorspt)
inputstensor([[ 2, 106, 1645, 108, 235285, 1144, 2319, 577, 7127,235269, 1212, 1412, 590, 1443, 235336, 107, 108, 106,2516, 108, 29437, 235269, 573, 6037, 576, 6081, 235269,603, 3836, 604, 1277, 24912, 16333, 235269, 3096, 52054,235269, 13457, 82625, 235269, 578, 23939, 13795, 235265, 5698,708, 1009, 576, 573, 2267, 39664, 577, 1443, 575,7127, 235292, 108, 235274, 235265, 714, 125957, 22643, 235292,714, 34829, 125957, 22643, 603, 974, 576, 573, 1546,93720, 82625, 575, 573, 2134, 578, 6952, 79202, 7651,576, 573, 3413, 235265, 108, 235284, 235265, 714, 91182,9850, 235292, 714, 91182, 603, 974, 576, 573, 2134,235303, 235256, 10155, 578, 1546, 10964, 52054, 235269, 12986,671, 20110, 5488, 576, 3096, 578, 51728, 235269, 3359,573, 37417, 25380, 235265, 108, 235304, 235265, 32370, 235290,76463, 41998, 235292, 1417, 4964, 57046, 603, 974, 576,573, 1546, 10964, 82625, 575, 7127, 578, 603, 3836,604, 1277, 60151, 16333, 578, 24912, 44835, 5570, 11273,235265, 108, 8652, 708, 1317, 476, 2619, 576, 573,1767, 39664, 674, 7127, 919, 577, 3255, 235265, 3279,712, 1683, 577, 1443, 578, 749, 235269, 665, 235303,235256, 793, 5144, 674, 7127, 603, 974, 576, 573,1546, 5876, 18408, 42333, 575, 573, 2134, 235265, 107,108, 106, 1645, 108, 1841, 603, 712, 1775, 1105,1700, 235274, 235336, 107, 108, 106, 2516, 108]])2.2.4 生成结果
和文本生成一样也是model.generate
outputs model.generate(input_idsinputs.to(model.device), max_new_tokens500)
print(tokenizer.decode(outputs[0]))bosstart_of_turnuser
I am going to Paris, what should I see?end_of_turn
start_of_turnmodel
Paris, the capital of France, is known for its stunning architecture, art museums, historical landmarks, and romantic atmosphere. Here are some of the top attractions to see in Paris:
1. The Eiffel Tower: The iconic Eiffel Tower is one of the most recognizable landmarks in the world and offers breathtaking views of the city.
2. The Louvre Museum: The Louvre is one of the worlds largest and most famous museums, housing an impressive collection of art and artifacts, including the Mona Lisa.
3. Notre-Dame Cathedral: This beautiful cathedral is one of the most famous landmarks in Paris and is known for its Gothic architecture and stunning stained glass windows.
These are just a few of the many attractions that Paris has to offer. With so much to see and do, its no wonder that Paris is one of the most popular tourist destinations in the world.end_of_turn
start_of_turnuser
What is so great about #1?end_of_turn
start_of_turnmodel
The Eiffel Tower is one of the most iconic landmarks in the world and offers breathtaking views of the city. It is a symbol of French engineering and architecture and is a must-see for any visitor to Paris.eos
