【LLM】模型参数支持列表

[DrownFish19]

模型参数支持专区

大家好，PaddleNLP 团队在这里为大家整理了各个模型参数的详细信息，方便大家使用。

模型参数

Base Models

Model	0.5B	1~2B	3~4B	6~7B	13~14B	30~32B	50~60B	65~72B	110B	>110B
LLaMA				✅	✅	✅		✅
LLaMA2				✅	✅			✅
LLaMA3				✅	✅			✅
Baichuan				✅	✅
Baichuan2				✅	✅
Bloom	✅	✅	✅	✅
ChatGLM				✅
ChatGLM2				✅
ChatGLM3				✅
Gemma		✅		✅
Mistral				✅
Mixtral							✅
OPT	✅	✅	✅	✅	✅	✅		✅
Qwen				✅	✅			✅
Qwen1.5	✅	✅	✅	✅	✅	✅		✅	✅
Qwen2	✅	✅		✅				✅

Chat Models

Model	0.5B	1~2B	3~4B	6~7B	13~14B	30~32B	50~60B	65~72B	110B	>110B
LLaMA
LLaMA2				✅	✅			✅
LLaMA3				✅				✅
Baichuan					✅
Baichuan2				✅	✅
Bloom	✅	✅	✅	✅
ChatGLM				✅
ChatGLM2				✅
ChatGLM3				✅
Gemma		✅		✅
Mistral				✅
Mixtral							✅
OPT
Qwen				✅	✅			✅
Qwen1.5	✅	✅	✅	✅	✅	✅		✅	✅
Qwen2	✅	✅		✅				✅

模型系列	模型名称
LLaMA	facebook/llama-7b, facebook/llama-13b, facebook/llama-30b, facebook/llama-65b
LLama2	meta-llama/Llama-2-7b, meta-llama/Llama-2-7b-chat, meta-llama/Llama-2-13b, meta-llama/Llama-2-13b-chat, meta-llama/Llama-2-70b, meta-llama/Llama-2-70b-chat
LLama3	meta-llama/Meta-Llama-3-8B, meta-llama/Meta-Llama-3-8B-Instruct, meta-llama/Meta-Llama-3-70B, meta-llama/Meta-Llama-3-70B-Instruct
Baichuan	baichuan-inc/Baichuan-7B, baichuan-inc/Baichuan-13B-Base, baichuan-inc/Baichuan-13B-Chat
Baichuan2	baichuan-inc/Baichuan2-7B-Base, baichuan-inc/Baichuan2-7B-Chat, baichuan-inc/Baichuan2-13B-Base, baichuan-inc/Baichuan2-13B-Chat
Bloom	bigscience/bloom-560m, bigscience/bloom-560m-bf16, bigscience/bloom-1b1, bigscience/bloom-3b, bigscience/bloom-7b1, bigscience/bloomz-560m, bigscience/bloomz-1b1, bigscience/bloomz-3b, bigscience/bloomz-7b1-mt, bigscience/bloomz-7b1-p3, bigscience/bloomz-7b1, bellegroup/belle-7b-2m
ChatGLM	THUDM/chatglm-6b, THUDM/chatglm-6b-v1.1
ChatGLM2	THUDM/chatglm2-6b
ChatGLM3	THUDM/chatglm3-6b
Gemma	google/gemma-7b, google/gemma-7b-it, google/gemma-2b, google/gemma-2b-it
Mistral	mistralai/Mistral-7B-Instruct-v0.3, mistralai/Mistral-7B-v0.1
Mixtral	mistralai/Mixtral-8x7B-Instruct-v0.1
OPT	facebook/opt-125m, facebook/opt-350m, facebook/opt-1.3b, facebook/opt-2.7b, facebook/opt-6.7b, facebook/opt-13b, facebook/opt-30b, facebook/opt-66b, facebook/opt-iml-1.3b, opt-iml-max-1.3b
Qwen	qwen/qwen-7b, qwen/qwen-7b-chat, qwen/qwen-14b, qwen/qwen-14b-chat, qwen/qwen-72b, qwen/qwen-72b-chat,
Qwen1.5	Qwen/Qwen1.5-0.5B, Qwen/Qwen1.5-0.5B-Chat, Qwen/Qwen1.5-1.8B, Qwen/Qwen1.5-1.8B-Chat, Qwen/Qwen1.5-4B, Qwen/Qwen1.5-4B-Chat, Qwen/Qwen1.5-7B, Qwen/Qwen1.5-7B-Chat, Qwen/Qwen1.5-14B, Qwen/Qwen1.5-14B-Chat, Qwen/Qwen1.5-32B, Qwen/Qwen1.5-32B-Chat, Qwen/Qwen1.5-72B, Qwen/Qwen1.5-72B-Chat, Qwen/Qwen1.5-110B, Qwen/Qwen1.5-110B-Chat, Qwen/Qwen1.5-MoE-A2.7B, Qwen/Qwen1.5-MoE-A2.7B-Chat
Qwen2	Qwen/Qwen2-0.5B, Qwen/Qwen2-0.5B-Instruct, Qwen/Qwen2-1.5B, Qwen/Qwen2-1.5B-Instruct, Qwen/Qwen2-7B, Qwen/Qwen2-7B-Instruct, Qwen/Qwen2-72B, Qwen/Qwen2-72B-Instruct, Qwen/Qwen2-57B-A14B, Qwen/Qwen2-57B-A14B-Instruct

[jzhang533]

links in the table are broken.