Skip to content

Issues: ModelTC/lightllm

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

55 Open 118 Closed
55 Open 118 Closed
Author
Filter by author
Loading
Label
Filter by label
Loading
Use alt + click/return to exclude labels
or ⇧ + click/return for logical OR
Projects
Filter by project
Loading
Milestones
Filter by milestone
Loading
Assignee
Filter by who’s assigned
Sort
Sort by
Newest Oldest Most commented Least commented Recently updated Least recently updated Best match
Most reactions
👍 👎 😄 🎉 😕 ❤️ 🚀 👀

Issues list

有不通过http的其他推理入口吗 bug Something isn't working
#466 opened Jul 15, 2024 by mmdbhs
2
[Feature]: Suport for InternVL-Chat-V1-5 bug Something isn't working
#462 opened Jul 10, 2024 by JingofXin
1
How do you decide the tune params for triton kernels?
#457 opened Jul 5, 2024 by sleepwalker2017
Add Support to Florence-2 ! bug Something isn't working
#456 opened Jul 5, 2024 by KaifAhmad1
1
我看到列表里支持qwen7B，请问是否支持qwen1.5-14B呢？
#422 opened Jun 7, 2024 by koalaaaaaaaaa
1
[BUG]Ask aboout Qwen models with weight quantization . bug Something isn't working
#408 opened May 15, 2024 by Cesilina
1 task
2
[Question] How does lightllm implement nopad batching?
#405 opened Apr 25, 2024 by Tomorrowdawn
请问是否有计划支持MiniCPM-V-2
#404 opened Apr 23, 2024 by xiabo0816
1
[BUG] There already is a lightllm in pypi bug Something isn't working
#380 opened Mar 26, 2024 by rlippmann
1 task
weight only int4 is slower than cutlass int4
#362 opened Mar 19, 2024 by zhoutianzi666
1
Are there any efficient way to command kill the lightllm process?
#343 opened Mar 4, 2024 by yy9996
3
Qwen-14B-INT8 face the issue: 'QwenTransformerLayerWeight' object has no attribute 'q_weight_' bug Something isn't working
#333 opened Feb 20, 2024 by wangr0031
1
[BUG] stop_words bug Something isn't working
#326 opened Feb 2, 2024 by baisechundu
[BUG] Support for DeepSeek? bug Something isn't working
#325 opened Feb 2, 2024 by suhjohn
1
是否能支持sqlcoder系列模型
#310 opened Jan 22, 2024 by 2496289471
2
Inconsistent Output between LightLLM and Transformers Inference Library bug Something isn't working
#309 opened Jan 19, 2024 by Lvjinhong
2
请问lightllm可以离线推理吗，有没有参考代码 bug Something isn't working
#308 opened Jan 19, 2024 by monkeyZhy
1 task
1
请问现在支持Yi-34B的awq 4bit部署吗？
#291 opened Jan 9, 2024 by xyfZzz
5
What is the plan to support beam search
#286 opened Jan 8, 2024 by feifeibear
4
[Feature]请帮忙提供load_from_weight_dict(weight_dict)接口。
#277 opened Jan 4, 2024 by bingo787
10
Is there any comparison of the effects related to token attention? For example, compare with page attention
#268 opened Dec 27, 2023 by skykiseki
2
[BUG] Qwen-7B-Chat AttributeError: 'LlamaSplitFuseInferStateInfo' object has no attribute 'logn_values' bug Something isn't working
#239 opened Dec 2, 2023 by exceedzhang
1 task
4
no attribute 'qkv_weight_' AttributeError when load Qwen-14B-Chat-Int4 bug Something isn't working
#234 opened Dec 1, 2023 by jarviszeng-zjc
8
[BUG] Assertation error self.config["num_attention_heads"] % self.world_size_ == 0 when not perfectly divisible bug Something isn't working
#233 opened Nov 30, 2023 by getorca
1
An error occurred while deploying the 4bit version of Yi-34B-Chat
#230 opened Nov 29, 2023 by wx971025
1
ProTip! Mix and match filters to narrow down what you’re looking for.