Skip to content

Issues: NVIDIA/FasterTransformer

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

231 Open 372 Closed
231 Open 372 Closed
Author
Filter by author
Label
Filter by label
Use alt + click/return to exclude labels
or ⇧ + click/return for logical OR
Projects
Filter by project
Milestones
Filter by milestone
Assignee
Filter by who’s assigned
Sort
Sort by
Newest Oldest Most commented Least commented Recently updated Least recently updated Best match
Most reactions
👍 👎 😄 🎉 😕 ❤️ 🚀 👀

Issues list

Incorrect inline ptx device assembly code usage bug Something isn't working
#766 opened Oct 13, 2023 by zhiweij1
CUDA code compile error with clang: function template partial specialization is not allowed bug Something isn't working
#765 opened Oct 12, 2023 by zhiweij1
How to calculate local batch size?
#764 opened Oct 11, 2023 by fotstrt
src/fastertransformer/kernels/decoder_masked_multihead_attention /decoder_masked_multihead_attention_template.hpp:36 open this macro definition, it'll find a build error bug Something isn't working
#763 opened Oct 11, 2023 by pengl
terminate called after throwing an instance of 'std::runtime_error'
#761 opened Sep 19, 2023 by HalFTeen
fastertransformer/utils/nccl_utils.cc:62 'unhandled cuda error' bug Something isn't working
#760 opened Sep 14, 2023 by wangweiwei1188
Support for "no_repeat_ngram_size" parameter for generation
#759 opened Sep 13, 2023 by shreysingla11
Does FasterTransformer use FlashAttention?
#758 opened Sep 9, 2023 by niyunsheng
Which part should I modify to achieve inference pipeline schedule (like micro-batch)?
#757 opened Sep 6, 2023 by dannyxiaocn
flashattention only enabled for gpt-styled models
#754 opened Aug 31, 2023 by flexwang
5
How to get a npz file that satisfy the input requirement? bug Something isn't working
#753 opened Aug 30, 2023 by jy00161yang
1
[Long seq length] GPT Seq length constrain
#752 opened Aug 25, 2023 by zhen-jia
10
specify the recognition language for Whisper
#751 opened Aug 24, 2023 by echodjx
Is it possible to serve GPT-NeoX ONNX exported through optimum?
#749 opened Aug 23, 2023 by sonientaegi
[feature request] transformer on orin
#748 opened Aug 22, 2023 by superpigforever
core dumped of swin model bug Something isn't working
#746 opened Aug 17, 2023 by chiemon
Can't run translate_example.py found in T5 model
#745 opened Aug 15, 2023 by EmanElrefai12
Failed building t5 model in FastTransformer (Reached 82% then stopped) bug Something isn't working
#744 opened Aug 15, 2023 by EmanElrefai12
GPT-MoE supports for expert parallel
#743 opened Aug 15, 2023 by YJHMITWEB
decoupled model with non-streaming mode
#741 opened Aug 10, 2023 by flexwang
decoupled mode not working when beam_width > 1
#740 opened Aug 10, 2023 by flexwang
Limit cuda memory growth
#739 opened Aug 9, 2023 by coderchem
The int8 model saved by run_squad can't import by effective_transformer bug Something isn't working
#738 opened Aug 9, 2023 by modkzs
is_return_output_log_probs doesn't return logits for T5 model bug Something isn't working
#737 opened Aug 8, 2023 by swairshah
Using faster transformers to infer the bloom model, the accuracy rate is 0 bug Something isn't working
#736 opened Jul 29, 2023 by hurun
1
ProTip! What’s not been updated in a month: updated:<2023-09-12.