We read every piece of feedback, and take your input very seriously.
To see all available qualifiers, see our documentation.
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
I want to add my tensorrt llm model that is served in tritonserver. But in tensorrt llm backend project the input template is diferent with the triton that dify supports. you can see it in: https://github.com/triton-inference-server/tensorrtllm_backend/blob/main/all_models/inflight_batcher_llm/ensemble/config.pbtxt
can you add this template?
tensorrtllm_backend is officially supported by nvidia. If you add thier template, it will help us.
The text was updated successfully, but these errors were encountered:
Please checkout the new version https://github.com/langgenius/dify/releases/tag/1.0.0-beta.1, if you have new model providers or tools that you want to contribute. #11588
Sorry, something went wrong.
No branches or pull requests
Self Checks
1. Is this request related to a challenge you're experiencing? Tell me about your story.
I want to add my tensorrt llm model that is served in tritonserver.
But in tensorrt llm backend project the input template is diferent with the triton that dify supports. you can see it in:
https://github.com/triton-inference-server/tensorrtllm_backend/blob/main/all_models/inflight_batcher_llm/ensemble/config.pbtxt
can you add this template?
2. Additional context or comments
tensorrtllm_backend is officially supported by nvidia. If you add thier template, it will help us.
3. Can you help us with this feature?
The text was updated successfully, but these errors were encountered: