How to add kvcache functionality to the current inference framework#63
Open
How can I add functionality to the current source code to support KV cache? If I want to implement such a feature, what steps should I take? And are there any potential risks or incompatibilities? I hope you can give me some guidance as I'd like to try adding this feature.
AI Analysis
This issue appears to be discussing a feature request or bug report related to the repository. Based on the content, it seems to be still under discussion. The issue was opened by mailonghua and has received 0 comments.
Add a comment
Comment form would go here