Skip to content

[Feature] optimize prefix cache - #3107

Closed
ltd0924 wants to merge 18 commits into
PaddlePaddle:developfrom
ltd0924:RL
Closed

[Feature] optimize prefix cache#3107
ltd0924 wants to merge 18 commits into
PaddlePaddle:developfrom
ltd0924:RL

Conversation

@ltd0924

@ltd0924 ltd0924 commented Jul 31, 2025

Copy link
Copy Markdown
Collaborator
  • Decouple cache transfer and cache messager
  • Modify the location of cache initialization
    • Prefix Caching:
      Cache initialization is shifted from the cache transfer process to the worker process.
    • Prefill-Decode Disaggregation:
      Cache initialization is handled by the cache messager process.

@paddle-bot

paddle-bot Bot commented Jul 31, 2025

Copy link
Copy Markdown

Thanks for your contribution!

rainyfly and others added 3 commits August 4, 2025 20:32
* Support external module

* Support external module

* Support external module

* Support external module

* refactor code to make it more clear

* refactor code to make it more clear

* refactor code to make it more clear

* refactor code to make it more clear

* fix according to review

* fix according to review

* fix according to review

* fix according to review

* fix according to review

* fix according to review

* fix bug

* fix bug

* fix bug

* merge

---------

Co-authored-by: root <root@tjdm-inf-sci-k8s-hzz2-h12ni8-0202.tjdm.baidu.com>
@paddle-bot paddle-bot Bot closed this Aug 18, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants