Hugging Face's logo

nvidia
/
Llama-3_3-Nemotron-Super-49B-v1_5

Text Generation
Transformers
Safetensors
PyTorch
English
nemotron-nas
nvidia
llama-3
conversational
custom_code
Model card Files
xet
Community
12

Since `transformers` v4.56.0` the dictionary `ALL_STATIC_CACHE_IMPLEMENTATIONS` replaced `NEED_SETUP_CACHE_CLASSES_MAPPING`

#9
by blewis-hir - opened
base: refs/heads/main
←
from: refs/pr/9
Discussion Files changed
+14
-2
blewis-hir
•
edited about 1 month ago

This should fix using the model with new transformers versions

Since `transformers` v4.56.0` the dictionary `ALL_STATIC_CACHE_IMPLEMENTATIONS` replaced the existing dictionaryf4a9fe8f
blewis-hir changed pull request title from Since `transformers` v4.56.0` the dictionary `ALL_STATIC_CACHE_IMPLEMENTATIONS` replaced the existing dictionary to Since `transformers` v4.56.0` the dictionary `ALL_STATIC_CACHE_IMPLEMENTATIONS` replaced `NEED_SETUP_CACHE_CLASSES_MAPPING`
blewis-hir

See these transformers commits:
https://github.com/huggingface/transformers/commit/a2e76b908b1fd68a71117be1bcc9389869c6f445
https://github.com/huggingface/transformers/commit/242bb2cafccec9f90479f5f688bca9d240b1031f

Make changes backwards compatible with old `transformers` versions2b886402
Fix incorrect version check277a89dd
Fix crash on model forward with fix by @shmuli57d602ba
NePe
•
edited 15 days ago

Hmm is this really enough? I had to change a few lines in variable_cache.py too for the latest transformers. https://huggingface.co/nvidia/Llama-3_3-Nemotron-Super-49B-v1_5/discussions/12

blewis-hir

@NePe It worked for me... but I will check and see if I missed something that you fixed

NePe

I checked your code and it has a few syntax errors/typos and ALL_STATIC_CACHE_IMPLEMENTATIONS is a tuple of str so there is no way this could work:
https://github.com/huggingface/transformers/blob/v4.57.3/src/transformers/generation/configuration_utils.py#L46

Ready to merge
This branch is ready to get merged automatically.

Sign up or log in to comment