old_llama_for_causal_lm_forward() got an unexpected keyword argument 'cache_position'

[Kevinstone-199898]

When I was running a demo following the code in Quick Start for DuoAttention, an error occurs as follow:

Traceback (most recent call last):
File "/home/sxy/duo-attention/demo.py", line 57, in
outputs = model.generate(
File "/home/sxy/python_env/duo/lib/python3.10/site-packages/torch/utils/_contextlib.py", line 116, in decorate_context
return func(*args, **kwargs)
File "/home/sxy/python_env/duo/lib/python3.10/site-packages/transformers/generation/utils.py", line 2047, in generate
result = self._sample(
File "/home/sxy/python_env/duo/lib/python3.10/site-packages/transformers/generation/utils.py", line 3007, in _sample
outputs = self(**model_inputs, return_dict=True)
File "/home/sxy/python_env/duo/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1739, in _wrapped_call_impl
return self._call_impl(*args, **kwargs)
File "/home/sxy/python_env/duo/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1750, in _call_impl
return forward_call(*args, **kwargs)
TypeError: old_llama_for_causal_lm_forward() got an unexpected keyword argument 'cache_position'

Does anyone know how to solve this problem?

[pingzhili]

same here