problems with using huggingface flash attention 2 backend on windows

[BBC-Esq]

I'm having multiple problems testing out the huggingface backend. Here's one example error:

  File "C:\PATH\Scripts\WhisperS2T-batch-process\Lib\site-packages\transformers\tokenization_utils.py", line 391, in added_tokens_encoder
    return {k.content: v for v, k in sorted(self._added_tokens_decoder.items(), key=lambda item: item[0])}
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\PATH\Scripts\WhisperS2T-batch-process\Lib\site-packages\transformers\tokenization_utils.py", line 391, in <dictcomp>
    return {k.content: v for v, k in sorted(self._added_tokens_decoder.items(), key=lambda item: item[0])}
                             ^^^^
TypeError: 'tokenizers.AddedToken' object does not support the context manager protocol

And here's the script that produced it:

import whisper_s2t

model = whisper_s2t.load_model(model_identifier="large-v2", backend='HuggingFace')

files = ['test_audio_flac.flac']
lang_codes = ['en']
tasks = ['transcribe']
initial_prompts = [None]

out = model.transcribe_with_vad(files,
                                lang_codes=lang_codes,
                                tasks=tasks,
                                initial_prompts=initial_prompts,
                                batch_size=8)

op_files = ["transcription.txt"]

whisper_s2t.write_outputs(out, format='txt', op_files=op_files)

if __name__ == "__main__":
    main()

Here's another kind of error that I got:

  File "C:\PATH\Scripts\WhisperS2T-batch-process\Lib\site-packages\transformers\tokenization_utils.py", line 391, in added_tokens_encoder
    return {k.content: v for v, k in sorted(self._added_tokens_decoder.items(), key=lambda item: item[0])}
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\PATH\Scripts\WhisperS2T-batch-process\Lib\site-packages\transformers\tokenization_utils.py", line 391, in <dictcomp>
    return {k.content: v for v, k in sorted(self._added_tokens_decoder.items(), key=lambda item: item[0])}
            ^^^^^^^^^
AttributeError: 'list_iterator' object has no attribute 'content'

It DOES NOT make sense because this error is different even though I only changed the "batch_size" from 8 to 16 in the above script?

I'm on Windows and am using a custom flash attention 2 wheel from here:

https://github.com/bdashore3/flash-attention/releases/

I've struggled for hours to get flash attention 2 to work on Windows and with whispers2t specifically, using multiple scripts, not just the one above.

Any help would be much appreciated. I'd love to get the huggingface backend working along with ctranslate2's...Here is my pip freeze if it helps...

accelerate==0.25.0
aiohttp==3.9.3
aiosignal==1.3.1
attrs==23.2.0
certifi==2024.2.2
cffi==1.16.0
charset-normalizer==3.3.2
colorama==0.4.6
coloredlogs==15.0.1
ctranslate2==4.0.0
datasets==2.18.0
dill==0.3.8
einops==0.7.0
filelock==3.13.1
flash_attn @ https://github.com/bdashore3/flash-attention/releases/download/v2.5.2/flash_attn-2.5.2+cu122torch2.2.0cxx11abiFALSE-cp311-cp311-win_amd64.whl#sha256=9a6a9bd30861a988b95e64402adb4fa15f84b1fdcae31251ab5fc0e7f691c0f2
frozenlist==1.4.1
fsspec==2024.2.0
huggingface-hub==0.21.3
humanfriendly==10.0
idna==3.6
Jinja2==3.1.3
llvmlite==0.42.0
markdown-it-py==3.0.0
MarkupSafe==2.1.5
mdurl==0.1.2
more-itertools==10.2.0
mpmath==1.3.0
multidict==6.0.5
multiprocess==0.70.16
networkx==3.2.1
ninja==1.11.1.1
numba==0.59.0
numpy==1.26.4
nvidia-ml-py==12.535.133
openai-whisper==20231117
optimum==1.15.0
packaging==23.2
pandas==2.2.1
platformdirs==4.2.0
protobuf==4.25.3
psutil==5.9.8
pyarrow==15.0.0
pyarrow-hotfix==0.6
pycparser==2.21
Pygments==2.17.2
pyreadline3==3.4.1
PySide6==6.6.1
PySide6-Addons==6.6.1
PySide6-Essentials==6.6.1
python-dateutil==2.9.0.post0
pytz==2024.1
PyYAML==6.0.1
regex==2023.12.25
requests==2.31.0
rich==13.7.0
safetensors==0.4.2
sentencepiece==0.2.0
shiboken6==6.6.1
six==1.16.0
sounddevice==0.4.6
soundfile==0.12.1
sympy==1.12
tiktoken==0.6.0
tokenizers==0.15.2
torch @ https://download.pytorch.org/whl/cu121/torch-2.2.0%2Bcu121-cp311-cp311-win_amd64.whl#sha256=d79324159c622243429ec214a86b8613c1d7d46fc4821374d324800f1df6ade1
tqdm==4.66.2
transformers==4.37.2
typing_extensions==4.10.0
tzdata==2024.1
urllib3==2.2.1
whisper_s2t @ git+https://github.com/shashikg/WhisperS2T.git@33e305fd447004c18fbc73848fa1e3385f42c93c
xxhash==3.4.1
yarl==1.9.4