Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Streaming] OpenAI token usage counts are missing for streamed responses #405

Closed
aaronvg opened this issue Feb 8, 2024 · 0 comments
Closed

[Streaming] OpenAI token usage counts are missing for streamed responses #405

aaronvg opened this issue Feb 8, 2024 · 0 comments

Comments

@aaronvg
Copy link
Contributor

aaronvg commented Feb 8, 2024

This is a limitation of their API. They do not currently expose usage during streamed responses.

We could try to approximate it, but if we do we should probably tag these counts as approximated somewhere.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
2 participants
@hellovai @aaronvg