Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

AI Model uform-gen2-qwen-500m returns an error when given a temperature parameter #16646

Open
brettimus opened this issue Sep 5, 2024 · 0 comments
Assignees
Labels
content:edit Request for content edits documentation Documentation edits product:workers-ai Workers AI: https://developers.cloudflare.com/workers-ai/

Comments

@brettimus
Copy link
Contributor

Existing documentation URL(s)

What changes are you suggesting?

The input JSON schema for model @cf/unum/uform-gen2-qwen-500m lists temperature as a possible input, but when I add the temperature, I get an error from the AI service:

{
  "name":"InferenceUpstreamError",
  "message":"Model input is not valid: input tensor `temperature` is not present in the model"
}

Change I am suggesting is to modify the JSON schema. (Realizing this might be involved, since I'm guessing this schema is shared across models where some do actually support the temperature parameter, and the corresponding input types may need to be updated on the typescript side of things.)

Additional information

My payload (without the image buffer) is:

{
"image": [],
"prompt":"Describe the image in detail, ..."
"max_tokens":100
"temperature":0.3
"top_p":0.9
"top_k":50
"seed":42
"repetition_penalty":1.2
"frequency_penalty":0.5
"presence_penalty":0.3
"raw":false
}
@brettimus brettimus added content:edit Request for content edits documentation Documentation edits labels Sep 5, 2024
@github-actions github-actions bot added the product:workers-ai Workers AI: https://developers.cloudflare.com/workers-ai/ label Sep 5, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
content:edit Request for content edits documentation Documentation edits product:workers-ai Workers AI: https://developers.cloudflare.com/workers-ai/
Projects
None yet
Development

No branches or pull requests

7 participants