[FEEDBACK] Inference Providers
Any inference provider you love, and that you'd like to be able to access directly from the Hub?
Love that I can call DeepSeek R1 directly from the Hub ๐ฅ
from huggingface_hub import InferenceClient
client = InferenceClient(
provider="together",
api_key="xxxxxxxxxxxxxxxxxxxxxxxx"
)
messages = [
{
"role": "user",
"content": "What is the capital of France?"
}
]
completion = client.chat.completions.create(
model="deepseek-ai/DeepSeek-R1",
messages=messages,
max_tokens=500
)
print(completion.choices[0].message)
Is it possible to set a monthly payment budget or rate limits for all the external providers? I don't see such options in billings tab. In case a key is or session token is stolen, it can be quite dangerous to my thin wallet:(
@benhaotang you already get spending notifications when crossing important thresholds ($10, $100, $1,000) but we'll add spending limits in the future
@benhaotang you already get spending notifications when crossing important thresholds ($10, $100, $1,000) but we'll add spending limits in the future
Thanks for your quick reply, good to know!
Would be great if you could add Nebius AI Studio to the list :) New inference provider on the market, with the absolute cheapest prices and the highest rate limits...
Could be good to add featherless.ai
TitanML !!
@OFT so we can take a look can you share (here or in DM) which model(s) you're mostly hitting?
And i'm assuming it's on
provider=hf-inference
?
@julien-c Thank you for having a look into this.
I am mostly using models that are:
- IMG 2 TXT
- TXT 2 IMG (Flux, SD 3.5, own models, ....)
- TXT 2 TXT
I don't know where I can verify provider=hf-inference
. Could you give me a CURL example with this option? (not phyton please)
However I do have the following billing information:
Mar 1 - Mar 31 | HF Inference API (4,585) | $2.02
Feb 1 - Feb 28 | HF Inference API (17,156) | $1.72
It is unclear what happend between the two months since I don't think I changed something?
Extra: After noticing the problem, I also jumped to "https://huggingface.co/settings/inference-providers" and disabled everything except the HF Interference API.
Please add kluster.ai!!
@julien-c please give us prices per request for each model using hf-inference. The question is because the prices for different models vary widely, and I want to know what models are expensive to manage my bills. At least detail view in billing page.
I had cancelled my OpenAI subscription in favor of the HF Pro subscription. Now I realize you took the inference away. I will be cancelling my Pro subscription because frankly I don't understand who asked for this. If I want to use other providers I will use other providers, why would I route it through HF? There is no value proposition whatsoever. Anyone doing production loads will get their own provider anyways.
I would have accepted a price increase or a decreased request limit in the Pro plan over this bad business move. I don't see a point in the Pro subscription anymore, why do you keep it?
Many users are complaining because of prices, let's add these two and enable real free open source for everyone
can i add more money beside the 2$ limitation per month?
since the pay as you go dosent work yet
what are my options?