[FEATURE] Community Tools

#569
by nsarrazin HF staff - opened
Hugging Chat org
โ€ข
edited 28 days ago

thumbnail.png

Community Tools on HuggingChat!

Today we're releasing Community Tools on HuggingChat. This feature lets you create custom tools using Hugging Face Spaces! With this feature we're also making it possible for tools to use new modalities, like video, speech and more! You can find community tools on the following page: https://huggingface.co/chat/tools

In order to use tools in your conversations you will need to use a model compatible with tool calling. Currently we support the following models:

  • meta-llama/Meta-Llama-3.1-70B-Instruct
  • CohereForAI/c4ai-command-r-plus-08-2024

It's now also possible to add Community Tools to your assistants! When you create an assistant with a compatible model, you will see a search bar for tools where you can add your own tools. This lets you group tools together with a system prompt to create more complete experiences using assistants.

We've also created a blog post to explain how to create tools, either turning existing Spaces into tools or creating your own custom Spaces. You can find it here ๐Ÿ“!

The feature is still new and we're excited to hear your feedback. Feel free to share your tools in this thread, we will review them and possibly feature them on the community page!

nsarrazin pinned discussion

Thank you, Huggingface and all huggingchat team for this amazing feature.
Here are some tools I created:

  1. Chat with Image and video (uses qwen2 vl 7b)
    https://hf.co/chat/tools/66e85bb396d054c5771bc6cb

(video is not working in Llama 3.1 70b IDK why)

  1. Critical Thinker
    https://hf.co/chat/tools/66e864a034d40bac65628668

  2. Flux Dev Fast
    https://hf.co/chat/tools/66e9b279bbf94ad91c808f68

Dunno how to use
for example, I make dice roll active. what do i do to call it into the chat?

Dunno how to use
for example, I make dice roll active. what do i do to call it into the chat?

You can say model to use the specific ACTIVE TOOL and it will use it. Alternatively, during regular conversations, it will auto use them according to need.

What's the reason that the tool selector states there are 83 available but when you browse them there are only 28 to choose from?

Hugging Chat org

@ChrisUnimportant counter was showing the total number of tools but we only show featured ones, will fix ๐Ÿ˜…

Better Doc reader: https://hf.co/chat/tools/66ed8236a35891a61e2bfcf2

This tool can read all types of file and contents.
From code files to Excel, Pdf to PPT, Docx to Csv. and Many more...

Hugging Chat org

@KingNish Nice, Just tried it and it's super fast! Featured it as well :)

Looks like your link is broken: it should be https://huggingface.co/chat/tools instead of https://huggingface.co/chat/tool.

Hugging Chat org

Oof good catch @EveryPizza , fixed!

I've activated a tool, but can't figure out how to use it? The chat interface doesn't seem to know how to use it. It just gives a normal response as if it didn't ask to use the tool. Tried the image generation (flux) one, and the Voice cloner Tool ( in specs it mentions /predict and create_voice function name). It doesn't work to put those into the chat.

I've activated a tool, but can't figure out how to use it? The chat interface doesn't seem to know how to use it. It just gives a normal response as if it didn't ask to use the tool. Tried the image generation (flux) one, and the Voice cloner Tool ( in specs it mentions /predict and create_voice function name). It doesn't work to put those into the chat.

Hi fellow Scott!

coincidently, I asked this same question, heres the answer:

Dunno how to use
for example, I make dice roll active. what do i do to call it into the chat?

You can say model to use the specific ACTIVE TOOL and it will use it. Alternatively, during regular conversations, it will auto use them according to need.

https://hf.co/chat/tools/66f1a8159d41ad4398ebb711
This tool helps you extract any content (like: images, videos, pdfs, docs , etc.)

Example Chat:
https://hf.co/chat/r/0zCEqI_

https://hf.co/chat/tools/66f1a8159d41ad4398ebb711
This tool helps you extract any content (like: images, videos, pdfs, docs , etc.)

Another update Now it also can now also extract normal webpages and also resolved issue in link fetching and image fetching in default Fetch URL.
@nsarrazin can you please also feature it.

Is it support multiple function?, my Space has 2 function and I want to add all my function to my tool.

Hugging Chat org

@eienmojiki you will have to create two tools for this, but should not be an issue otherwise! you can combine them in an assistant if you want afterwards

I've tried configuring a tool from one of our spaces, but when I "save" it in the end it just briefly shows "Saving..." and stays on the same configuration page. The tool doesn't appear in the list of available tools.

Hugging Chat org

Could you share the space in question ? @jgrivolla

Could you share the space in question ? @jgrivolla

I tried with this space: BSC-LT/EADOP_RAG_EXPERIMENTAL

It's a simple RAG system, and the idea was to use just the retrieval portion (the idea is to then slim it down to just the required functionality for actual use, but for testing I used the existing space). Input is input_, output is markdown-2 (the retrieval result / context), all parameters fixed to their default value. Other than those configurations I just added the tool description and the description of the input_ argument.

Document parser fails at times.

Document parser fails at times.

https://hf.co/chat/tools/66ed8236a35891a61e2bfcf2

Try this.

Roughly when will agents be a thing?

I can't save the tool.

Hugging Chat org

could you share which space you tried to use ? @Taf2023

Image edition says "error: no tool found", and for image generation, the chat says "can't generate an image" (it doesn't try to load the extension at all) so both of these tools does not work at all

Please add tools support to assistants which uses the model supporting tools at least.
Those who like this idea ๐Ÿ’–this comment

Hugging Chat org

That should already be supported! Try creating an assistant with command R+ or llama 3.1 70B and you should be able to select tools to add to your assistant. @KarthiDreamr

That should already be supported! Try creating an assistant with command R+ or llama 3.1 70B and you should be able to select tools to add to your assistant. @KarthiDreamr

I don't see any text to select tools.

Hugging Chat org

Whoops guys, feel free to try again, the feature should be there now ๐Ÿ˜…

Whoops guys, feel free to try again, the feature should be there now ๐Ÿ˜…

very good

Taf2023/HTML-Generator
SyntaxError: Unexpected end of JSON input

Whoops guys, feel free to try again, the feature should be there now ๐Ÿ˜…

@nsarrazin there is no option to upload files or images in the assistant. Even when it's using tool which supports file upload.

Love the idea of community tools, BUT I would love if the huggingface team itself would help curate and maintain them. I know the idea of curation is bad to many people, but from a user perspective, having many choices to do the same thing worded slightly different is a nightmare, no way around it.

It would be great to consolidate efforts towards improving a tool, instead of spreading it out. Then a single person (hopefully a huggingface staff member, but you can pick anyone where there's a clear path of responsibility), would curate and help integrate efforts on git (basically a git maintainer).

As for the growing amount of ways to generate images, there's an interesting nuance there that I would like to address, with this as an example.
Since each image generator might have something unique to it (different backend, lora, prompt technique unique enough, wathever), it would make sense to throw them all under one category and let the user pick whatever they want if the default is not good enough.

But when it comes to say for example rag applications, there are so many commonalities that must work across all of them, I don't see a reason to overwhelm the user so much with choices of slightly different rag implementations. The data has to be ingested correctly, the data must be searched and preferably reranked correctly, and then being smart about the insertion as well. The commonality across all rag applications of this makes it general enough that I don't see the point of having 6 rag tools, just one which does things really well and works. This way you also don't polute the community store with noise, even if the applications themselves might be good, it does not really give the impression of quality (user confidence)

Then, if you want a rag tool which uses something like a vllm instead to parse things that might be unique enough to add in the "unix' philosophy kind of way, one tool, one function, one output, but that is now in the debating territory, I hope the point is clear.

Kind regards

Hope this gets added to the recent Nvidia model, this feature seems pretty cool.

On the topic of that, it should be noted on the frontend that this is currently limited to those two models. I just had a bit of confusion (as I'd imagine other people would have) as to why the model wasn't interacting or acknowledging any of the tools I activated, right up until I went to this discussion page where it mentions that limitation.

I've created a Space using the default free CPU basic hardware set to 'public', yet when trying to create a tool using it, there's an error saying:
SyntaxError: JSON.parse: unexpected end of data at line 1 column 1 of the JSON data

Is there any specific reason why my Space can't be used as a tool?

Hugging Chat org

Could you share the tool? I could take a look @testnow720

@nsarrazin there is no option to upload files or images in the assistant. Even when it's using tool which supports file upload.

Hugging Chat org

Could you share an assistant that has this issue @KingNish ?

Could you share an assistant that has this issue @KingNish ?

https://hf.co/chat/assistant/6710d67d57b98c857f7259db

Hugging Chat org

Thanks, i can reproduce the issue, let me take a look

Could you share the tool? I could take a look @testnow720

The Space is up at : testnow720/hfchat_code_execution

(I couldn't create a tool that uses the Space because of said error)

Hugging Chat org

@KingNish the issue should be fixed!

Sign up or log in to comment