Tool Assisted API (Beta)
What is Tool Assisted API?
Tool Assisted API enhances a model’s capabilities by integrating tools that extend its functionality beyond simple conversational interactions. By using this API, the model becomes more dynamic, providing more comprehensive and actionable responses.
Currently, Friendli Serverless Endpoints supports a variety of built-in tools specifically designed for Chat Completion tasks.
What is Chat Completion?
Chat completion refers to a model’s ability to generate responses in a conversation. Given a sequence of messages or conversation turns, the model processes the input and generates a response based on its internal knowledge and training data.
- Example:
- User: “What is the capital of France?”
- Model: “The capital of France is Paris.”
However, chat completion has its limitations—it is restricted to the knowledge the model has learned during its training and cannot access real-time or external data.
Is Chat Completion Different from Tool Assisted Chat Completion?
Yes, Tool Assisted Chat Completion goes beyond basic chat completion by integrating external tools to enhance the conversation. This allows the model to access real-time data, perform specific tasks, and interact with external systems in ways that chat completion alone cannot achieve.
- Example:
- User: “What is the weather today?”
- Model without Tool Access: Relies on pre-learned information, potentially giving outdated or generalized answers.
- Model with Tool Access: Calls a weather API to retrieve live data and responds: “The weather today in New York is 72°F with clear skies.”
With tool access, the model provides a more accurate and up-to-date response.
Additionally, some tasks—such as file processing or complex calculations—cannot be performed by the model alone but can be handled with the help of tools.
- Example:
- User: “Can you extract the text from this document?” (provides a file)
- Model without Tool Access: “I cannot extract data from files directly.”
- Model with Tool Access: Extracts the text from the provided file and responds: “Using the
file:texttool, I’ve extracted the following text: [Text from the file].”
When no tools are specified, the model will respond using only its internal knowledge.
Benefits of Tool Assisted Chat Completion
Tool Assisted Chat Completion offers several advantages over basic chat completion:
- Real-Time Data Access: The model can pull live information.
- Extended Capabilities: The model can perform complex tasks like running calculations, executing code, extracting text from files, and interacting with databases and APIs.
Comparison: Chat Completion vs. Tool Assisted Chat Completion
| Feature | Chat Completion | Tool Assisted Chat Completion |
|---|---|---|
| Response Type | Based on internal knowledge | Uses external tools for enhanced, real-time responses |
| Capabilities | Limited to pre-learned knowledge | Can interact with tools for data retrieval and task execution |
| Example | “What is the weather today?” (general knowledge) | “What is the weather today?” (live API result) |
| Use Cases | General conversation and Q&A | Complex tasks like real-time updates, data analysis, file processing |
Conclusion
- Chat Completion: Best for general conversations that rely on the model’s pre-existing knowledge.
- Tool Assisted Chat Completion: Ideal for real-time, dynamic tasks and more advanced interactions, leveraging external tools to enhance functionality.
Explore APIs
To get started with Tool Assisted Chat Completion, follow this tutorial: Tool calling with Serverless Endpoints.
For more details, check out the API Reference documentations below:
Was this page helpful?