Skip to main content

Introducing Friendli Containers

While Friendli Serverless Endpoints and Dedicated Endpoints offer convenient cloud-based solutions, some users crave even more control and flexibility. For those pioneers, Friendli Containers are the answer.

What are Friendli Containers?​

Unmatched Control: Friendli Containers provide the Friendli Engine, our cutting-edge serving technology, as a Docker container. This means you can:

  • Run your own data center or cluster: Deploy the container on your existing GPU machines, giving you complete control over your infrastructure and data security.
  • Choose your own cloud provider: If you prefer the cloud, you can still leverage your preferred cloud provider and GPUs.
  • Customize your environment: Fine-tune the container configuration to perfectly match your specific needs and workflows.

Greater Responsibility, Greater Customization: With Friendli Containers, you handle the cluster management, fault tolerance, and scaling. This responsibility comes with these potential benefits:

  • Controlled environment: Keep your data within your own environment, ideal for sensitive applications or meeting compliance requirements.
  • Unmatched flexibility: Tailor your infrastructure and workflows to your specific needs, pushing the boundaries of AI innovation.
  • Cost saving opportunities: Manage your resources on your GPU machines, potentially leading to cost savings compared to cloud-based solutions.

Ideal for:

  • Data-sensitive users: Securely run your models within your own infrastructure.
  • Control enthusiasts: Take full control over your AI environment and workflows.
  • Existing cluster owners: Utilize your existing GPU resources for cost-effective generative AI serving.

Getting Started with Friendli Containers:​

  1. Generate Your User Token: Visit the Friendli Container page through the Friendli Suite website and generate your unique token.
  2. Login with Docker Client: Use your token to authenticate with the Docker client on your machine.
  3. Pull the Friendli Container Image: Run the docker pull command with the provided image name.
  4. Launch the Friendli Container: Run the docker run command with the desired configuration and credentials.
  5. Expose Your Model: The container will expose the model for inference.
  6. Send Inference Requests: Use tools like curl or Python's requests library to send input prompts or data to the container.

Take generative AI to the next level with unmatched control, security, and flexibility through Friendli Containers. Start your journey today and elevate your AI endeavors on your own terms!