Skip to main content

Fine-Tuning

Effortlessly fine-tune your model with Friendli Dedicated Endpoints, which leverages the Parameter-Efficient Fine-Tuning (PEFT) method to reduce training costs while preserving model quality, similar to full-parameter fine-tuning.

Hugging Face Integration

You need to integrate your Hugging Face account to access your private repo or gated repo. Go to User settings > Account > Hugging Face integration and save your Hugging Face access token. This access token will be used whenever creating fine-tuning jobs.

huggingface integration

Creating Fine-Tuning Job

To create a new fine-tuning job, follow these steps:

  1. Go to your project and click Fine-tuning tab.
  2. Click New job.
  3. Fill out the job configuration based on the following field descriptions:
    • Job name: The name of fine-tuning job to create.
    • Hugging Face integration: Your Hugging Face access token. Set up Hugging Face access token as described at Hugging Face Integration.
    • Model: Hugging Face Models repository name to fine-tune.
    • Dataset: The fine-tuning dataset at Hugging Face Datasets repository. Read Dataset Format to see the available dataset format.
    • Weights & Biases (W&B): Option for W&B integration.
      • W&B API key: Your W&B API key.
      • W&B project: Your W&B project name.
    • Hyperparameters: Fine-tuning Hyperparameters.
      • learning_rate: The initial learning rate for AdamW optimizer.
      • batch_size: Total training batch size.
      • Total number of training: Configure the number of training cycles with either num_train_epochs or max_steps.
        • num_train_epochs: The number of training epochs.
        • max_steps: The number of training steps.
      • max_seq_length: The maximum sequence length.
      • lora_r: The rank of LoRA.
      • *`lora_alpha`*: Scaling factor that determines the influence of the low-rank matrices during fine-tuning.
      • lora_dropout: Dropout rate applied during fine-tuning.
  4. Click Create button to create a job with the configuration.

Dataset Format

The dataset used for fine-tuning should satisfy the following conditions:

  1. The dataset must contain a column named “messages”, which will be used for fine-tuning.
  2. Each row in "messages" column should be compatible with the chat template of the base model. For example, tokenizer_config.json of mistralai/Mistral-7B-Instruct-v0.2 repeats messages of user and assistant. In this case, each row in "messages" field should be like [{"role": "user", "content": "The 1st user's message"}, {"role": "assistant", "content": "The 1st assistant's message"}]. HuggingFaceH4/ultrachat_200k is the dataset that is compatible with this case.

Monitoring Progress and Results

After launching the fine-tuning job, you can monitor the job overview, including progress information and fine-tuning configuration. If you provided your W&B API key and project name when creating the job, you can also monitor the training status in your W&B project.

Deploy Fine-Tuned Model

Once the fine-tuning job is complete, you can deploy the fine-tuned model immediately.