friendli model convert-adapter
Usage
friendli model convert-adapter [OPTIONS]
Summary
Convert huggingface's adapter checkpoint to Friendli format.
When an adapter checkpoint is in the Hugging Face PEFT format, it cannot be directly served in Friendli. It requires conversion to the Friendli format. The conversion process involves copying the original adapter checkpoint and transforming it into a checkpoint in the Friendli format (*.h5).
This function does not include the friendli checkpoint convert
command. i.e.
friendli checkpoint convert-adapter
only converts adapter's parameters, not backbone's.
caution
The friendli checkpoint convert-adapter
is available only when the package is installed with
pip install "friendli-client[mllib]"
.
Options
Option | Type | Summary | Default | Required |
---|---|---|---|---|
--adapter-name-or-path , -a | TEXT | Hugging Face pretrained adapter name or path to the saved adapter checkpoint. | - | ✅ |
--output-dir , -o | TEXT | Directory path to save the converted adapter checkpoint and related configuration files. Two files will be created in the directory: adapter.h5 , and attr.yaml . The adapter.h5 is the converted checkpoint and can be renamed using the --output-adapter-filename option. The attr.yaml is the adapter checkpoint attribute file, to be used when uploading the converted model to Friendli. You can designate the file name using the --output-attr-filename option. | - | ✅ |
--data-type , -dt | CHOICE: [bf16, fp16, fp32, int8, int4] | The data type of converted checkpoint. | - | ✅ |
--base-model-name-or-path , -b | TEXT | Hugging Face model name or path to the saved backbone checkpoint. By default, we use the base_model_name_or_path in adapter_config.json. | None | ❌ |
--cache-dir | TEXT | Directory for downloading checkpoint. | None | ❌ |
--dry-run | BOOLEAN | Only check conversion avaliability. | False | ❌ |
--output-adapter-filename | TEXT | Name of the converted adapter checkpoint file. | adapter.h5 | ❌ |
--output-attr-filename | TEXT | Name of the adapter checkpoint attribute file. | adapter_attr.yaml | ❌ |