| --- |
| library_name: peft |
| tags: |
| - code |
| - instruct |
| - code-llama |
| datasets: |
| - ehartford/dolphin-2.5-mixtral-8x7b |
| base_model: codellama/CodeLlama-7b-hf |
| license: apache-2.0 |
| --- |
| |
| ### Finetuning Overview: |
|
|
| **Model Used:** codellama/CodeLlama-7b-hf |
|
|
| **Dataset:** ehartford/dolphin-2.5-mixtral-8x7b |
|
|
| #### Dataset Insights: |
|
|
| [No Robots](https://huggingface.co/datasets/HuggingFaceH4/no_robots) is a high-quality dataset of 10,000 instructions and demonstrations created by skilled human annotators. This data can be used for supervised fine-tuning (SFT) to make language models follow instructions better. |
|
|
| #### Finetuning Details: |
|
|
| With the utilization of [MonsterAPI](https://monsterapi.ai)'s [no-code LLM finetuner](https://monsterapi.ai/finetuning), this finetuning: |
|
|
| - Was achieved with great cost-effectiveness. |
| - Completed in a total duration of 1h 15m 3s for 2 epochs using an A6000 48GB GPU. |
| - Costed `$2.525` for the entire 2 epochs. |
|
|
| #### Hyperparameters & Additional Details: |
|
|
| - **Epochs:** 2 |
| - **Cost Per Epoch:** $1.26 |
| - **Total Finetuning Cost:** $2.525 |
| - **Model Path:** codellama/CodeLlama-7b-hf |
| - **Learning Rate:** 0.0002 |
| - **Data Split:** 100% train |
| - **Gradient Accumulation Steps:** 64 |
| - **lora r:** 64 |
| - **lora alpha:** 16 |
|
|
| --- |
| license: apache-2.0 |
