LLM Fine-Tuning

• Tech Stack: Python, Transformers, PEFT, BitsAndBytes, HuggingFace, Weights & Biases (wandb)
• Github URL: Project Link

Fine-tuned a 4-bit quantized pre-trained LLM (Llama 3.2) to summarize conversational text efficiently using LoRA (Low-Rank Adaptation) for parameter-efficient training.

Leveraged Weights and Biases (WandB) for real-time monitoring of training metrics and hyperparameter optimization. Evaluated the model’s summarization performance using the ROUGE score.