Work xFinance - BloombergGPT Challenger

Summary

Replicated BloombergGPT’s financial capabilities using open-source LLaMA 13B and XTuring with a $1,000 budget, outperforming BloombergGPT on 3 key benchmarks.

My Role

Project Lead

Challenges

• Building high-quality financial dataset with public sources
• Fine-tuning on low-budget infrastructure
• Evaluating performance across nuanced financial tasks

What We Did

• Built unsupervised and instruction datasets from 2022–2023 data
• Used XTuring to fine-tune LLaMA 13B with LoRA on 8xA100s
• Outperformed BloombergGPT on FPB, Headline, and FICA-SA tasks

Outcomes

• Model trained in 2 weeks for <$1,000
• Achieved higher F1 scores than BloombergGPT on open tasks
• Demonstrated feasibility of open-source finance-specific LLMs