Code

Code LLM Leaderboard

Embrace federated LLM fine-tuning on coding tasks and secure your spot on the leaderboard!

In the domain of Code Generation, developing models that can write, complete, and debug code across multiple programming languages is increasingly valuable.

Federated LLM fine-tuning on coding tasks allows organizations to improve code models collaboratively without exposing proprietary codebases.

This approach ensures fine-tuned models are robust across diverse coding benchmarks and can generalize to real-world software development scenarios.

RankTeamCodeBase ModelComm. CostsAverage (↑)MBPPHumanEvalMultiPL-E (JS)MultiPL-E (C++)Date
1ZeroOne.AIQwen3-8B3.5 GB65.2756.2070.7368.3265.8414.01.26
2Massimo R. ScamarciaQwen3-4B5.3 GB60.4554.8064.6361.4960.8710.05.25
3CAR@AIMLdeepseek-coder-7b-instruct-v1.52.9 GB58.7756.8064.6355.9057.7625.04.25
4FL-finetune-JB-DCQwen2.5-Coder-7B-Instruct1.5 GB56.0864.0027.4372.6760.2420.04.25
5FL-finetune-JB-DCQwen2.5-Coder-7B-Instruct1.5 GB54.5564.0023.1770.1860.8620.04.25
6Massimo R. ScamarciaPhi-4-mini-instruct3.6 GB49.0046.2059.7652.7937.2714.03.25
7CAR@AIMLstarcoder2-7b35.6 GB44.0848.6045.7338.5043.4825.04.25
8Massimo R. ScamarciaQwen2.5-7B-Instruct2.0 GB34.4048.0025.0010.5654.0420.03.25
9Massimo R. ScamarciaQwen2.5-Coder-0.5B-Instruct8.7 GB34.3425.6037.8141.0032.9227.01.25
10CAR@AIMLCodeLlama-7b-hf25.0 GB33.7838.2631.7133.5431.6825.04.25
11BaselineLlama-3.2-3B27.4 GB28.3333.8031.7124.8422.9809.12.24
12BaselineMistral-7B-v0.340.7 GB27.3631.6023.7828.5725.4701.10.24
13Massimo R. ScamarciaSmolLM2-1.7B-Instruct16.7 GB26.9335.0030.4918.6323.6020.03.25
14Massimo R. ScamarciaQwen2.5-1.5B-Instruct3.7 GB17.8223.607.9313.6626.0920.03.25
15Massimo R. ScamarciaQwen2.5-0.5B-Instruct4.0 GB16.4920.4022.566.8316.1520.03.25
16T-IoI@UNRSmolLM2-360M-Instruct2.3 GB14.6518.6017.6812.429.9314.04.25
17T-IoI@UNRSmolLM2-360M2.3 GB12.9217.2014.0211.808.6914.04.25
18CAR@AIMLTinyLlama-1.1B-Chat-v1.03.4 GB8.672.6012.209.949.9425.04.25
19T-IoI@UNRSmolLM2-135M-Instruct1.4 GB5.807.205.485.594.9614.04.25
20T-IoI@UNRSmolLM2-135M1.4 GB4.512.603.046.216.2114.04.25