Qwen2.5 7B Instruct の必要メモリ
量子化別の必要メモリ 推定
| 量子化 | 4K文脈 | 32K | 128K |
|---|---|---|---|
| Q4_K_M | 6.1 GB | 7.7 GB | 13.4 GB |
| Q5_K_M | 6.9 GB | 8.6 GB | 14.2 GB |
| Q8_0 | 9.7 GB | 11.4 GB | 17.0 GB |
| FP16 | 17.2 GB | 18.9 GB | 24.5 GB |
| 量子化 | 4K文脈 | 32K | 128K |
|---|---|---|---|
| Q4_K_M | 6.1 GB | 7.7 GB | 13.4 GB |
| Q5_K_M | 6.9 GB | 8.6 GB | 14.2 GB |
| Q8_0 | 9.7 GB | 11.4 GB | 17.0 GB |
| FP16 | 17.2 GB | 18.9 GB | 24.5 GB |