Qwen2.5 72B Instruct の必要メモリ
量子化別の必要メモリ 推定
| 量子化 | 4K文脈 | 32K | 128K |
|---|---|---|---|
| Q4_K_M | 48.6 GB | 58.0 GB | 90.2 GB |
| Q5_K_M | 56.6 GB | 66.0 GB | 98.2 GB |
| Q8_0 | 83.4 GB | 92.8 GB | 125.1 GB |
| FP16 | 155.0 GB | 164.4 GB | 196.6 GB |
| 量子化 | 4K文脈 | 32K | 128K |
|---|---|---|---|
| Q4_K_M | 48.6 GB | 58.0 GB | 90.2 GB |
| Q5_K_M | 56.6 GB | 66.0 GB | 98.2 GB |
| Q8_0 | 83.4 GB | 92.8 GB | 125.1 GB |
| FP16 | 155.0 GB | 164.4 GB | 196.6 GB |