DeepSeek-R1-Distill-Qwen-14B の必要メモリ
量子化別の必要メモリ 推定
| 量子化 | 4K文脈 | 32K | 128K |
|---|---|---|---|
| Q4_K_M | 11.2 GB | 16.9 GB | 36.2 GB |
| Q5_K_M | 12.9 GB | 18.5 GB | 37.8 GB |
| Q8_0 | 18.3 GB | 24.0 GB | 43.3 GB |
| FP16 | 32.9 GB | 38.5 GB | 57.8 GB |
| 量子化 | 4K文脈 | 32K | 128K |
|---|---|---|---|
| Q4_K_M | 11.2 GB | 16.9 GB | 36.2 GB |
| Q5_K_M | 12.9 GB | 18.5 GB | 37.8 GB |
| Q8_0 | 18.3 GB | 24.0 GB | 43.3 GB |
| FP16 | 32.9 GB | 38.5 GB | 57.8 GB |