Models for Local GPU
Models for Local GPU
| VRAM Size | Category | Models |
|---|---|---|
| 4 GB | Text | llama3.2 3b |
| Vision | moondream, llava-phi3 | |
| Speech | Whisper - tiny | |
| 6 GB | Text | llama3.1 8b, mistral7b, ministral8b, qwen2.5 |
| Vision | llava, minicpm-v | |
| Speech | Whisper - medium | |
| 12 GB | Text | mistral-nemo |
| Vision | llava | |
| 16 GB | Text | |
| Vision | llama3.2 vision |