Hardware & Models
Overview of all available hardware components and models in the system.
Qwen/Qwen2.5-Coder-32B-Instruct-GGUF
32BType: LLM
Qwen 2.5 Coder is a code-specialized LLM from Alibaba Cloud
Available Quantizations:
Q2_KQ3_K_MQ4_0Q4_K_MQ5_0
bartowski/DeepSeek-R1-Distill-Qwen-32B-GGUF
32BType: LLM
DeepSeek R1 Distill Qwen is a distilled version of DeepSeek R1 based on Qwen architecture
Available Quantizations:
IQ2_XXSIQ2_XSIQ2_SQ2_KQ2_K_LQ3_K_SQ3_K_MQ3_K_LQ3_K_XLIQ4_XSQ4_K_SIQ4_NLQ4_0Q4_1Q4_K_MQ4_K_LQ5_K_SQ5_K_MQ5_K_LQ6_KQ6_K_LQ8_0BF16
unsloth/DeepSeek-R1-Distill-Llama-70B-GGUF
70BType: LLM
DeepSeek R1 Distill Llama is a distilled version of DeepSeek R1 based on Llama architecture
Available Quantizations:
Q2_KQ2_K_LQ3_K_MQ4_K_MQ5_K_MQ6_KQ8_0FP16