Qwen2.5-1.5B-Instruct

Name: Qwen2.5-1.5B-Instruct
Rating: 5

Generate text responses to prompts, enabling natural language conversations
and content creation

Model Properties

The pipeline consists of a prefill step and a token-by-token step

License name: Apache License 2.0
Number of parameters: 1.5B
Model Size: 1.64 GB

Select device..

Hailo-10H

Operations: 29.4 GOPs per input token

Context Length: 2048

Numerical Scheme: A8W4, symmetric, group-wise

Inference Api: C++, Python, Hailo-Ollama

Compiled Model:

First Load Time In Sec 5.05

Time To First Token In Sec 0.37

TPS 6.82

GenAI Models

Generate text responses to prompts, enabling natural language understanding, multilingual support, and code generation

GenAI Models

Generate text responses to prompts, enabling natural language understanding, multilingual support, content creation and advanced reasoning