granite-4.0-h-small

We've been covering IBM and their Granite LLM series for a while. With this series, IBM finally scaled up the model size as well, bringing a series of hybrid (attention + mamba) models, ranging from a 3B dense to a 32B-A9B MoE. We used the models and were impressed, although not surprised, by the quality, given the continued persistence of IBM's team to release better and better models.

Granite, for at least the 3B variant, is roughly in the SmolLM3 quality range, being only surpassed by Qwen3 4B in terms of multilingual and instruction following capabilities. The tone of Granite 4.0 is refreshingly non-exciting compared to the sloptimized models recently (i.e. the trend across the industry for playful, emoji-filled, and often sycophantic models), making it feel like old Mistral models in a good way. Interestingly enough, they are also following Qwens lead and will release a separate reasoning model later in the year. We've heard many reports from people training models that hybrid reasoning - i.e. a toggle of thinking tokens on and off - adds a major complexity cost in training that lowers the peak performance of both modes. IBM debuted the hybrid thinking approach (togglable via prompts) very early on for open models, which was adopted by others later.