20GB @ 100,000 context.
But for some reason... LM studio isnt loading it onto gpu for me?
I just updated to 0.3.28 and still wont load onto gpu.
Switching from Vulkan to rocm. It's now working properly?
https://docs.unsloth.ai/new/ibm-granite-4.0
Fantastic work from unsloth folks as usual.
As it's running in roo code, it's using more like 26GB of vram.
~30TPS
Roo code does not work with it.
Kilo code next. It seems to be about 22GB of vram.
Kilo code works great.
The model however didn't 1 shot my first benchmark. That's pretty bad news for this model given magistral 2509 or apriel 15b are better.
Better on pass 2, still no 100%
3rd pass achieved.
Im predicting it'll be around 30% on livecodebench. Probably like 15% on aiderpolyglot. Very disappointed in its coding capability.
I just found:
https://artificialanalysis.ai/models/granite-4-0-h-small
25.1% on livecodebench. Absolutely deserved.
2% terminal bench.
16% on coding index. Completely deserved.
20GB @ 100,000 context.
But for some reason... LM studio isnt loading it onto gpu for me?
I just updated to 0.3.28 and still wont load onto gpu.
Switching from Vulkan to rocm. It's now working properly?
https://docs.unsloth.ai/new/ibm-granite-4.0
Fantastic work from unsloth folks as usual.
As it's running in roo code, it's using more like 26GB of vram.
~30TPS
Roo code does not work with it.
Kilo code next. It seems to be about 22GB of vram.
Kilo code works great.
The model however didn't 1 shot my first benchmark. That's pretty bad news for this model given magistral 2509 or apriel 15b are better.
Better on pass 2, still no 100%
3rd pass achieved.
Im predicting it'll be around 30% on livecodebench. Probably like 15% on aiderpolyglot. Very disappointed in its coding capability.
I just found:
https://artificialanalysis.ai/models/granite-4-0-h-small
25.1% on livecodebench. Absolutely deserved.
2% terminal bench.
16% on coding index. Completely deserved.