Repository · GitHub

IST-DASLab/marlin

FP16xINT4 LLM inference kernel that can achieve near-ideal ~4x speedups up to medium batchsizes of 16-32 tokens.

Owner	IST-DASLab
License (as declared)	Apache-2.0
Stars	1,162
Forks	95
Main language	Python
Created	17 January 2024
Last updated at source	4 September 2024

python4bitkernelllmquantization

Embed this entry

<iframe src="https://silytics.si/embed/item/6112fda5-8510-4f0e-a8c0-acea39513573" width="420" height="190" style="border:0" loading="lazy" title="Silytics entry"></iframe>

Free, no tracking. More about feeds and embeds.

Silytics shows public metadata from GitHub and links to the original. Information may be incomplete or out of date. Last refreshed 15 October 2026. Licenses and terms of the original apply to the work itself. Not financial, legal or professional advice.