hkust-nlp/LeetCode-O

This is the LeetCode-O benchmark proposed in CodeI/O paper (Arxiv 2502.07316). The data file is in leetcode.jsonl and we provide an example prediction file (gpt-4.1-nano) in prediction.jsonl. To evaluate your model on this benchmark, please prepare your outputs as in the format of prediction.jsonl, which is to add an output field to each line in leetcode.jsonl corresponding to the output of a LLM to the messages. To calculate the scores, please simply follow evaluate.py, and you will see the… See the full description on the dataset page: https://huggingface.co/datasets/hkust-nlp/LeetCode-O.

<iframe src="https://silytics.si/embed/item/7a92b520-95d8-4e48-9326-7fd2c6066efd" width="420" height="190" style="border:0" loading="lazy" title="Silytics entry"></iframe>

Silytics shows public metadata from Hugging Face and links to the original. Information may be incomplete or out of date. Last refreshed 16 October 2026. Licenses and terms of the original apply to the work itself. Not financial, legal or professional advice.