liswei/Taiwan-Text-Excellence-2B
High quality corpus for Taiwanese culture and Traditional Chinese Taiwan Text Excellence (TTE) Contains high quality news and articles in Traditional Chinese. The data processing pipeline is optimized for LLM performance. Is de-duplicated and cleaned using both rule-based and learning-based filters. E.g., urls/emails/html tags/abnormal characters are cleaned, and numbers (full-width or half-width) are normalized. Contains ~2 billion tokens, measured using BPE… See the full description on the dataset page: https://huggingface.co/datasets/liswei/Taiwan-Text-Excellence-2B.
| Author / organization | liswei |
|---|---|
| License (as declared) | cc-by-nc-sa-4.0 |
| Downloads | 190 (as reported by Hugging Face) |
| Likes | 22 |
| Created | 7 May 2024 |
| Last updated at source | 2 June 2024 |
More from liswei
- liswei/common-crawl-zhtwDataset
- liswei/c4-zhtwDataset
- liswei/news-collection-zhtwDataset
Silytics shows public metadata from Hugging Face and links to the original. Information may be incomplete or out of date. Last refreshed 7 November 2026. Licenses and terms of the original apply to the work itself. Not financial, legal or professional advice.