darshanmakwana/cc3m_tokenized
Experiments for training Auto Regressive models for text-to-image generation This dataset is derived from conceptual captions (CC3M) which contains roughly 3.3M image and caption pairs. For images we use 1d-tokenizer by bytedance which tokenizes a 256 * 256 image into 32 tokens while still achieving SOTA fidelity ratio. For text we train a BPE based tokenizer on the image captions dataset with a vocab size set to 30K, where 4096 tokens where used to represent images, 9 to… See the full description on the dataset page: https://huggingface.co/datasets/darshanmakwana/cc3m_tokenized.
| Author / organization | darshanmakwana |
|---|---|
| Downloads | 557 (as reported by Hugging Face) |
| Likes | 1 |
| Created | 23 June 2024 |
| Last updated at source | 25 June 2024 |
Popularity, last 3 days
557 now (-34 since 2026-10-16). Counts as reported by the source, snapshotted daily by Silytics.
Recent changes
- 16 Oct 2026 Description: Experiments for training Auto Regressive models for text-to-image generation This dataset is derived from conceptual captions (CC3M) which contains roughly 3.3M image and caption pairs. For images we use 1d-tokenizer by bytedance which toke → Experiments for training Auto Regressive models for text-to-image generation This dataset is derived from conceptual captions (CC3M) which contains roughly 3.3M image and caption pairs. For images we use 1d-tokenizer by bytedance which toke
Changes detected by Silytics between refreshes of the source.
Silytics shows public metadata from Hugging Face and links to the original. Information may be incomplete or out of date. Last refreshed 18 October 2026. Licenses and terms of the original apply to the work itself. Not financial, legal or professional advice.