FLUX-Font: Stylized Font Generation Based on Fine-Tuning Pre-trained Diffusion Transformer

The generation of Chinese fonts presents both significant challenges and practical value. Despite numerous advances, how to generate high-quality font images with a few reference samples remains an open problem. Existing mainstream approaches typically rely on training small diffusion models from scratch, which is not only computationally intensive and time-consuming, but also prone to errors in glyph structure and inconsistencies in style when handling complex or rare character forms. To address these issues, this paper proposes FLUX-Font, a Chinese font generation method based on the pre-trained large diffusion model FLUX [1]. Particularly, FLUX-Font adopts LoRA-based fine-tuning as its core strategy and uses two separate LoRA modules to fine-tune the glyph and style networks, respectively. To further enhance the performance of the model and the quality of the generated outputs, we incorporate a character-aware loss function that leverages an OCR model to guide the optimization process, thus enhancing the structural stability of generated glyphs. Comprehensive experiments conducted on a benchmark font demonstrate that FLUX-Font consistently outperforms existing methods on various metrics while significantly reducing computational cost and training time. These findings suggest that FLUX-Font offers a promising and efficient approach for Chinese font generation under a few-shot setting.