Enhancing Code Generation for Large Language Models Using Fine-Grained Distillation

2025
OriginalPaper
Chapter

Published in:

Authors: HaoRan Jin; ZhiQiu Huang; WeiWei Li; YaoShen Yu; Chao Wang
Published in: Advanced Intelligent Computing Technology and Applications
Publisher: Springer Nature Singapore

Read first chapter Log in

Activate our intelligent search to find suitable subject content or patents.

search-config

AI-assisted search

Off

Abstract

With the rise of powerful code generation large language models (LLMs), there is increasing interest in distilling the capabilities of these LLMs to smaller LLMs. Due to the difference between code and text, many methods use reinforcement learning for guidance in distillation. However, most of these methods only utilize the results of unit tests or are too complex and difficult to apply in practice, resulting in poor performance after model distillation. Therefore, we propose a method to make the granularity in reinforcement learning more nuanced and simple to apply. We incorporated the process of generating code for the model into reinforcement learning as a way to guide the distillation process. We evaluated on benchmark datasets HumanEval and MBPP, and experimented with different types of LLMs. The experimental results show that our method achieves good performance.

Please log in to get access to your license.

Dont have a licence yet? Then find out more about our products and how to get one now:

Springer Professional "Business + Economics & Engineering + Technology"

Online-Abonnement

Springer Professional "Business + Economics & Engineering + Technology" gives you access to:

more than 102.000 books
more than 537 journals

from the following subject areas:

Automotive
Construction + Real Estate
Business IT + Informatics
Electrical Engineering + Electronics
Energy + Sustainability
Finance + Banking
Management + Leadership
Marketing + Sales
Mechanical Engineering + Materials
Insurance + Risk

Secure your knowledge advantage now!

inform now

Springer Professional "Engineering + Technology"

Online-Abonnement

Springer Professional "Engineering + Technology" gives you access to:

more than 67.000 books
more than 390 journals

from the following specialised fileds:

Automotive
Business IT + Informatics
Construction + Real Estate
Electrical Engineering + Electronics
Energy + Sustainability
Mechanical Engineering + Materials

Secure your knowledge advantage now!

inform now

Springer Professional "Business + Economics"

Online-Abonnement

Springer Professional "Business + Economics" gives you access to:

more than 67.000 books
more than 340 journals

from the following specialised fileds:

Construction + Real Estate
Business IT + Informatics
Finance + Banking
Management + Leadership
Marketing + Sales
Insurance + Risk

Secure your knowledge advantage now!

inform now

previous chapter Automated Construction of High-quality Evaluation Datasets Based on LLMs

next chapter Morphological Recombination-Based Neural Machine Translation with Self-Supervised Data Augmentation

Metadata
Footnotes
Literature

Title: Enhancing Code Generation for Large Language Models Using Fine-Grained Distillation
Authors: HaoRan Jin
ZhiQiu Huang
WeiWei Li
YaoShen Yu
Chao Wang
Publisher: Springer Nature Singapore
Book: Advanced Intelligent Computing Technology and Applications
Print ISBN: 978-981-9500-13-0

Electronic ISBN: 978-981-9500-14-7

Copyright Year: 2025
DOI: https://doi.org/10.1007/978-981-95-0014-7_32

This content is only visible if you are logged in and have the appropriate permissions.