Towards a Digital Archivist: Applications of LLMs in Automated Web Archive Description

This chapter delves into the automation of web archive descriptions using large language models, specifically the Qwen3-8B model. The study focuses on fine-tuning the model with a curated dataset of high-quality web page descriptions to generate accurate and contextually relevant metadata. The process involves extracting and cleaning HTML records from WARC files, followed by generating descriptive metadata. The fine-tuned model is evaluated by professionals, including archivists and librarians, to assess its reliability and usability. The results show that the model can produce descriptions with high semantic fidelity and minimal need for manual edits. The chapter also discusses the system's implementation, including a user-friendly interface for processing and previewing archived web content. Additionally, it highlights the challenges and future improvements needed for handling sensitive content and non-English web pages. The study concludes that the automated system significantly enhances metadata workflows in digital preservation, offering a robust solution for generating trustworthy archival descriptions.