Optimizing RAG-Assisted Code Generation for Cloud-Native ML Pipelines Through Dynamic Context Window Management

Abstract

The proliferation of cloud-native machine learning (ML) infrastructure has created significant demand for automated code generation systems capable of producing syntactically and semantically correct pipeline configurations. Retrieval-Augmented Generation (RAG) offers a principled mechanism for grounding large language model (LMM) inference in dynamically retrieved contextual knowledge; however, naive implementations suffer from context window saturation and relevance dilution when applied to complex multi-component ML pipeline repositories. This paper proposes a Dynamic Context Window Management (DCWM) framework integrated within a RAG-assisted code generation system targeting cloud-native ML workflows. The framework introduces a multi-stage retrieval mechanism coupled with an adaptive context compression module that selectively ranks and prioritizes retrieved code artifacts through attention-weighted scoring. An abstract syntax tree guided generation pipeline further enforces syntactic validity of produced pipeline code. Empirical evaluation across three benchmark task categories demonstrates a 23.7% improvement in pass@1 accuracy and a 19.4% reduction in hallucinated API calls relative to static context window baselines. These findings establish dynamic context management as a first-class design consideration in RAG-enhanced code generation for cloud-native environments.