Simin Sun will present her scientific thesis for the award of the Licentiate of Engineering degree in Data and Information Technology.
Background: As software systems grow in size and complexity, traditional Continuous Integration and Continuous Deployment/Delivery (CI/CD) pipelines face significant efficiency bottlenecks. Although researchers have begun using Large Language Models (LLMs) to optimize pipeline tasks, these intelligent agents are rarely integrated directly into pipeline control flows or gating decisions. This disconnection stems from a focus on agent design at the expense of understanding internal pipeline dynamics and structural constraints, limiting practical efficiency gains.
Objective: The goal of this thesis is to investigate how LLM-based agents can be integrated into industrial CI/CD deployment pipelines in ways that improve pipeline efficiency while remaining compatible with pipeline structure, governance, and high-assurance requirements.
Method: This research employs a multi-phase, mixed-methods approach across five appended papers. The methodology includes (1) computational experiments using open datasets to probe the latent representations of LLMs; (2) experimental study to design a multi-modal, LLM-supported retrieval systems in industrial settings; and (3) mixed-method case studies involving ten companies to analyze CI/CD failures and optimization strategies, and (4) qualitative study exploring the socio-technical challenges of AI adoption in safety-critical and embedded software domains.
Findings: The results indicate that, for the adoption of LLM-based agents in industrial CI/CD, the pre-merge stage emerges as the most practical point of integration, since it is both a major source of efficiency bottlenecks and a stage at which multiple artifacts are available for analysis. When analyzing these artifacts, only trillion-parameter scale models are close to realizing literate programming and achieving a high degree of semantic alignment across different artifacts. In settings where such computational resources are unavailable, performance can be augmented by deliberate orchestration of smaller models, supplemented by techniques such as data augmentation and prompt optimization. Overall, given the constraints imposed by pipeline feedback latency and quality assurance requirements, LLM-based agents are better deployed as constrained, human-supervised components than as standalone solutions.
Conclusions: The findings of this thesis point to a realistic evolutionary path in the form of the LLM-assisted agentic pipeline: a constrained integration framework in which LLM-based agents, traditional tools, and human actors collaborate within explicit control flow, artifact handoffs, and gating mechanisms. Under these conditions, LLM-based agents can improve pipeline efficiency without weakening reliability and traceability.