Evaluating AI Agent Prompting and Fine-Tuning Strategies for Structured Data Extraction from Industrial PDFs

Industrial environments generate large volumes of heterogeneous, unstructured PDF documents, such as material certificates, making reliable automated data extraction a methodological challenge. Traditional LLM-based pipelines often struggle with noisy OCR output, inconsistent layouts, and schema variability, leading to unstable outputs and reduced accuracy. This study addresses the problem of designing a robust, consistent extraction pipeline capable of operating across diverse document structures under both limited and scaled domain-specific supervision regimes. We propose a coordinated multi-agent framework integrating layout-aware OCR processing, expert-guided prompt engineering, systematic prompt refinement, controlled synthetic dataset expansion, and parameter-efficient adaptation via LoRA. The architecture includes specialised agents for raw text extraction, dataset preparation, prompt optimisation, LoRA configuration search, output processing, and structured evaluation, forming an iterative workflow that improves stability and reduces error propagation. Structural compliance and retrieval accuracy are evalu- ated separately using binomial modelling with Wilson confidence intervals. Zero-shot, few-shot, and LoRA fine-tuned LLMs are assessed across commercial and open-weight models. Results show that few-shot prompting enables rapid structural alignment under low-resource conditions, while fine-tuning effectiveness depends on supervision diversity. Merged supervi- sion, defined as fine-tuning on combined original and synthetic certificates (FT-Merged), sub- stantially outperforms limited-scale LoRA adaptation. Under FT-Merged, Mistral 7B achieves 97.05% retrieval accuracy with near-perfect structural compliance, approaching commercial reliability. GPT-4o and DeepSeek Chat achieve approximately 99% accuracy without task- specific fine-tuning. Cross-domain validation confirms structural generalisation, and runtime– cost analysis highlights trade-offs between accuracy, latency, financial cost, and deployment con- trol. Overall, robust industrial extraction emerges from coordinated architectural orchestration rather than model choice alone.