A Consensus-Driven Information Extraction Method via Voting and Pruning over LLM Chatbot Outputs for a Novel Agricultural Dataset
Bildiriler > Tam Metin Bildiri
Yazarlar: Esat Fazlullah ÇELİK, Aybars UĞUR
2026 8th International Congress on Human-Computer Interaction, Optimization and Robotic Applications (ICHORA), , , Uluslararası, Tam Metin Bildiri, ss 1-5
DOI: 10.1109/ichora69329.2026.11537216
Özet
Text-derived and expert-curated structural knowledge remains significantly underrepresented in agricultural Artificial Intelligence (AI) benchmarks, even though it plays a crucial role in knowledge graph construction and structured reasoning. This study introduces a novel agricultural dataset curated from the Agrios’ Plant Pathology book, comprising 589 gold subject–relation–object (SRO) triples extracted from 168 sentences. Five commercial Large Language Model (LLM) chatbots, Gemini 3 Pro, ChatGPT 5.2 Thinking, Claude Opus 4.6 Thinking, DeepSeek DeepThink, and Grok 4.1 Thinking, are evaluated for triple extraction using semantic matching. DeepSeek achieves the best single-model performance with an F1 score of 65.31%. Majority and weighted voting methods fail to outperform the best model, yielding F1 scores of 57-58%. To address this, the Anchored Pruning and Consensus Augmenting (APCA) method is proposed. APCA is an ensemble method that selects the best model as the anchor, prunes anchor-only triples to suppress model-specific noise and augments the anchor with high-consensus triples from the remaining models to capture what the anchor cannot. An ablation study reveals that while pruning primarily enhances precision, augmentation primarily improves recall. APCA combines these two processes, yielding the best F1 score of 68.48%. These results indicate that anchored pruned and consensus-driven editing can improve LLM-based triple extraction in specialized agricultural text beyond both the strongest standalone model and conventional voting ensembles.