Ontology as a hyperparameter: Task-aware knowledge graph construction using LLMs

Abstract

The project investigates the construction of task-aware knowledge graphs (KGs) from text, treating graph structure as an optimization target rather than a fixed representation. Existing frameworks such as Neo4j-based pipelines and LLM-driven tools (e.g., LlamaIndex) focus on constructing large, general-purpose graphs without incorporating feedback from downstream tasks, despite evidence from evaluation frameworks (e.g., KGrEaT, GEval, KGBench) that KG utility is task-dependent. The research addresses this gap by proposing a feedback-driven approach in which ontology is treated as a tunable component influencing predictive performance. The study uses document-based datasets linking text to measurable outcomes, including financial news (FNSPID) paired with stock price movements. LLM agents are employed for ontology evolution and triple extraction, enabling scalable generation of multiple KG variants. Graph-derived features are then computed over temporal windows and used in predictive models, whose performance is fed back to the agents to refine ontology design, closing the loop between graph construction and downstream task. Preliminary results show that graph structure significantly impacts predictive performance, with adaptive ontology refinement leading to measurable improvements.