-
Developing a Scalable Benchmark for Assessing Large Language Models in Knowledge Graph Engineering
Abstract: As the field of Large Language Models (LLMs) evolves at an accelerated pace, the critical need to assess and monitor their performance emerges. We introduce a benchmarking framework focused on knowledge graph engineering (KGE) accompanied by three challenges addressing syntax and error correction, facts extraction and dataset generation. We show that while being a useful tool, LLMs are yet unfit t… ▽ More
Submitted 31 August, 2023; originally announced August 2023.
Comments: To be published in SEMANTICS 2023 poster track proceedings. SEMANTICS 2023 EU: 19th International Conference on Semantic Systems, September 20-22, 2023, Leipzig, Germany
-
arXiv:2307.06917 [pdf, ps, other]
LLM-assisted Knowledge Graph Engineering: Experiments with ChatGPT
Abstract: Knowledge Graphs (KG) provide us with a structured, flexible, transparent, cross-system, and collaborative way of organizing our knowledge and data across various domains in society and industrial as well as scientific disciplines. KGs surpass any other form of representation in terms of effectiveness. However, Knowledge Graph Engineering (KGE) requires in-depth experiences of graph structures, we… ▽ More
Submitted 13 July, 2023; originally announced July 2023.
Comments: to appear in conference proceedings of AI-Tomorrow-23, 29.+30.6.2023 in Leipzig, Germany
Journal ref: Informatik aktuell. First Working Conference on Artificial Intelligence Development for a Resilient and Sustainable Tomorrow 2023. AIDRST 2023. p. 103-115