The Two Sides of the Coin : Hallucination Generation and Detection with LLMs as Evaluators for LLMs
In: CLEF 2024 : Working Notes of the Conference and Labs of the Evaluation Forum / Faggioli, Guglielmo; Ferro, Nicola; Galuščáková, Petra; García Seco de Herrera, Alba (Eds.). - CLEF 2024, Conference and Labs of the Evaluation Forum; Grenoble, France; 09.09-12.09.2024 - In: CEUR Workshop Proceedings - CEUR-WS, Vol. 3740, pp. 727 - 758
2024Essay (Conference) in Conference proceedingsOA Gold
Faculty of Information Science and Communication Studies » Institute of Information ScienceFaculty of Computer Science and Engineering Science » Institut für Data Science, Engineering, and Analytics
Title in English:
The Two Sides of the Coin : Hallucination Generation and Detection with LLMs as Evaluators for LLMs
Author:
Bui, Anh Thu Maria
- SCOPUS
- 59253227800
- Other
- corresponding author
- SCOPUS
- 57232029600
- Other
- corresponding author
- SCOPUS
- 59253046300
- Other
- corresponding author
- SCOPUS
- 59253046400
- Other
- corresponding author
- SCOPUS
- 59253494000
- Other
- corresponding author
- DHSB-ID
- THK0002110
- ORCID
-
0000-0002-1765-2449
- SCOPUS
- 57210368795
- Other
- person connected with TH Köln
corresponding author
- DHSB-ID
- THK0048082
- SCOPUS
- 57188957425
- SCOPUS
- 57219734508
- SCOPUS
- 58361418600
- Other
- person connected with TH Köln
corresponding author
- DHSB-ID
- THK0002510
- ORCID
-
0000-0002-8817-4632
- SCOPUS
- 35758004800
- Other
- person connected with TH Köln
corresponding author
Year of publication:
2024
„Publication Channel“:
OA Gold
Scopus ID
Language of text:
English
Conference
CLEF 2024, Conference and Labs of the Evaluation Forum; Grenoble, France; 09.09-12.09.2024
Editor:
Faggioli, Guglielmo;Ferro, Nicola;Galuščáková, Petra;García Seco de Herrera, Alba
Place of publication:
Aachen
Publisher:
RWTH Aachen
Year of publication:
2024
„Publication Channel“:
OA Gold
in:
pp. 727 - 758
Language of text:
English
Appears in
Title (abbreviated):
CEUR Workshop Proc.
Publisher:
CEUR-WS
in:
Vol. 3740
ISSN
ZDB ID
WWW URL:
Type of resource:
Text
Type of resource:
Text
Access Rights:
open access
Practice Partner:
No
Category:
Research
Part of statistic:
Not part of statistic
Keyword, Topic:
Ensemble Majority Voting ; Gemma ; GPT-3.5 Turbo ; GPT-4 ; Hallucination Detection ; Hallucination Generation ; Llama 3 ; LLMs as Evaluators
Type of resource:
Text
Access Rights:
open access
Practice Partner:
No
Category:
Research
Part of statistic:
Part of statistic
Abstract in English:
Hallucination detection in Large Language Models (LLMs) is crucial for ensuring their reliability. This work presents our participation in the CLEF ELOQUENT HalluciGen shared task, where the goal is to develop evaluators for both generating and detecting hallucinated content. We explored the capabilities of four LLMs: Llama 3, Gemma, GPT-3.5 Turbo, and GPT-4, for this purpose. We also employed ensemble majority voting to incorporate all four models for the detection task. The results provide valuable insights into the strengths and weaknesses of these LLMs in handling hallucination generation and detection tasks.