Skip to main content
Computer Science > Artificial Intelligence
[Submitted on 5 Jul 2024 (this version), latest version 8 Aug 2024 (v2)]
Code Hallucination
Mirza Masfiqur Rahman, Ashish Kundu
Generative models such as large language models are extensively used as code copilots and for whole program generation. However, the programs they generate often have questionable correctness, authenticity and reliability in terms of integration as they might not follow the user requirements, provide incorrect and/or nonsensical outputs, or even contain semantic/syntactic errors - overall known as LLM hallucination. In this work, we present several types of code hallucination. We have generated such hallucinated code manually using large language models. We also present a technique - HallTrigger, in order to demonstrate efficient ways of generating arbitrary code hallucination. Our method leverages 3 different dynamic attributes of LLMs to craft prompts that can successfully trigger hallucinations from models without the need to access model architecture or parameters. Results from popular blackbox models suggest that HallTrigger is indeed effective and the pervasive LLM hallucination have sheer impact on software development.
Subjects:	Artificial Intelligence (cs.AI); Software Engineering (cs.SE)
Cite as:	arXiv:2407.04831 [cs.AI]
 	(or arXiv:2407.04831v1 [cs.AI] for this version)
 	
https://doi.org/10.48550/arXiv.2407.04831
Focus to learn more
Submission history
From: Ashish Kundu [view email]
[v1] Fri, 5 Jul 2024 19:37:37 UTC (45 KB)
[v2] Thu, 8 Aug 2024 01:01:47 UTC (45 KB)

Access Paper:
View PDFHTML (experimental)TeX Source
view license
Current browse context: cs.AI
< prev next >

newrecent2024-07
Change to browse by: cs cs.SE
References & Citations
NASA ADS
Google Scholar
Semantic Scholar
Export BibTeX Citation
Bookmark
Bibliographic Tools
Bibliographic and Citation Tools
Bibliographic Explorer Toggle
Bibliographic Explorer (What is the Explorer?)
Connected Papers Toggle
Connected Papers (What is Connected Papers?)
Litmaps Toggle
Litmaps (What is Litmaps?)
scite.ai Toggle
scite Smart Citations (What are Smart Citations?)
Code, Data, Media
Demos
Related Papers
About arXivLabs
Which authors of this paper are endorsers? | Disable MathJax (What is MathJax?)
About
Help
Contact
Subscribe
Copyright
Privacy Policy
Web Accessibility Assistance

arXiv Operational Status