Byun at SemEval-2024 Task 6: Text Classification on Hallucinating Text with Simple Data Augmentation

AbstractThis paper aims to classify sentences to see if it is hallucinating, meaning the generative language model has output text that has very little to do with the user’s input, or not. This classification task is part of the Semeval 2024’s task on Hallucinations and Related Observable Over-generation Mistakes, AKA SHROOM, which aims to improve awkward-sounding texts generated by AI. This paper will first go over the first attempt at creating predictions, then show the actual scores achieved after submitting the first attempt results to Semeval, then finally go over potential improvements to be made.