OpenAI releases new GPT-4 model to find errors in ChatGPT's responses

The internet is filled with stories of generative artificial intelligence (GenAI) chatbots passing the toughest exams, and at the same time, messing up simple facts. Google recently added a GenAI feature to its search engine, its bread-and-butter, and after it gave inaccurate answers to simple questions, rolled it back within weeks.

OpenAI, maker of ChatGPT which catapulted the GenAI technology to fame, has taken a step to address errors like this, with its new model CriticGPT.

CriticGPT is OpenAI's latest model based on its GPT-4 model, specifically designed to critique and catch errors in ChatGPT's output code. The tool will help the company with the process of alignment with AI systems, through what developers term Reinforcement learning from human feedback or RLHF. This will help make responses from large language models more accurate.

RLHF is a machine learning (ML) technique that uses human feedback to optimise language models to self-learn more efficiently. A key part of it is collecting comparisons in which people, called AI trainers, rate different ChatGPT responses against each other.

OpenAI said that with ChatGPT becoming more accurate and errors more subtle, a model like CriticGPT became necessary to find inaccuracies.

"CriticGPT’s suggestions are not always correct, but we find that they can help trainers to catch many more problems with model-written answers than they would without AI help," the company said in a blog post.
OpenAI iterated that despite such feedback, models still hallucinate and trainers also make labelling mistakes, as CriticGPT can help trainers only so much.