According to Meta’s chief AI scientist, the public considers OpenAI’s ChatGPT as innovative, although the same techniques and efforts are being done at numerous research labs.

“In terms of underlying techniques, ChatGPT is not particularly innovative,” Yann LeCun, Meta’s top AI scientist, spoke to a select group of press and executives via Zoom last week.

“It’s nothing revolutionary, although that’s the way it’s perceived in the public,” said LeCun. “It’s just that, you know, it’s well put together, it’s nicely done.”

Many firms and academic institutes have constructed data-driven AI systems in the past, according to LeCun. He disagrees with the notion that OpenAI is unique in its field.

“OpenAI is not particularly an advance compared to the other labs, at all,” said LeCun.

“It’s not only just Google and Meta but there are half a dozen startups that basically have very similar technology to it,” added LeCun. “I don’t want to say it’s not rocket science, but it’s really shared, there’s no secret behind it, if you will.”

LeCun emphasized how ChatGPT and the program on which it is based, OpenAI’s GPT-3, are made up of numerous pieces of technology developed over many years by different parties.

“You have to realize, ChatGPT uses Transformer architectures that are pre-trained in this self-supervised manner,” observed LeCun. “Self-supervised learning is something I’ve been advocating for a long time, even before OpenAI existed,” he said.

“Transformers is a Google invention,”

LeCun was referring to Google’s language neural net, which launched in 2017 and has since been the foundation for a wide range of language programs, including GPT-3.

According to LeCun, such language programs have been in development for decades.

“Large language models, the first neural net language model — at the time, it was large, by today’s standards, it’s tiny — was by Yoshua Bengio, about 20 years ago,”

LeCun mentioned the leader of Canada’s MILA AI institute and highlighted that Bengio’s idea of attention was instrumental in developing the Transformer at Google, which has now become a critical component in all language models.