Blog

What’s Behind ChatGPT? The AI That Understands (Almost) Everything

During Girls in ICT Day at the Universitat Politècnica de Catalunya – BarcelonaTech (UPC) in Manresa, UPC Manresa, and with the support of the MERIT project, we had the opportunity to hear from Noèlia Català, an expert in artificial intelligence at IThinkUPC. She explained what lies behind tools like ChatGPT and how they are now capable of performing tasks traditionally done by humans—such as writing, summarising, providing information on almost any topic, or chatting like a real person. 

Artificial intelligence might seem like something new, but intelligent systems have existed in some form for a long time. 

• In ancient Greece, automatons were created for theatrical performances—machines that moved using pulleys, steam, and ropes. Some temples had automatic doors that opened with the heat from altar fires, activating a hydraulic system. There were even machines that dispensed holy water when a coin was inserted and triggered a lever.
• Centuries later, in the 17th century, scientists like Descartes and Pascal built machines capable of performing calculations. That’s still far from true thinking, but it was another step forward.
• In the 20th century, the first computers appeared—though they were huge and limited. At that time, Alan Turing asked the big question: “Can machines think?” Not long after, the first “intelligent” programs emerged, including chess-playing systems and problem solvers. There was also ELIZA, considered the first chatbot in history.
• By the 2000s, virtual assistants like Siri and Alexa came onto the scene, along with self-driving cars and automatic translators. Deep learning also emerged—using neural networks inspired by the human brain. With this, AI started recognizing images, generating text, and handling much more complex tasks. And in 2022, ChatGPT was born—the first of a new wave of generative AIs.

But how does ChatGPT actually work? What did it take for it to answer almost like a human and know so much?

First, it undergoes massive training: it’s shown billions of words—texts, conversations, articles—sourced from databases, books, Wikipedia, and more. The goal is to learn the patterns in how we write and speak, and how words combine to create meaning. 

To make this process possible, all the text is broken down into tokens—like pieces of a puzzle. A token might be a full word or just part of one. The AI doesn’t understand in a human way, but it learns that if it often sees “Good morning, how…” in the texts, the next word is likely to be “are you?” This allows it to predict what comes next in a conversation. 

To understand these connections, ChatGPT uses a technique called embeddings, which turns words into numbers and performs calculations to find meaning and relationships. It’s like placing each word on a map—so it can “see” which words are closer. For example, “king” and “queen” are more related than “king” and “cheese.” Noèlia gave some fun examples: 

• “Queen,” “princess,” “woman,” and “crown” all relate to femininity.
• “King,” “Michael Jackson,” and “dance” have a shared connection.
• “Penguin” and “emperor” are connected by species and title.
• “Queen,” “princess,” “crown,” “king,” and “emperor” all belong to the world of royalty.

After training, tokenisation, and embeddings, human feedback is essential. Real people review ChatGPT’s answers to specific questions, rating whether they’re good or not. This feedback helps the system improve and avoid repeating mistakes. In other words, it learns to prioritise more helpful, safe, and respectful responses. 

Preventing bias is also key. If the training data isn’t handled carefully, the AI could learn from hateful content, stereotypes, or discriminatory ideas. That’s why filters and oversight are needed to ensure fairness. 

And this is only the beginning! Just three years after ChatGPT’s debut, many other generative AIs now exist. Some specialise in converting speech to text, creating or interpreting images, composing music, and more. 

The next step is to integrate logic and math to help AI systems reason more accurately. With this, they could explain ideas, suggest recipes based on ingredients, or speak with a realistic human voice. 

But with great power comes great responsibility. We need to think about how to use AI ethically and wisely. We must also consider its impact on work, justice, education, and beyond. To manage all this, we’ll need many trained professionals—like the ones studying at UPC Manresa, first in the Bachelor’s Degree in ICT Systems Engineering and then in the Master’s degree in Machine Learning and Cybersecurity for Internet Connected Systems, developed under the European MERIT project. 

Most of us have probably used ChatGPT at least once (if not many times!). So next time you ask it a question, you’ll know just how much science, maths, engineering, and data are working behind the scenes. And this is just the start—we’ll see how it evolves!