All the fuss about ChatGPT
Chat Generative Pre-Trained Transformer
ChatGPT is a state-of-the-art AI technology made by OpenAI, which Elon Musk and Sam Altman co-founded. It is a significant language model meaning the AI sucks up a tremendous amount of text and information from social media posts to news articles. It uses that information to train the AI software to generate and predict related content when given a prompt. ChatGPT’s uniqueness comes from its ability to imitate and respond in a human-like fashion. For example, it can help the AI tell the difference between Bark and how context changes the meaning. Tree bark vs. a dog barking.
What is ChatGPT?
ChatGPT (Chat Generative Pre-Trained Transformer) is the new trending artificial intelligence program that provides you with the best answers to your questions; ChatGPT is a “large language chatbot model developed by OpenAI based on GPT-3.5” GPT-3.5 model is a fined-tuned version of GPT3 (Generative Pre-Trained Transformer) model. The GPT-3.5 model was created in January 2022, being trained on vast amounts of code data and information from the internet, including sources such as Reddit discussions, to help ChatGPT learn dialogue and achieve a human-like response style.
Major languages like GPT-3 are often used to follow instructions to “execute the user’s tasks. However, these models often generate toxic or untruthful outputs unrelated to the input instruction” InstructGPT is the result of fine-tuning GPT-3 using the RLHF (Reinforcement learning from human feedback). This technique was used to fine-tune.
ChatGPT was created on November 30, 2022, by San Francisco-based artificial intelligence company OpenAI. OpenAI is an artificial intelligence research company, and its mission is to develop a safe and beneficial artificial general intelligence system or to help others do so; OpenAI has the most “talent-dense researchers and engineers in the field of AI” Also Microsoft is a partner and investor investing about 10 billion dollars (about $31 per person in the US). ChatGPT is the non-profit parent company of the for-profit OpenAI Lp. Being launched as a prototype and quickly gaining attention, anyone can try out their services simply by setting up an account and going through all the terms & conditions along with the disclaimer.
You can use ChatGPT, and if asked, a question will be given detailed responses and articulate answers across its main knowledge domains. Being able to remember the thread of your dialogue using earlier questions and answers to inform its following responses, it derives its solutions from vast volumes of information on the internet, being deeply knowledgeable in areas where there are good training data for it to learn from, after a few days after launch ChatGPT had over 1 million users (about the population of Delaware) trying it out.
Limitations
ChatGPT does have its limitations, however. OpenAi acknowledged that ChatGPT sometimes writes plausible-sounding but incorrect or nonsensical answers. This behavior is common to large language models and is called artificial intelligence hallucinations. ChatGPT, designed around human oversight, can be over-optimized and thus hinder performance, known as Goodhart’s law. A software developer’s site, Stack Overflow, banned ChatGPT answers to programming questions. “Administrators caution because the average rate of getting correct answers from ChatGPT is too low, the posting of answers created by ChatGPT is substantially harmful to the site and users asking or looking for correct answers.” These flaws can be very problematic. We cannot rely on this technology for programming questions. However, we can use it to solve simpler problems, such as what I should get my family member or significant other as a birthday present or what is 2 + 2 ChatGPT can give good advice and suggestions when asked.
ChatGPT works with AI-trained models to supply their users with an immersive experience with AI-trained models for providing an immersive experience to their users via textual forms. The models have been trained in such a way that it chats with humans just like other humans, and they continuously reinforce learning from the feedback received from human beings. Overall ChatGPT is a program that was created to help others with the questions they may have by using users and other people on the internet by RLHF (Reinforcement Learning from Human Feedback), making a first model using supervised fine-tuning, “human AI trainers provided conversations in which they played both sides– the user and an AI assistant.” Using these detailed conversations, they composed their responses and mixed them with the InstructGPT dataset, which was transferred into a dialogue format.
Design / Architecture of ChatGPT
ChatGPT is based on transformer architecture. First suggested in the “Attention Is All You Need” paper, now the latest and greatest technology in NLP (Natural Language Processing).
It started with RNN (recurrent neural network), which is made to receive a series of inputs. “The term ‘series’ here denotes that each input of that sequence has some relationship with its neighbors”. RNN networks and basic networks both learn and remember things from their training. But RNN can learn through prior inputs while creating outputs. RNN is learning from each input to find the correct output of the related information. RNN has two big pitfalls. One is long word sequences. The other is that it takes a long time to train.
Consider this example, “I grew up in Germany with my parents, I spent many years there and have proper knowledge about their culture. That’s why I speak fluently ____.”. The correct word here is German, but because of the gap between Germany and the predicted word, German RNN struggles to predict it, and as you can imagine, the longer the gap, the more memory fades. So, the idea of attention was introduced in a paper, focusing on only the relevant parts instead of metaphorically reading the whole book; it could go to the table of contents and find relevant things. “Attention Is All You Need” introduced a way for GPUs to code and decode these attention layers, naming this architecture “transformer”, which fixed the memory fading issue and let the AI learn faster, allowing the AI to process a large amount of text more effectively.
Fun facts about ChatGPT:
According to ciocoverage.com, ChatGPT allegedly costs 100,000 dollars a day to run. This comes from the price it costs to run A100 GPU, which is 3 dollars an hour. And each word generated costs roughly $0.0003. With millions of users, it’s no wonder that ChatGPT is super expensive. ChatGPT has released a premium version called ChatGPT Plus to get some of that money back. At $20 a month, it guarantees access to ChatGPT, which is helpful during the hottest times when all the available servers are being used. It also has other perks, like a decrease in the time it takes to respond, different priorities, and access to new features.
The dataset used for ChatGPT’s training was over 570 GB worth of text, with over 300 billion words. Many schools are upset and banning Chat GPT, saying it poses a risk to “student’s creativity and problem-solving abilities”.
ChatGPT Process
As an AI language model, ChatGPT follows a systematic process to answer user questions. This process typically involves several steps, which are outlined below:
1. Understanding the user’s question: ChatGPT starts by carefully reading and analyzing the user’s query to understand its context, purpose, and intent. This step is crucial because it helps ChatGPT identify the specific information the user seeks and tailor its response accordingly.
2. Searching for relevant information: Once ChatGPT understands the user’s question, it searches its vast knowledge database to identify relevant information. This database includes a variety of sources, such as books, articles, and online resources.
3. Analyzing the question and information: After gathering relevant information, ChatGPT analyzes the user’s query and the information it has collected. It uses natural language processing and machine learning algorithms to identify the most relevant and accurate information to include in its response.
4. Generating a response: ChatGPT generates a response to the user’s question based on the information gathered and analyzed. It uses natural language generation techniques to ensure the answer is well-written and easily understood.
5. Verifying the accuracy of the response: Once the response is generated, ChatGPT checks it for accuracy, coherence, and relevance. It verifies any facts or figures and ensures up-to-date and accurate information.
Overall, the process that ChatGPT follows to answer a user’s question is a complex one that involves understanding the question, searching for relevant information, analyzing the question and information, generating a response, and verifying the accuracy of the answer.”
How can ChatGPT and AI help us in the future?
ChatGPT won’t necessarily be the AI for the future, as large corporations such as Google will focus on Bard.
One big thing that has come out of ChatGPT is the idea of having AI replace the likes of writers, artists, and coders.
With any change and new technologies, creative professionals must be willing to embrace such “intelligent tools” and learn new skills. For example, writers may need natural language processing (NLP) tools to analyze and optimize their writing.
Takeaway
ChatGPT’s unique conversational style makes it truly special compared to other Chatbots. The transformer architecture also makes it efficient. It is no doubt expected that other large language models will be able to compete and potentially overcome OpenAI’s ChatGPT. But for now, we can chat with ChatGPT!
Try it out for yourself and check human ingenuity; click here https://chat.openai.com/chat
