Generative AI is one of the most exciting and innovative fields of artificial intelligence, with applications ranging from art and music to medicine and engineering. But what exactly is generative AI, and what are some of the key terms and concepts you need to know to understand it better? In this article, we will explain 12 of the most common and important keywords related to generative AI and its recent developments.
Why is this necessary?
The primary purpose of this glossary is to help people understand how AI systems work. With the rapid development of new models, apps, software, and AI-enhanced devices, having a fundamental grasp of these keywords will make it easier to comprehend these advancements.
Moreover, data shows that generative AI is already significantly impacting the economy in various ways. Staying updated with the latest developments is crucial, but first, you need to know the basics!
Keywords
The keywords start at the foundations and then build up in difficulty so enjoy!
Note: This list is not comprehensive.
1. Generative AI
Generative AI refers to a type of artificial intelligence that creates new, original content, such as text, images, or music, based on patterns and examples it has learned from existing data.
2. LLM
LLM commonly stands for “Large Language Model.” It refers to advanced AI models, like GPT-3, designed to understand and generate human-like text on a large scale, enabling a broad range of natural language processing tasks.
They are trained on large datasets of text which allow them to predict the next word in a sentence. By learning from a large and diverse corpus of text, these models can capture general linguistic patterns and common sense knowledge, but they may also inherit biases and errors from the data.
Examples include:
- GPT-o1, Claude Sonnet 3.5, Mistral , and Gemini— Closed Source
- Llama 3.2 — Open Source
3. Chatbot
A chatbot is a type of software program, often employing generative AI or large language models, that engages in conversation with users. It uses natural language processing to comprehend and respond to user input, serving purposes like providing information or assistance.
Nowadays, you can find general purpose chatbots like Gemini or ChatGPT are not ideal for more specfic use cases. As a result, there are thousands of other chatbots online for finance, healthcare, shopping, recreations and more.
4. GPT
GPT, short for Generative Pre-trained Transformer, is a powerful language model. It creates human-like text based on patterns learned from diverse data, showcasing advanced natural language processing capabilities.
Most AI applications such as chatbots or image generators will be built on top of this model but not all.
5. Prompt
A prompt is a specific input or request given to a generative AI, like GPT, to elicit a response or generate content in a conversational manner, as seen in chatbots.
It is the main way to interact with a LLM but we also input files, images and even audio depending on what model you are using.
6. Prompt Engineering
Prompt engineering involves strategically crafting inputs or queries to guide generative AI models in producing desired outputs. It’s a technique to influence the responses of systems like chatbots or language models like GPT.
Some examples of prompt engineering techniques include:
- Few-shot prompting
- ReAct prompting
- Chain-of-thought (CoT)
7. Tokens
Tokens, in the context of language models, are individual units of meaning such as words or characters. They form the basis for comprehension and generation within generative AI, like GPT, when processing text.
You can use OpenAI’s tokenizer to see how many tokens you are using for a prompt.
Generate a recipe list for organic and meat-free dishes for a family of 5. Ensure there are no nuts present in any of recipes and include details about macronutrients.
So 29 words is equivalent to around 36 tokens. The conversions will vary depending on which model you are using.
8. Context Window / Context Length
A context window, or context length, in the realm of language models, refers to the number of preceding words or tokens that a model considers when generating or understanding a specific word or token in a sequence of text.
Most recently, OpenAI announced their new GPT-4 Turbo which has a context window of 128,000 which is equivalent to around 300 pages of text.
9. Fine-Tuning
A fine-tuned model is an AI model that has undergone additional training on specific data or tasks after its initial pre-training. This process refines the model’s performance for particular applications or domains.
Common use cases of fine-tuning include creating specialized chatbots that you might see in your mobile banking app or when online shopping.
If you are interested in learning to fine-tune models, you can check out this blog which shows you how to fine-tune Llama on customer service chat logs for improved responses.
10. Attention Mechanism
An attention mechanism in AI refers to a component that enables models to focus on specific parts of input data when making decisions or generating outputs. It helps capture relevant information, enhancing performance.
11. AI Image Generation
Certain models are tuned to turn text inputs into images and these are called AI image generators. They have been trained on millions of examples images, artworks, illustrations, and pretty much anything visual you can find on the web.
Examples Include:
- DALL-E — An AI model by OpenAI that generates images from textual descriptions, known for its creativity and versatility.
- Stable Diffusion —A latent diffusion model that generates high-resolution images from text inputs. Created by Stability AI
- MidJourney — A popular AI tool that creates high-quality, artistic images from text prompts.
- TEV1 — An advanced AI model designed to create abstract and thematic artworks based on an input theme. Created by Asycd, it makes uses of a ReAct prompt framework, multiple AI agents, and prompt engineering to create novel images.
12. Embeddings
Embeddings are numerical representations of words or entities in a way that captures their semantic relationships. They enable AI models, like language models, to understand and process language effectively.
Below is an example of how a sentence might be decomposed into a vector.
You can read our article on embeddings and context loading for more information.
13. Retrieval Augmented Generation (RAG)
Retrieval Augmented Generation allows us to provide additional context in our prompt by automatically sourcing relevant information from a vector database to be injected into your prompt at inference.
Your initial prompt might look like this
“How do you feel about Donald Trump being president?”
With RAG, the prompt that actually reaches the LLM might look like this.
“How do you feel about Donald Trump being president?” — Use the following information to answer the question: As of December 2024, Donald Trump has been elected as the next President of the United States, defeating Kamala Harris in the 2024 election. He is set to take office in January 2025, marking his second term as President. Trump has announced plans to impose significant tariffs on goods imported from Canada, Mexico, and China.
The latter should illicit a more informed response from the AI.
RAG is the standard approach to providing additional context. It involves integrating a vector database and a retrieval mechanism to find relevant text your prompt/query.
14. Graph Retrieval Augmented Generation
This advanced technique enhances the traditional Retrieval Augmented Generation (RAG) by leveraging graph data structures. Graph RAG uses nodes and edges to represent and retrieve information, allowing for more accurate and contextually relevant responses.
By incorporating relationships and connections between data points, Graph RAG improves the quality and coherence of generated content.
15. AI Agent
An AI agent is an intelligent entity integrated into systems like RAG to perform specific tasks autonomously. These tasks can include document retrieval, summarization, response generation, and more. AI agents enhance the efficiency and adaptability of AI systems by handling complex operations and providing specialized support based on the context and requirements of the task at hand.
