What are LLM Agents?

Introduction

LLM agents, short for Large Language Model agents, are advanced AI systems that utilize large language models (LLMs) as their central computational engine.

Let's consider a scenario where you're using Robo, your helpful robot assistant, to plan a vacation.

You ask Robo, "What's the best time to visit the Grand Canyon?"

Now, Robo is equipped with a general-purpose LLM, which can provide information on a wide range of topics. However, the question about the best time to visit the Grand Canyon requires specific knowledge about weather patterns, tourist seasons, and other factors that influence the visitor experience.

Robo starts by consulting its general-purpose LLM to gather basic information about the Grand Canyon. It can tell you about the location, history, and general attractions of the Grand Canyon.

But to answer your question accurately, Robo needs more specialized knowledge. It needs to consider factors like weather conditions, crowd levels, and peak tourist seasons. For this, it reaches out to a specialized LLM trained in meteorology and tourism.

The specialized LLM provides Robo with detailed insights into the weather patterns at the Grand Canyon throughout the year. It explains that the best time to visit is typically during the spring or fall when the weather is mild, and the crowds are smaller.

Now, Robo has the information it needs to answer your question accurately. It combines the general knowledge from its main LLM with the specialized insights from the meteorology-trained LLM to provide you with a comprehensive response.

These elements combined form what we now call an LLM Agent, which integrates both a general-purpose LLM and specialized LLMs to provide comprehensive responses to user queries.

Components of LLM Agents

Tools ‍

In the context of LLM (Large Language Model) agents, tools refer to external resources, services, or APIs (Application Programming Interfaces) that the agent can utilize to perform specific tasks or enhance its capabilities. These tools serve as supplementary components that extend the functionality of the LLM agent beyond its inherent language generation capabilities. 

Tools could also include databases, knowledge bases, and external models. 

As an illustration, agents can employ a RAG pipeline for producing contextually relevant responses, a code interpreter for addressing programming challenges, an API for conducting internet searches, or even utilize straightforward API services such as those for weather updates or instant messaging applications.

Agent Core

It refers to the foundational component built around the LLM model itself. At the heart/core of the LLM agent lies another LLM for example GPT ( generative pre-trained transformer ).

The core of the LLM agent is also the place where we define the goals of the agent, the tools used and relevant memory. It also defines the persona of the agent by leveraging carefully crafted prompts and instructions that guide its responses and behavior. 

These prompts are designed to encode the identity, expertise, behaviors, and objectives of the agent, effectively shaping its persona and defining its role in interactions with users.

Planning

Let’s again use our robo example. In the context of our robot assisting with trip planning, "planning" refers to the systematic process by which the robot analyzes user inquiries, gathers relevant information, and strategizes its actions to provide optimal recommendations or solutions.

Planning with Feedback

• In this scenario, the LLM agent engages in continuous planning based on feedback received during interactions. For instance, if the user asks the robot about the best time to visit the Grand Canyon, and the response provided by the LLM agent includes outdated information or doesn't fully address the user's query, the user might provide feedback or ask follow-up questions.
• Upon receiving feedback, the LLM agent re-evaluates its response and plans its next action accordingly. It may refine its search criteria, access more reliable sources of information, or adjust its communication strategy to better meet the user's needs.
• Planning with feedback involves an iterative process where the LLM agent learns from each interaction, continuously improving its planning and decision-making abilities over time.

Planning without Feedback

• In this scenario, the LLM agent operates without immediate feedback from the user. For example, if the user asks the robot to provide a list of recommended activities near the Grand Canyon, the LLM agent must plan its response based solely on the initial query and available data sources.
• Without feedback, the LLM agent relies on pre-defined strategies, heuristics, and its internal knowledge base to generate a response. It may consider factors such as popular tourist attractions, weather conditions, and user preferences inferred from past interactions.
• Planning without feedback requires the LLM agent to anticipate user needs and preferences, making informed decisions based on the context of the query and available information.

Memory

In the framework of an LLM agent, memory modules play a crucial role in facilitating contextual understanding and retention of information over time. These memory components typically encompass both short-term and long-term memory systems.

Short Term Memory

Serves as a dynamic repository of the agent's current actions and thoughts, akin to its "train of thought," as it endeavors to respond to a user's query in real-time. It allows the agent to maintain a contextual understanding of the ongoing interaction, enabling seamless and coherent communication.

Long Term Memory

Acts as a comprehensive logbook, chronicling the agent's interactions with users over an extended period, spanning weeks or even months. It captures the history of conversations, preserving valuable context and insights gleaned from past exchanges. This repository of accumulated knowledge enhances the agent's ability to provide personalized and informed responses, drawing upon past experiences to enrich its interactions with users.

Hybrid Memory

It combines the advantages of both STM and LTM to enhance the agent's cognitive abilities. STM ensures that the agent can quickly access and manipulate recent data, maintaining context within a conversation or task. LTM expands the agent's knowledge base by storing past interactions, learned patterns, and domain-specific information, enabling it to provide more informed responses and make better decisions over time.

‍

Types of LLM agents and use cases

Conversational Agents

These agents are designed to engage in natural language conversations with users, providing information, answering questions, and assisting with various tasks. They often rely on large language models to understand and generate human-like responses.

Task Oriented Agents

These agents are focused on performing specific tasks or completing predefined objectives. They interact with users to understand their needs and then execute actions to fulfill those needs. Examples include virtual assistants and task automation tools.

Creative Agents

These agents are capable of generating original and creative content such as artwork, music, or writing. They may use LLMs to understand human preferences and artistic styles, enabling them to produce content that resonates with audiences.

Collaborative Agents

Collaborative Agents**: Collaborative agents work alongside humans to accomplish shared goals or tasks. They facilitate communication, coordination, and cooperation between team members or between humans and machines. LLMs may support collaborative agents by assisting in decision-making, generating reports, or providing insights.

Examples of LLM Agents

• VisualGPT - VisualGPT links ChatGPT with a series of Visual Foundation Models, facilitating the exchange of images during conversations.
• Lindy AI - Your AI personal assistant.
• CensusGPT - Allows users to inquire about topics related to census data.
• Hearth AI - Agentic Relationship Management.
• RCI Agent for MiniWoB++ - Language Models can Solve Computer Tasks
• Babyagi - An AI-powered task management system.
• ChemCrow - AI agent focused on chemistry tasks, utilizing large language models to plan and execute processes like organic synthesis and drug discovery autonomously.
• Blind Judgement - AI agent that simulates decision-making processes akin to real-world judges. It employs several language models to predict decisions of actual Supreme Court cases with better-than-random accuracy.

What makes these Agents Autonomous?

Let’s again consider our helpful robot assistant “Robo” who can answer your questions and do tasks for you, like fetching your slippers or telling you the weather.

But Robo isn't perfect. Sometimes it gives the wrong answer or forgets what you asked. To help Robo get better, there's another robot called Supervisor. Supervisor's job is to check Robo's answers and give it feedback.

Here's how it works: You ask Robo a question, like "What's the weather today?" Robo gives an answer, but before you see it, Supervisor quickly checks it. If Robo's answer is good, Supervisor lets it go. But if Robo's answer is wrong or unclear, Supervisor steps in and gives Robo a hint or correction. Then, Robo tries again with the new information.

This process repeats over and over, with Supervisor guiding Robo to improve its answers each time. Eventually, Robo gets really good at answering questions and doing tasks on its own, without Supervisor's help. That's when Robo becomes truly autonomous - it can think and act on its own, just like a human, thanks to the continuous guidance and feedback from Supervisor.

Essentially, autonomy arises from the interaction between agents within a system that prompts each other. Autonomous capabilities develop through consistent guidance from a dedicated supervisor agent, offering direction, corrections, and progressively more demanding tasks. Continuous prompting fosters the development of reasoning, effectiveness, and self-driven decision-making.

Implementation and Evaluation of LLM Agents

Implementing LLM agents involves several steps to train, deploy, and optimize the system for specific tasks. Here's a general overview of the implementation process:

• DATA COLLECTION : Gather a diverse and relevant dataset that aligns with the tasks the LLM agent will perform. This dataset should cover a wide range of topics and scenarios to ensure the model's robustness.

• PREPROCESSING DATA : Clean and preprocess the collected data by removing noise, formatting inconsistencies, and irrelevant information. Tokenize the text data and prepare it for training.

• TRAINING AND LANGUAGE MODEL : Use machine learning techniques, particularly natural language processing (NLP) methods, to train the LLM on the preprocessed dataset. Train the model using deep learning architectures such as transformers, recurrent neural networks (RNNs), or convolutional neural networks (CNNs).

• FINE-TUNING: Fine-tune the pre-trained language model to adapt it to the specific tasks or domains relevant to the LLM agent. Fine-tuning involves retraining the model on task-specific data while retaining the knowledge gained during pre-training.

• INTEGRATION OF COMPONENTS: Integrate the core LLM with other components such as memory modules, planning modules, and tool APIs. Design the architecture to facilitate communication and interaction between these components effectively.

• DEPLOYMENT: Deploy the LLM agent in a production environment or integrate it into the desired platform or application. Provide APIs or interfaces for communication with the agent and ensure seamless integration with existing systems.

• LEARNING AND IMPROVEMENT : Continuously update and retrain the LLM agent with new data to improve its performance and relevance over time. Monitor the agent's interactions and gather feedback to identify areas for optimization and enhancement.

• EVALUATION : Evaluating LLM agents is a very challenging task. Here are some of the ways to evaluate a LLM agent:

• EVALUATION PROTOCOLS: Standardized evaluation protocols delineate the methodologies for deploying metrics and conducting assessments. These protocols span diverse scenarios, including real-world simulations, social assessments, multi-task evaluations, and software testing, ensuring a comprehensive evaluation framework.
• BENCHMARKS : Numerous benchmarks have been devised to assess the performance of LLM agents. Some of them include : ALFWorld, IGLU , Tachikuma , AgentBench , SocKET , AgentSims , ToolBench , WebShop , Mobile-Env , WebArena , GentBench , RocoBench , EmotionBench , PEB , ClemBench , and E2E.
• ASSESSMENT BY HUMANS: Human assessment offers valuable insights in LLM evaluation across key dimensions such as sincerity, usefulness, engagement, and impartiality.
• METRICS :  Evaluation metrics play a crucial role in assessing the performance of Large Language Models (LLMs). One notable library for LLM evaluation is the OpenAI Eval library. For example, HellaSwag evaluates the model's common sense reasoning by analyzing its ability to complete sentences with plausible endings.  TruthfulQA focuses on measuring the accuracy and truthfulness of the model's responses, ensuring reliable outputs. Additionally, the Multilingual Multitask Learning Benchmark (MMLU) evaluates the model's multitasking proficiency across different languages. These metrics, along with others provided by the Eval library, contribute to a comprehensive assessment of LLM performance.

Figure 3. "Framework of EduChat." Adapted from “EduChat: A Large-Scale Language Model-based Chatbot System for Intelligent Education,” by Yuhao Dan et al., 2023. arXiv:2308.02773 [cs.CL], https://doi.org/10.48550/arXiv.2308.02773.

Challenges

• SCALABILITY AND RESOURCE INTENSIVENESS: Training and deploying LLMs at scale require significant computational resources, including high-performance computing infrastructure and large datasets. Scaling LLMs while optimizing resource utilization and minimizing environmental impact poses a challenge.

• MEMORY AND CONTEXT MANAGEMENT : Managing memory and context effectively is crucial for LLM agents to maintain coherent conversations and provide relevant responses over extended interactions. Handling long-term memory, short-term memory, and contextual information in a dynamic manner presents a significant challenge.

• BIAS AND FAIRNESS IN RESPONSES : Similar to LLMs, LLM agents are susceptible to biases present in the training data, leading to biased or unfair responses. Mitigating biases and ensuring fairness in LLM agent-generated content is essential for ethical and equitable interactions.

• INTERPRETABILITY AND TRANSPARENCY : LLM agent responses should be interpretable and transparent to users, allowing them to understand how the agent arrived at a particular answer or decision. Enhancing the interpretability and transparency of LLM agents' reasoning processes is critical for building trust and fostering user acceptance.

• SECURITY AND PRIVACY CONCERNS : LLM agents may encounter security and privacy challenges, including unauthorized access to sensitive information shared during conversations and susceptibility to adversarial attacks. Implementing robust security measures and privacy-preserving techniques to safeguard user data and interactions is paramount for LLM agent deployment.

‍