In the ever-evolving landscape of technology, the synergy between serverless architectures, NoSQL databases, and Large Language Models (LLMs) is opening new frontiers in application development. This article delves into the integration of these cutting-edge technologies using Google’s PaLM 2 and the LangChain framework, demonstrated through the development of a ‘shopper’ chat-bot.

In this entry I will describe an example I am preparing to showcase the possibility of using ReAct (Reasoning & Acting) paradigm of Large Language Model and incorporate serverless apps into our GenAI-powered applications

So here it is – a shopper architecture. Fairly straight forward. We are going to utilize Firestore as our NoSQL database, 3 Cloud functions that can accept API calls to list or modify content of the database, and 3 python-developed tools that will be utilized by LangChain Agent, powered by PaLM 2 Large Language model. But I’m getting ahead of myself. Let’s start step by step.

Serverless Architecture: A Modern Approach

Serverless computing has revolutionized the way we deploy and manage web applications. By leveraging Cloud Functions, developers can focus on writing code without worrying about the underlying infrastructure. Key benefits include scalability, cost-efficiency, and reduced operational overhead.

NoSQL Database Integration: Embracing Flexibility

NoSQL databases offer unmatched flexibility, making them ideal for handling unstructured or semi-structured data. In this project, we’ll integrate a NoSQL database with our serverless application to store and manage data generated by the chat-bot. 

Why NoSQL? I want my shopper to be able to add items to the list with two fields:

• name – what do I need to buy
• quantity – how many (or how much) I need of that thing. And here it become a bit more tricky. There is a different quantity for cucumbers, flour, meat and spices, isn’t it? Well, NoSQL with it’s flexibility for semi-structured data will be a perfect fit here!

 

Large Language Models: Enhancing User Interaction

PaLM 2, as part of our system, serves as the brain behind our chat-bot. With its state-of-the-art natural language processing capabilities, PaLM 2 can interpret user queries, maintain context over a conversation, and generate responses that are coherent, relevant, and surprisingly human-like. This allows our ‘shopper’ bot to not just respond to user requests but also to anticipate needs based on previous interactions.

LangChain: Orchestrating the Flow

LangChain is the glue that binds our NoSQL database and PaLM 2 in a serverless environment. It acts as the middleware that interprets the outputs from PaLM 2 and translates them into database operations, such as adding items to a list or retrieving user history. This modular approach means we can update or swap out components without disrupting the bot’s overall functionality. An the ‘magic’ behind it is under Agent, which enables us to write custom tools (in simplified way: custom python code) that Large Language Model can use to get to the result it is asked to do.

Putting It All Together: A Step-By-Step Implementation

Set up the Firestore

gcloud alpha firestore databases create \
--database=DATABASE_ID \
--location=LOCATION \
--type=DATABASE_TYPE \
[--delete-protection]

If you need more info on this step, be sure to check out official google cloud documentation.

Later on create a collection, in my case I created “grocery_lists”

 

Set up the serverless environment and deploy Cloud Functions that will interact with the NoSQL database.

OK for that step let me just showed you example code I used, treat it as example, and definitely “proof of concept” application, not a production-ready app that can hold millions of connections at the time 🙂

import firebase_admin
from firebase_admin import credentials
from firebase_admin import firestore

# Initialize Firebase Admin SDK
cred = credentials.ApplicationDefault()
firebase_admin.initialize_app(cred)
db = firestore.client()

def add_item(request):
    request_json = request.get_json(silent=True)
    request_args = request.args

    if request_json and 'name' in request_json:
        name = request_json['name']
        quantity = request_json.get('quantity', 1)  # default quantity is 1
        
        # Add item to Firestore
        doc_ref = db.collection('grocery_lists').document()
        doc_ref.set({
            'name': name,
            'quantity': quantity
        })
        return f'Item {name}, {quantity} added successfully.'

    else:
        return 'Missing "name" in the request', 400

import firebase_admin
from firebase_admin import credentials
from firebase_admin import firestore

# Initialize Firebase Admin SDK
cred = credentials.ApplicationDefault()
firebase_admin.initialize_app(cred)
db = firestore.client()

def list_items(request):
    items = db.collection('grocery_lists').stream()
    item_list = [{item.id: item.to_dict()} for item in items]
    return str(item_list)

import firebase_admin
from firebase_admin import credentials
from firebase_admin import firestore

# Initialize Firebase Admin SDK
cred = credentials.ApplicationDefault()
firebase_admin.initialize_app(cred)
db = firestore.client()

def clear_list(request):
    items = db.collection('grocery_lists').stream()
    for item in items:
        db.collection('grocery_lists').document(item.id).delete()
    return 'All items cleared.'

Create a LangChain Agent

LangChain Agents integrates custom-built tools, such as Python scripts, with the capabilities of Large Language Models (LLMs) to execute complex tasks requested by end-users. These agents are designed to seamlessly blend the analytical and processing power of Python with the linguistic and contextual understanding of LLMs. This synergy allows LangChain Agents to understand and respond to a wide range of user requests, from data analysis and visualization to more sophisticated tasks like real-time decision making and problem-solving. By harnessing the strengths of both custom tools and LLMs, LangChain Agents can deliver more accurate, context-aware, and efficient solutions, revolutionizing the way we interact with AI technology.

To continue with that part, make sure you’ve got installed Python packages:

• langchain
• google-cloud-aiplatform

# import proper libraries
import vertexai
from langchain.chat_models import ChatVertexAI
from langchain.tools import tool
from langchain.agents import initialize_agent, AgentType
from langchain.chains.conversation.memory import ConversationBufferWindowMemory

# create llm object that calls PaLM 2 model chat-bison

llm_bison = ChatVertexAI(temperature=0,max_output_tokens=1024)

# create proper tools - my_list, add_to_list and clear_list that send proper message to cloud function

# import proper libraries
import vertexai
from langchain.chat_models import ChatVertexAI
from langchain.tools import tool

# create llm object that calls PaLM 2 model chat-bison

llm_bison = ChatVertexAI(temperature=0,max_output_tokens=1024)

# create proper tools - my_list, add_to_list and clear_list that send proper message to cloud function

import json

@tool()
def my_list(query: str = "") -> str:
    ''' Returns the full content of user shopping list '''


    url = "https://<YOUR-CLOUD-FUNTION-URL>.cloudfunctions.net/list_items"
    creds = service_account.IDTokenCredentials.from_service_account_file(
        keyFilePath,
        target_audience=url
    )
    authed_session = AuthorizedSession(creds)
    resp = authed_session.get(url)

    headers = {
        "Authorization": f"Bearer {creds.token}",
        "Content-Type": "application/json"
    }
    data = {}
    response = requests.post(url, json=data, headers=headers)

    fixed_response = response.text.replace("'",'"').replace("None", "null")
    product_descriptions = []
    try:
        items = json.loads(fixed_response)
        for item in items:
          for _, value in item.items():
              name = value.get('name', 'Unknown product')
              quantity = value.get('quantity', 0)
              product_descriptions.append(f"product: {name} - quantity: {quantity}")
    except ValueError as e:
        return f"Error parsing JSON: {e}\nResponse text: {response.text}"

    result_string = ', '.join(product_descriptions)

    return result_string

@tool()
def clear_list(query: str = "") -> str:
    ''' Use this tool to clear the list'''


    url = "https://<YOUR-CLOUD-FUNTION-URL>.cloudfunctions.net/clear_list"
    creds = service_account.IDTokenCredentials.from_service_account_file(
        keyFilePath,
        target_audience=url
    )
    authed_session = AuthorizedSession(creds)
    resp = authed_session.get(url)

    headers = {
        "Authorization": f"Bearer {creds.token}",
        "Content-Type": "application/json"
    }
    data = {}
    response = requests.post(url, json=data, headers=headers)
    result_string = response.text

    return result_string

@tool
def add_to_list(name) -> str:
    '''
    Use this tool to add an item to the shopping list.

    Args:
        name (str): Name of the item to be added to the list.
                    Quantity must be defined after a comma, example:
                    "fresh cucumber, 1 piece"
                    "flour, 1 kg"
                    "onions, 3 pieces"
                    To add multiple items, use the ";" character, for example:
                    "onions, 3 pieces; cucumbers, 1 piece"

    Returns:
        str: Confirmation whether the item was correctly added to the list
    '''

    url = "https://<YOUR-CLOUD-FUNTION-URL>.cloudfunctions.net/add_item"
    creds = service_account.IDTokenCredentials.from_service_account_file(
        keyFilePath,
        target_audience=url
    )
    authed_session = AuthorizedSession(creds)
    resp = authed_session.get(url)

    headers = {
        "Authorization": f"Bearer {creds.token}",
        "Content-Type": "application/json"
    }

    items = name.split(";")  # Split the input string into individual items

    response_texts = []  # List to store responses for each item

    for item in items:
        item = item.strip()  # Remove leading/trailing whitespace
        if "," in item:
            name, quantity = [x.strip() for x in item.split(",", 1)]
        else:
            name = item
            quantity = "1"

        data = {
            "name": name,
            "quantity": quantity
        }

        response = requests.post(url, json=data, headers=headers)  # Make the POST request for each item
        print(f"Added: {response.text}")
        response_texts.append(response.text)  # Append the response text to the list

    return "; ".join(response_texts)  # Join all responses into a single string

# Now let's initialize those tools!

shopper_tools = [my_list,clear_list,add_to_list]

OK, once we have our LLM and tools defined we can continue to combine those into Agent. Since our Agent is a chat-bot it also needs to have access to a memory to store chat history, that we will also provide using LangChain capabilities.

# memory

shopper_memory = ConversationBufferWindowMemory(
        memory_key='chat_history',
        k=5,
        return_messages=True
)

# agent

shopper = initialize_agent(
    agent=AgentType.STRUCTURED_CHAT_ZERO_SHOT_REACT_DESCRIPTION,
    tools=shopper_tools,
    llm=llm_bison,
    verbose=True,
    max_iterations=1,
    early_stopping_method='generate',
    memory=shopper_memory
)

And with that we have our shopper agent, we can ask queries like:

result = shopper.run("Can you remove everything from my list please?")
print(textwrap.fill(result, width=100))

> Entering new AgentExecutor chain...
 Action:
```
{
  "action": "clear_list",
  "action_input": ""
}
```

Observation: All items cleared.
Thought: ```
{
  "action": "Final Answer",
  "action_input": "Your shopping list is now empty."
}
```

> Finished chain.
Your shopping list is now empty.

Or if we want to add some items and present the list:

result = shopper.run("Can add to my list five potatos?")
print("\n\n RESULT \n\n")
print(result)

result = shopper.run("Can list my items?")
print("\n\n RESULT \n\n")
print(result)

> Entering new AgentExecutor chain...
 Action:
```
{
  "action": "add_to_list",
  "action_input": "potatoes, 5"
}
```
Added: Item potatoes, 5 added successfully.

Observation: Item potatoes, 5 added successfully.
Thought:Potatoes, 5 added successfully.


> Finished chain.


 RESULT 


Potatoes, 5 added successfully.



> Entering new AgentExecutor chain...
 Action:
```
{
  "action": "my_list",
  "action_input": ""
}
```

Observation: product: potatoes - quantity: 5
Thought: ```
{
  "action": "Final Answer",
  "action_input": "Here is your shopping list: \nproduct: potatoes - quantity: 5"
}
```



> Finished chain.


 RESULT 


Here is your shopping list: 

product: potatoes - quantity: 5

Notice how agent knows exactly (thanks to our docstrings) which function to use in which case.

But there is more, we can utilize LLM knowledge not only to execute a current action, but also to figure out what we want. For instance, what would happen if we ask to add ingredients for pizza, without specyfying what we mean by that? Say no more!

result = shopper.run("Can you add to my list integriends for pizza?")

> Entering new AgentExecutor chain...
 Action:
```
{
  "action": "add_to_list",
  "action_input": "pizza dough, 1 piece; tomato sauce, 1 jar; mozzarella cheese, 1 pack; pepperoni, 1 pack"
}
```
Added: Item pizza dough, 1 piece added successfully.
Added: Item tomato sauce, 1 jar added successfully.
Added: Item mozzarella cheese, 1 pack added successfully.
Added: Item pepperoni, 1 pack added successfully.

Observation: Item pizza dough, 1 piece added successfully.; Item tomato sauce, 1 jar added successfully.; Item mozzarella cheese, 1 pack added successfully.; Item pepperoni, 1 pack added successfully.
Thought: ```
{
  "action": "Final Answer",
  "action_input": "Items for pizza added to yoru list"
}
```

Pretty awesome, isn’t it ?

Conclusion

In conclusion, the integration of serverless architectures with NoSQL databases and Large Language Models like Google’s PaLM 2 and the LangChain framework signifies a leap forward in developing innovative and efficient applications. Through the step-by-step walkthrough provided, we’ve explored how a ‘shopper’ chat-bot can be constructed within this modern technological landscape. This example not only demonstrates the practical application of these technologies but also illustrates the seamless interconnectivity and potential for scalability.

And I cannot stress that enough – presented code is just a “proof of concept” and in no way it should be used as production-ready “ctrl+c / ctrl+v” type of a solution. 🙂 But feel free to copy it, modify to your needs and play with it. Learn by doing! That’s the best way!