Sign in Subscribe

Getting Started with LangChain, ChatGPT API, and Python

Getting Started with LangChain, ChatGPT API, and Python

LangChain is a framework for building LLM powered apps. If you're coming from webdev-land, a useful analogy might be to think of LangChain as the [Django | Laravel | Rails] of the LLM ecosystem: while it is possible to to build a web app using vanilla [Python | PHP | Ruby],  you'll spend a lot of time reinventing wheels instead of working on the features that make your app unique. Similarly, once you get past "getting started" with the ChatGPT API, you'll hit some predictable speed bumps. For example:

  • You want to bring your own data but bump into max token limits.
  • You hit timeouts and rate limit errors and need to implement logic for retries and exponential backoffs.
  • You want to swap out models and/or LLM providers but it requires rearchitecting your app.

These are just a few of the areas where LangChain's abstractions and features can rapidly accelerate the building of new features in your LLM powered app.

This tutorial looks at two scripts:

  1. A bare-bones ChatGPT implementation on the command line. This is "Hello World" for working with vanilla ChatGPT API in Python.
  2. Reproducing that same app "Hello World" app using LangChain, ChatGPT API, and Python.

When we're done, we'll have a jumping off point to start building LLM powered apps with the LangChain framework.

Hello World with Vanilla ChatGPT API

Let's first look at "Hello World" with ChatGPT and Python, which we previously discussed in this tutorial:

import openai

messages = []

while True: 
    message = input()
    messages.append({"role": "user", "content": message})
    response = openai.ChatCompletion.create(
        model="gpt-3.5-turbo",
        messages=messages)
    reply = response["choices"][0]["message"]["content"]
    print("\n" + reply + "\n")
    messages.append({"role": "assistant", "content": reply})

For this to work you'll need to pip install openai and have your OpenAI API key set as an environment variable.

These 13 lines of code give us ChatGPT on the command line. We store our chat history in a list of messages. A message takes the form of a dictionary with two parameters: role and content. Role indicates who said the thing, and can be either system (silent instructions to the model), user, or assistant. content is the body of the message. We pass the list of messages into the chatCompletion endpoint at OpenAI, along with a parameter indicating which model we want to use. We parse the reply, print it, and and append it to messages.

When considering what's possible with ChatGPT out of the box, this might the most bang-for-the-buck Python code I've ever written. If you want more detail on what that code's doing, check out this tutorial on getting started with ChatGPT and Python.

Hello World with LangChain and ChatGPT API

Now let's look at a implementation of this same minimal app using LangChain. 

from langchain.chat_models import ChatOpenAI
from langchain.schema import HumanMessage

chat = ChatOpenAI(model="gpt-3.5-turbo", temperature=0)

messages = []

while True:
    message = input("> ")
    usr_msg = HumanMessage(content=message)
    messages.append(usr_msg)
    ai_msg = chat(messages)
    print(ai_msg.content)
    messages.append(ai_msg)

Again, you'll need to pip install langchain for this to work, and set your OpenAI API Key as an environment variable for this to work.

Here's what's happening, step by step.

Import Langchain modules

from langchain.chat_models import ChatOpenAI
from langchain.schema import HumanMessage

The first two lines import the minimal required classes from the LangChain library. ChatOpenAI is responsible for creating the chatbot instance. HumanMessage is a class used to represent messages from the user.

A fundamental concept to understand when working with LangChain and ChatGPT is that instead of passing around text, LangChain uses Message objects. LangChain also has AIMessage and SystemMessage classes, which we do not need to import for this minimal app – but the existence of these three classes marks a significant departure from our need to manually construct a dictionary to store the message in our vanilla example.

Create the chatbot instance

chat = ChatOpenAI(model="gpt-3.5-turbo", temperature=0)

We create an instance of the ChatOpenAI class with two parameters: model and temperature. The model parameter specifies which AI model to use, in this case, gpt-3.5-turbo.

temperature controls the randomness in ChatGPT's replies. When working directly with the ChatGPT API, the temperature parameter is optional, but LangChain requires it. You can set temperature between 0 and 2. The closer to 2, the more variance you'll see in ChatGPT's replies. Here we set temperature to 0, which makes the output deterministic and focused, meaning it will always generate the same response for the same input.

Initialize the messages list

messages = []

Like our previous example, we store our chat history in a list of messages.

Start the chat loop

while True:

An infinite loop (while True:) keeps the conversation going until the user terminates the script manually.

Gather user input

    message = input("> ")

We prompt the user to input a message, which is stored in the message variable.

Create a HumanMessage object

    usr_msg = HumanMessage(content=message)

We initialize a HumanMessage object, with the content parameter set to the user's input.

Append the user's message to the chat history

    messages.append(usr_msg)

We append usr_msg to the messages list, updating the chat history.

Generate the AI's response

    ai_msg = chat(messages)

This line, compared to the vanilla API call, best demonstrates the simplicity you get from LangChain's abstractions.

The chat instance is called with the messages list as its argument. The AI processes the conversation history and returns a response in the form of an AIMessage, which we store in the ai_msg variable.

Print ChatGPT's response

    print(ai_msg.content)

We print ChatGPT's response. Again, notice the simplicity here versus processing the full dictionary returned by the vanilla API call. You can access the content of a langchain Message object by using the dot operator.

Append the AI Message to the chat history

    messages.append(ai_msg)

Finally, the AI's response is appended to the messages list, updating the conversation

After this, our loop repeats, gathering input from the user, getting replies from ChatGPT, and appending both to the chat history, until the user kills the script.

What's Next

That's it! When you run this script you'll get a carbon-copy experience of the original Hello World script, and now you have the foundation in place to start building robust, feature rich LLM-powered apps using LangChain. In future posts we'll explore LangChain features such as:

  • Using Prompt Templates
  • Adding Memory to your LLM apps
  • Building LLM Chains to have models interact with each other
  • Building Agents which use LLMs in conjunction with other tools, such as data retrieval or code execution