ValueError: Got unknown type S when using GPT-4 with LangChain for Summarization

Question

I am trying to use LangChain with the GPT-4 model for a summarization task. When I use the GPT-3.5-turbo model instead of GPT-4, everything works fine. However, as soon as I switch to GPT-4, I get the following error:

ValueError: Got unknown type S

Here is the relevant part of my code that produces the error:

llm = ChatOpenAI(temperature=0, model_name="gpt-4")
text_splitter = RecursiveCharacterTextSplitter(
    chunk_size=4000, chunk_overlap=0, separators=[" ", ",", "\n"]
)

texts = text_splitter.split_text(readme_content)
docs = [Document(page_content=t) for t in texts]

prompt_template = """template"""
PROMPT = PromptTemplate(template=prompt_template, input_variables=["text"])
chain = load_summarize_chain(llm, chain_type="map_reduce", map_prompt=PROMPT, combine_prompt=PROMPT)
summary = chain.run(docs)

In this code, docs are the chunks created using the RecursiveCharacterTextSplitter class.

The full traceback of the error is as follows:

Traceback (most recent call last):
  File "/Users/maxhager/Projects2023/githubgpt/testing_env/test_single.py", line 136, in <module>
    print(create_tweet(info, readme_content))
  File "/Users/maxhager/Projects2023/githubgpt/testing_env/test_single.py", line 120, in create_tweet
    summary = llm(prompt)
  File "/Users/maxhager/.virtualenvs/githubgpt/lib/python3.10/site-packages/langchain/chat_models/base.py", line 128, in __call__
    return self._generate(messages, stop=stop).generations[0].message
  File "/Users/maxhager/.virtualenvs/githubgpt/lib/python3.10/site-packages/langchain/chat_models/openai.py", line 247, in _generate
    message_dicts, params = self._create_message_dicts(messages, stop)
  File "/Users/maxhager/.virtualenvs/githubgpt/lib/python3.10/site-packages/langchain/chat_models/openai.py", line 277, in _create_message_dicts
    message_dicts = [_convert_message_to_dict(m) for m in messages]
  File "/Users/maxhager/.virtualenvs/githubgpt/lib/python3.10/site-packages/langchain/chat_models/openai.py", line 277, in <listcomp>
    message_dicts = [_convert_message_to_dict(m) for m in messages]
  File "/Users/maxhager/.virtualenvs/githubgpt/lib/python3.10/site-packages/langchain/chat_models/openai.py", line 88, in _convert_message_to_dict
    raise ValueError(f"Got unknown type {message}")
ValueError: Got unknown type S

Has anyone encountered a similar issue when using GPT-4 with LangChain? Any suggestions on how to resolve this error would be greatly appreciated.

Answer 1

I had a similar error and it's fixed by replacing your PromptTemplate with a ChatPromptTemplate since it's a ChatOpenAI model.

I suspect you also switched from llm = OpenAI to llm = ChatOpenAI model at the same time as you switched from GPT3.5 to GPT4.

Try something like:

from langchain.prompts import (
    ChatPromptTemplate,
    HumanMessagePromptTemplate
)

human_message_prompt = HumanMessagePromptTemplate.from_template(prompt_template)

chat_prompt = ChatPromptTemplate.from_messages([human_message_prompt])

See docs at: https://python.langchain.com/en/latest/modules/prompts/chat_prompt_template.html