Unable to Generate Summary in Bullet Points using Langchain

Question

I am currently working with a chat agent from the Langchain project. My goal is to generate a summary of a video content in bullet points, but I've run into an issue. The agent is capable of summarizing the content, but it doesn't format the summary in bullet points as I'm instructing it to.

Here's the agent initialization code I've used:

agent = initialize_agent(
    agent=AgentType.CHAT_CONVERSATIONAL_REACT_DESCRIPTION,
    tools=tools,
    llm=llm,
    verbose=True,
    max_iterations=3,
    early_stopping_method='generate',
    memory=memory,
)

Then, I'm feeding the following prompt to the agent:

prompt = "summarize in bullets: https://www.youtube.com/watch?v=n2Fluyr3lbc"

The agent provides a summary but doesn't format the output into bullet points as expected. Instead, it simply generates the content in paragraph form.

Can anyone provide any guidance on why the agent might not be following the 'bullet point' instruction in the prompt? Is there a specific way I should be formulating my prompt or is there a setting in the agent initialization I might be missing?

Any help or guidance is appreciated.

What I Tried and Expected:

I have initialized the Langchain agent with the appropriate settings and then passed it a prompt to summarize a YouTube video link. My expectation was that the agent, given the prompt "summarize in bullets: [YouTube Link]", would provide a bullet-point summary of the content in the video.

I chose this approach as I believed the agent should be capable of interpreting and executing this instruction based on the understanding and processing capabilities the Langchain models typically exhibit. I expected the output to be a concise list of key points extracted from the video, each point presented as a separate bullet point.

What Actually Resulted:

In reality, the agent did provide a summary of the video content, indicating that it correctly processed the video and carried out the 'summarize' part of the instruction. However, it did not format the summary in bullet points as I instructed. Instead, the summary was provided in the form of a single, unstructured paragraph.

Therefore, while the agent demonstrated its ability to summarize the content, it did not adhere to the formatting instruction. The challenge, in this case, is figuring out why the 'bullet point' aspect of the instruction was not carried out.

Answer 1

I don think you can do it directly with an agent as of now. what you can do is you can give the output to the llm ask in bullet point form:

you got the output

output = agent_executor.run("....")

feed this into llm

from langchain.chat_models import ChatOpenAI
from langchain.schema import (AIMessage, HumanMessage,SystemMessage)

messages=[SystemMessage(content='you are an expert in summarizing documents'),
         HumanMessage(content=f"Please provide a short summary of the following text in bullet points:\n output :{output}")]
llm=ChatOpenAI(temperature=0,model_name='gpt-3.5-turbo')

summary=llm(messages)
print(summary)