Site icon Use AI the right way

Create our own Copilot style chat interface with LangChain

Rav

Copilot style interface becomes more and more mainstream as they have gained popularity with tools like Claude, ChatGPT and others that are fully using it. It is a very powerful feature but at the same time, it is not that complex.

So let’s build our own Chat Copilot style from scratch using LangChain, OpenAI and Streamlit so that you can understand how your favorite tool works!

As usual, we will strive for simplicity so let’s get to it!

What is a Copilot style interface?

A Copilot style interface is actually just an interface where you have on one side a panel that you are actively modifying and on the other side you have a helper panel but with far more intelligence.

The best example is Claude’s Artifacts feature (which is actually 1 year old now).

You have the chat on the left and the Artifacts panel on the right which can show different types of inputs, from code to markdown to mermaid diagram to rendered web app code (as you can see in the image) to better answer the query.

What is important is that, using LLMs, you can have an intelligent system that can give you and show you what you need at the right time:

This is what a copilot is, someone or something that help you when you do something and watch out for you.

Why is it actually a Simple to Implement Feature?

As said before, the copilot feature is really powerful.
But at the same time, it is not this complicated to create because all the hard work is done on the LLM side:

These 3 type of outputs are actually very powerful because they are covering a lot of use cases:

All these 3 formats are widely used and so libraries to integrate them exist in the most common coding language (python, javascript, …) and framework (react, angular, …).

The Prompt: the Core of the Autopilot

The core of the autopilot is the LLM and so the prompt. This is where you will explain to the LLM what type of visualisation you want and how you want it generated. There is a lot of formats possible but here’s one possible format:

{
    "explanation": "",
    "viz_content: "",
    "viz_type": "markdown | code | mermaid"
}

To craft such prompt you will need to handle multiple points:

The good point is that if you get a fairly powerful LLM, it will in most case generate correct output.

Here I showed only 3 types of visualisation because those are the easiest to do but you can use other formats. The only limit is what the LLM can generate in a robust way and what renderer exists for this format.

Copilot-Style chat the easy way

Now that we have seen the theory, let’s create our own copilot chat!

For that, we are going to use Python, LangChain, OpenAI and Streamlit. You can find all the code here.

Python Dependencies

First let’s install the required libraries:

pipenv install langchain langchain-openai streamlit streamlit_mermaid dotenv

Here we are using pipenv to add all our required libraries. Most of them are common, the only special ones are doten and streamlit_mermaid which are respectively a library to load env vars from env files and a streamlit component to render mermaid.

Create a file .env containing your OpenAI api key at the same level as your code:

OPENAI_API_KEY=

Copilot Prompt

Now that this is done, let’s look at the code. Let’s begin with the prompt:

prompt = ChatPromptTemplate.from_messages(
        [
            SystemMessage(
                content="""You are a helpful programming assistant.
            Your will give a detailed answer.
            If needed for the clarity of the answer, you can add a visualisation as a mermaid diagram, code, markdown or text.
            
            For mermaid diagrams, follow these rules:
            - Use '-->' for arrows (not -->)
            - Start with graph direction (TD, LR, etc.)
            - Each node definition on new line with \\n
            - Escape special characters
            
            Example mermaid format:
            {
                "explanation": "Here's a flowchart example",
                "viz_content": "graph TD\\n    A[Start] ---> B[Process]\\n    B ---> C[End]",
                "viz_type": "mermaid"
            }
            
            For markdown content, follow these rules:
            - Use valid markdown syntax
            - Use \\n for newlines
            - Escape special characters
            
            Example markdown format:
            {
                "explanation": "Here's a markdown example",
                "viz_content": "# Title\\n\\nSome **bold** text and a [link](https://example.com)",
                "viz_type": "markdown"
            }
            
            For code content, follow these rules:
            - Use valid code syntax
            - Use \\n for newlines
            - Escape special characters
            
            Example code format:
            {
                "explanation": "Here's a code example",
                "viz_content": "def hello_world():\\n    print('Hello, world!')",
                "viz_type": "code"
            }
            
            Rules for all responses:
            - Must be valid JSON
            - Use \\n for newlines
            - Use \\" for quotes
            - No raw backticks
            """
            ),
            HumanMessage(
                content=f"""
            Previous conversation:
            {conversation}
            
            Current user message: {current_msg}"""
            ),
        ]
    )

Here’s how this prompt is made:

LLM Output Format

Now let’s take care of the output format of the LLM answer. As mentionned before, we need 3 keys, “explanation”, “viz_content” and “viz_type”. 

class LLMOutput(BaseModel):
    explanation: str = Field(..., description="The text explanation for the user")
    viz_content: str = Field(..., description="The content to be visualized")
    viz_type: Literal["code", "markdown", "mermaid", "text"] = Field(...)

...

    model = ChatOpenAI(model="gpt-4", temperature=0)
    parser = JsonOutputParser(pydantic_object=LLMOutput)

    chain = prompt | model | parser
    response = chain.invoke({"messages": messages})

Let’s see how this code works:

Generate Different Type of Visualizations

Now that we have the correct output, we can see how it is used. This is using Streamlit code but the logic behind is the same and can be used with any framework like React or Angular.

if last_msg["viz_content"]:
    if last_msg["viz_type"] == "markdown":
        st.markdown(last_msg["viz_content"])
    elif last_msg["viz_type"] == "mermaid":
        stmd.st_mermaid(last_msg["viz_content"])
    elif last_msg["viz_type"] == "code":
        st.code(last_msg["viz_content"])
    else:
        st.write(last_msg["viz_content"])

Here’s what is happening in this code snippet:

Pretty simple right !? And this can be used pretty easily in any application.

Copilot chat execution

Now that we have the code, let’s run it!

pipenv run streamlit run app.py

And now let’s check out out beautiful app:

As you can see, we first asked how Metaclasses worked in python and the LLM decided to give us some Python code along the explanation. Then we asked for a diagram and it generated the mermaid diagram. Pretty neat right ?!

The best part is that it looks like what you can see on ChatGPT or Claude (though not as polished) so this showcase how easy it is to add these kind of features in your own applications.

Agents or No Agents: That Is the Question

As you can guess, what you saw before is is fully implementable using LLM agents. You can actually check out my earlier article for an example. So why not use the same approach?

Therefore the decision to agents or not depends of the use case and what you want. The best way is to begin with the most simple implementation and see how far this will take you!

Conclusion

In this post, we saw how copilot style chat works by using the LLM to decide and generate visualisation accordingly to the user query. By specifying information like the the type of visualisation in the LLM output, we can show many different type of visualisation using markdown and mermaid format.The implementation itself is not difficult which make the integration of such feature simple (the hard part is on the UX and frontend part).

The copilot style add a lot of value to an application, as you can give to the user an assistant that will help him better use you application, whether it is a RAG chat or a coding copilot. One of the best usage of LLMs is to help users be more productive and the copilot style interface really hits the mark for that.

Exit mobile version