Context
This post is about the project Minh Hoang and I worked on for the annual OpenBB Hackathon.
Our internal hackathon is a way to learn new tools or test projects that we would otherwise postpone because they fall outside the company’s priorities. My goal was to improve my frontend development skills, so I started looking for a project to do so.
This was in December, and Michael Struwig, our Head of AI, had been making cool demos of OpenBB Copilot, launched a few months later.
I had been working on our Excel Add-in, so it sounded interesting to combine the two and create a chat interface from scratch to interact with our LLM, which specializes in Finance.
Bringing the OpenBB Copilot to Excel
During the project, I learned more about using React to build a chat UI and handling streaming responses to avoid having the user look at a spinning wheel, waiting for a long response.
When you use applications like ChatGPT, all of this seems so seamless that you don’t even think about how it is implemented.
This is a simple ‘hello’ message being sent to OpenBB Copilot:
OpenBB Copilot lets you select widgets and ask questions about the data they display.
Excel does not have widgets, but it does have cells. So I decided to create a similar feature where you can ask questions with a range of cells as context.
Later, Minh asked me if we could go beyond this use case and ask questions about using our Add-in custom functions. This was a whole different challenge.
Previously, we just sent simple data back and forth. Now, we would need to do that, plus implement a backend capable of answering questions about how to use another application that GPT doesn’t know.
This is how it looked in the end:
Nowadays, it is a widespread practice to wrap the OpenAI API with the OpenAI Python API library , embed a few documents into a vector database, and provide bits of this data so the LLM can answer questions about the documents.
I followed the usual path and implemented a Python backend using fastapi with a single endpoint /query where questions are posted.
Then, langchain and faiss-cpu took care of the embeddings. I won’t go into details here because there are a lot of resources explaining how to do this. These links provide helpful information on where to start:
The most interesting part of the implementation is how you can control the response to reliably render the output in a website or add-in, as we wanted.
The format in which the model responds can be quite unpredictable, complicating the display of a nice visualization for the user.
Simply put, common data APIs always return your JSON in the same format; it does not change every time you call it as GPT might.
To overcome this and control the response, I used instructor, which lets us specify the output structure as a pydantic base model. This model has additional validations to check if the function returned by the LLM exists or was just a hallucination.
from pydantic import BaseModel, Field, root_validator
FUNCTIONS = {
"/equity/compare/peers": ...
"/equity/fundamental/income": ...,
...
}
class Parameter(BaseModel):
name: str
type_: str
value: str
class Function(BaseModel):
name: str = Field(description="Name of the function.")
parameters: List[Parameter] = Field(description="Parameters of the function.")
@root_validator(pre=True)
def validate_function(cls, values):
# Validate if 'name' in FUNCTIONS...
class FunctionMessage(BaseModel):
reasoning: str = Field(description="Short reasoning, keep it to 1 sentence.")
function: Optional[Function] = Field(description="Function to be executed.")With this setup, the response from OpenAI, whatever that be, would fit the pre-defined structure. From here, it was easy to transform the response into text plus a markdown snippet.
{
"content": "To retrieve the income statement for Apple, the default symbol AAPL can be used without specifying additional parameters.\n\n\`\`\`excel\n=OBB.EQUITY.FUNDAMENTAL.INCOME(\"AAPL\")\n\`\`\`\n"
}Then, on the website/add-in, we can render with npm: react-markdown.
And there you go…
Conclusion
Overall, this project revealed a learning experience not just about frontend but also about the interaction with LLMs and how to control their output to use downstream.
Whether OpenBB actually takes this idea forward and decides to bring the Copilot to Excel for all users, only time will tell.
You can check the backend implementation on GitHub - montezdesousa/openbb-xl-copilot.