summary

Prototype

Overview

The context of use that inspired this prototype was seeking an expert system for quick answers on the go. As such, the interface had to be mobile. I settled on a discord like chat interface for the UI which would support replying to past messages which is useful for asking followup questions.

I played around with prompt design quite a bit to get this to feel good. I settled for having GPT3 mimic a librarian with a neutral point of view to get answers that felt like they came from Wikipedia. To reduce hallucinations, there is a check to make sure the question has a reasonable answer. For example, “what books did Leonardo DiCaprio write?” would return something like “According to our records, Leonardo DiCaprio has not authored any books.”

When replying to prior messages, I prepend to the prompt context information extracted from the reply thread. This way, if you ask a question like “what were the major events of the war?” and the previous message in the reply chain includes a mention to the American Civil War, the language model will know which war you are referring to.

The last piece of prompt work is the explore functionality. This is perhaps the most interesting part. It generates a set of questions that you can choose to reply with. In talking with beta testers, this has been the most compelling part of the interface which is reflected in the recorded logs. These questions point to interesting adjacent possibilities to explore. Like an expert replying to your initial question and then saying “if you’re interested in that, then you’ll also like this.”

In the future, a retrieval augmented language model would be a better expert system allowing it to pull information from up to date sources, understand it and then construct an answer with references to the sources.

Reflection

In reflection, I think what makes the explore functionality feel good is because it mimics a good story. In an interview with producer of This American Life, Ira Glass, he mentions how the core of a good story is the anecdote.

A good story is constantly generating new questions and answering them. The explore functionality essentially creates a choose your own adventure story giving you interesting directions to take the conversation next.