Support one-shot and few-shot learning; ie add option to include messages in test cases
X
Xenon Alpaca
First of all- LOVE langtail.
Right now, when running tests, it only includes the system prompt. This means anyone using one-shot or few-shot learning cannot effectively use the entire testing module.
We just need a simple checkbox in the test to include the message history along with the prompt.
Great work!! Thank you and hope to be a long term user.
C
Coral Salmon
Xenon Alpaca - thanks for your feedback! In Langtail today, you can write a message that will be passed as a user message to the prompt when the test is run. The prompt itself can contain as many system messages and user messages as you like. On the left side of the playground, just click "Add Template", add the message, then save. Then the tests will run the entire prompt.
I hope that helps! If I'm not quite understanding what you're saying, please feel free to respond here or reach out at hi@langtail.com :)
- Ryan from Langtail