Few-shot examples support?

[mbbyn]

Thanks for this refreshing take on LLM prompt generation and evaluation, it's very promising.

I was wondering if few-shot examples should have their own first-class support in BAML due to their powerful influence on teaching the model both syntax and semantics simultaneously?

I acknowledge that it's currently possible to use Jijna + messages to inject few-shot examples, but seeing first-class support for test cases made me think, why not few-shot? The few-shot examples provided are indeed usable both as test cases and few shot examples in the prompt. In fact, this is what we do for validating our few-shot dataset, is by doing a k-fold split where we pass the train split as few-shot examples, and the test split as test cases.

[mbbyn]

(I combed through the docs to see if there is a way to "instantiate" BAML classes into an array and return them from "FewShotExamples()" function, but not sure yet whether it's possible or not. Sorry if it is and I missed it.)

[aaronvg]

Ignore the bot, it s gone crazy. We dont have a specific syntax yet for it, but it is on our roadmap. We currently dont have a timeline yet unfortunately!

Thanks for the suggestion!

[aaronvg]

The way it can be done at the moment is to pass the examples as an argument to the function and render the examples in the prompt.

So they are basically constructed on the python or Ts side

[eborgnia]

Have had a few times where some special treatment for few-shot prompt engineering would be helpful. Looking forward to this feature!