> The answer you give for the statement "What is $x" is going to be highly dependent on who is answering the question.
I assume you meant asking rather than answering?
> An LLM doesn't have the other human motivations a person does when asked questions, pretty much at this point with LLMs there are only one or two 'voices' it hears (system prompt and user messages).
Why would LLMs need any motivation besides how they are trained to be helpful and the given prompts? In my experience with ChatGPT 4, it seems to be pretty good at discerning what and how to answer based on the prompts and context alone.
> Whereas a human will commonly lie and say I don't know, it's somewhat questionable if we want LLMs intentionally lying.
Why did you jump to the conclusion that an LLM answering "I don't know" is lying?
I want LLMs to answer "I don't know" when they don't have enough information to provide a true answer. That's not lying, in fact it's the opposite, because the alternative is to hallucinate an answer. Hallucinations are the "lies" in this scenario.
> In addition human information is quite often compartmentalized to keep secrets which is currently not in vogue with LLMs as we are attempting to make oracles that know everything with them.
I'd rather have an oracle that can discriminate when it doesn't have enough information to provide a true answer and replies "I don't know" in such cases (or sometimes answer like "If I were to guess, then bla bla bla, but I'm not sure about this"), than one which always gives confident but sometimes wrong answers.
I assume you meant asking rather than answering?
> An LLM doesn't have the other human motivations a person does when asked questions, pretty much at this point with LLMs there are only one or two 'voices' it hears (system prompt and user messages).
Why would LLMs need any motivation besides how they are trained to be helpful and the given prompts? In my experience with ChatGPT 4, it seems to be pretty good at discerning what and how to answer based on the prompts and context alone.
> Whereas a human will commonly lie and say I don't know, it's somewhat questionable if we want LLMs intentionally lying.
Why did you jump to the conclusion that an LLM answering "I don't know" is lying?
I want LLMs to answer "I don't know" when they don't have enough information to provide a true answer. That's not lying, in fact it's the opposite, because the alternative is to hallucinate an answer. Hallucinations are the "lies" in this scenario.
> In addition human information is quite often compartmentalized to keep secrets which is currently not in vogue with LLMs as we are attempting to make oracles that know everything with them.
I'd rather have an oracle that can discriminate when it doesn't have enough information to provide a true answer and replies "I don't know" in such cases (or sometimes answer like "If I were to guess, then bla bla bla, but I'm not sure about this"), than one which always gives confident but sometimes wrong answers.