Models take your lead

In response to a post by @steipete.

It can be tempting to see this and think you automatically get better results if you simply ask “is this the best possible fix?”

But it all depends on if the answering agent has better context than the first.

If not, it’s almost guaranteed to doubt the fix.

But the irony is if you then ask it to be sure, it will usually then doubt the doubting.

The models simply take your lead. If you’re skeptical, they’ll confirm it. If you push back, they’ll find the justification that supports your reasoning.

What they ARE good at is uncovering more related evidence to your skepticism in the process.

Use that new evidence plus your judgment to fine tune the result.