parent:
linguistics
Related:
On its own, the model won't often ask for clarification of ambiguous topics. It'll just got with the one with the highest logprob (or other depending on temperature). See labels for additional disambiguation.
Intro
Maybe this should be something that isn't prompt now that I think about it, gotta figure out the category for this.
Anyway, there were 2 president Bushes and the model knows the difference based on context.
I am reading a book about President Bush and the 1988 campaign.
q: What is president bush's opponent?
a:[EOP] Michael Dukakis
q: Bush was what number president?
a:[EOP] 41I am reading a book about President Bush and the Global War on Terror.
q: Who were Bush's opponents for the presidential election?
a:[EOP] John Kerry and John Edwards.
q: what about in his first one?
a:[EOP] Al Gore and Joe Lieberman.It can keep them apart even when processing both.
I am reading a book about President Bush and the Global War on Terror.
q: what was Bush's full name?
a:[EOP] George Walker Bush
q: What about the Bush from 1988?
a:[EOP] George Herbert Walker Bush
q: what's the relationship between the two?
a:[EOP] father and son
q: which one is this one?
a:[EOP] George Walker BushTrying the WiC benchmark
Score:
28 July 2020 - 66%. Messed up some math at the end so submitted suboptimal results (see code at https://gist.github.com/brockmanmatt/7265297f21634693868c2aad9d2c5919, forgot to change the in the copy/paste evaluation score to the actual score of 262. Non-zero chance when I just resubmitted I flipped the Ts and Fs :P)