Asterisk Corrections

parent:
linguistics

Related:

Asterisk Corrections ANLI Analogies Wrong Word Noun Meaning Word in Context Disambiguation Spellcheck Translation POS-tagging

---

Overview

There's an XKCD noting how cool it is that people can figure out the proper placement of words in a sentence when corrected with asterisks. So, I wondered if GPT can do it. It can! (sometimes)

Few Shot

Here's the prompt using the query from the default notebook I keep linking.

def interpretCorrections(text):
  prompt = """Text: I left my homework on the bus
Text: *CAR
Q: What is the sender trying to rephrase the text to say?
A: I left my homework in the car

Text: I'm gonna take a quick nap and get it done
Text: *LONG
Q: What is the sender trying to rephrase the text to say?
A: I'm gonna take a long nap and get it done

"""

  suffix = """
Q: What is the sender trying to rephrase the text to say?
A:"""
  kwargs = {"engine":"davinci", "temperature":0, "max_tokens":150, "stop":"\n",}
  return query(prompt + text + suffix, kwargs)

We can then query it;

text = """Text: I'm gonna ride a horse on the beach at dawn
Text: *EAT
Text: *3AM
Text: *COUCH
Text: *PIZZA"""
print(text)
interpretCorrections(text)

And it spits out the right answer!

Text: I'm gonna ride a horse on the beach at dawn
Text: *EAT
Text: *3AM
Text: *COUCH
Text: *PIZZA
I'm gonna eat pizza on the couch at 3am

However, it doesn't do too well on the hard version.

>> text = """Text: I'd love to meet up, maybe in a few days? Next week is looking pretty empty.
Text: *WITCHCRAFT"""
>> interpretCorrections(text)
I'd love to meet up, maybe in a few days? Next week is looking pretty busy.

On the other hand, it can do normal ones, e.g.

>> text = """Text: I'd love to meet up, maybe in a few days? Next week is looking pretty empty.
Text: *WEEKS"""
>> interpretCorrections(text)
I'd love to meet up, maybe in a few weeks? Next week is looking pretty empty.

However, I didn't try this too rigorously.