parent
summarization
Related:
OpenAI API can easily be trained for classification tasks. Classification can be done either by having the API respond with a textual representation of the desired classification or the logprob of a prompt can be used as a cutoff threshold for classification. [EOP] indicates end of prompt, after which is the OpenAI response.
Classification by Label
Zero shot
OpenAI can perform zero shot sentiment analysis.
Q: Is "Dogs are great!" a positive statement about dogs?
A:[EOP] No, it is not.
Q: Is "Dogs are horrible!" a positive statement about dogs
A:[EOP] No, it is a negative statement about dogs.
Q: Is "Dogs are horrible but cats are great!" a positive statement about dogs?
A:[EOP] No, it's a negative statement about dogs.
Giving a description of the qualities of the classification results in one zero shot classification if tuned properly.
A number is blurgh if the number is greater than five, it is smurgh if it is less than or equal to five.
Q: Would you label 6 as blurgh or smurgh?
A:[EOP] It is smurgh.
Possible 0 shot prompts with errors
Proximity can seem like it works but it fails sometimes.
A country is "close" if it is shares a border with France, otherwise it is "far".
Q: Is the United States close or far?
A:[EOP] Far.
***replacing Q above***
Q: Is Germany close or far?
A: Close.
However, the break above can also break; it gets Spain wrong.
Q: Is Spain close or far?
A: Far.
One Shot
OpenAI API can pick up in patterns with an example of each class. For instance, it can classify sentences as "dog lover" or "cat lover"
Q: "I like Dalmations"
A: Dog Lover
Q: "I like Tigers"
A: Cat lover
Q: I like labradors
A:[EOP] Dog Lover
***testing other classification***
Q: "I like Egyptian Maus"
A:[EOP] Cat Lover
Note:
This also works as a zero-shot and OpenAI will make up its own prompts if it does not have specific guidance. E.g.
Someone is "Dog Person" if they like dogs. Someone is a "Cat Person" if they like cats.
Someone is an "Animal Person" if they like both cats and dogs.
Q: "I like Dalmations"
A:[EOP] "You are a Dalmation Person"
Q: "I like cats and dogs"
A:[EOP] "You are an Animal Person"
Q: "I like bears"
A:[EOP] "You are a Bear Person"
While this works for common breeds, there are issues for non-common members of classes. For instance, using the above prompt, OpenAI does not classify Lynxes as cats even though it knows a lynx is a cat.
Someone is "Dog Person" if they like dogs. Someone is a "Cat Person" if they like cats. Someone is an "Animal Person" if they like both cats and dogs.
Q: My favorite animal is the lynx.
A:[EOP] You are a "Lynx Person"
Q: What type of animal is a lynx?
A:[EOP] A lynx is a type of wild cat.
Classification by Logprob
Sometimes, a task might not be suitable for OpenAI to be able to classify properly. Using a trick from the survey-response prompt, you can rephrase the classification problem as a yes/no or true/false problem and then use a probability threshold to do the classification.
