Classification

parent
summarization

Related:


OpenAI API can easily be trained for classification tasks. Classification can be done either by having the API respond with a textual representation of the desired classification or the logprob of a prompt can be used as a cutoff threshold for classification. [EOP] indicates end of prompt, after which is the OpenAI response.

Classification by Label

Zero shot

OpenAI can perform zero shot sentiment analysis.

Q: Is "Dogs are great!" a  positive statement about dogs? 
A:[EOP] No, it is not.
Q: Is "Dogs are horrible!" a positive statement about dogs
A:[EOP] No, it is a negative statement about dogs.
Q: Is "Dogs are horrible but cats are great!" a positive statement about dogs?
A:[EOP] No, it's a negative statement about dogs.

Giving a description of the qualities of the classification results in one zero shot classification if tuned properly.

A number is blurgh if the number is greater than five, it is smurgh if it is less than or equal to five.
Q: Would you label 6 as blurgh or smurgh?
A:[EOP] It is smurgh.

Possible 0 shot prompts with errors

Proximity can seem like it works but it fails sometimes.

A country is "close" if it is shares a border with France, otherwise it is "far".
Q: Is the United States close or far?
A:[EOP] Far.

***replacing Q above***
Q: Is Germany close or far?
A: Close.

However, the break above can also break; it gets Spain wrong.

Q: Is Spain close or far?
A: Far.

One Shot

OpenAI API can pick up in patterns with an example of each class. For instance, it can classify sentences as "dog lover" or "cat lover"

Q: "I like Dalmations"
A: Dog Lover

Q: "I like Tigers"
A: Cat lover

Q: I like labradors
A:[EOP] Dog Lover

***testing other classification***
Q: "I like Egyptian Maus"
A:[EOP] Cat Lover

Note:

This also works as a zero-shot and OpenAI will make up its own prompts if it does not have specific guidance. E.g.

Someone is "Dog Person" if they like dogs. Someone is a "Cat Person" if they like cats.
Someone is an "Animal Person" if they like both cats and dogs.

Q: "I like Dalmations"
A:[EOP] "You are a Dalmation Person"

Q: "I like cats and dogs"
A:[EOP] "You are an Animal Person"

Q: "I like bears"
A:[EOP] "You are a Bear Person"

While this works for common breeds, there are issues for non-common members of classes. For instance, using the above prompt, OpenAI does not classify Lynxes as cats even though it knows a lynx is a cat.

Someone is "Dog Person" if they like dogs. Someone is a "Cat Person" if they like cats. Someone is an "Animal Person" if they like both cats and dogs.
Q: My favorite animal is the lynx.
A:[EOP] You are a "Lynx Person"

Q: What type of animal is a lynx?
A:[EOP] A lynx is a type of wild cat.

Classification by Logprob

Sometimes, a task might not be suitable for OpenAI to be able to classify properly. Using a trick from the survey-response prompt, you can rephrase the classification problem as a yes/no or true/false problem and then use a probability threshold to do the classification.

One Shot

Few Shot

Unless otherwise stated, the content of this page is licensed under Creative Commons Attribution-ShareAlike 3.0 License