Recently, I came across Scikit-LLM library which provides a Sklearn compatible wrapper around OpenAI REST API. This allows the users of the widely popular Sklearn library to use ChatGPT to perform tasks such as text classification, text vectorization, and text summarization using their familiar Sklearn steps. In this blog post, I am going to illustrate the use of Scikit-LLM library to perform sentiment classification.
The first step obviously is to install the Scikrit-LLM package.
pip install scikit-llmWe also need to obtain OpenAI API keys for use in our work. You can get the keys by going to the OpenAI platform and sign with your account. After that, you need to click on “Create New Secret Key”. The API also needs your organization ID which you can getting by clicking on Settings in your account page. Your organization name would be “Personal” if you created your account as a personal account. Needless to say that you should safely store your secret key. Use the secret key and the organizational ID to configure Scikit-LLM as shown below.
from skllm.config import SKLLMConfig
SKLLMConfig.set_openai_key("<YOUR_KEY>")
SKLLMConfig.set_openai_org("<YOUR_ORGANISATION>")Sentiment Classification
I decided to perform sentiment classification using the ZeroshotGPTClassifier from the package. A zero shot classifier uses a trained model. All you have to do it is to provide labels that are descriptive. Although Scikit-LLM library comes with a demo dataset for classification, I decided to use a small dataset of 40 reviews of Japanese restaurants in London, UK. In my experimentation, I used two OpenAI models: gpt-3.5-turbo (default ChatGPT) and text-davinci-003 (designed for instructions-following task). All it took to perform sentiment classification was the following code. including reading the reviews.
import pandas as pd
import numpy as np
from skllm import ZeroShotGPTClassifier
df = pd.read_excel("/Downloads/restaurant-reviews.xlsx")
XX = df.to_numpy()
X = XX[:,1]
clf = ZeroShotGPTClassifier(openai_model = "gpt-3.5-turbo")
clf.fit(None, ['positive', 'negative','neutral'])
labels = clf.predict(X)
labels[0:5] # Check the labels assigned to first five reviews['positive', 'neutral', 'neutral', 'positive', 'positive']
Let us now look at the first five reviews to see how well the sentiments match.
| 1 | For a chain, Shoryu is quite good and I went twice during my week in London because I love ramen that much. The ramen is definitely better than some that Ive had before which is a good thing. I tried multiple types and they all had unique tastes to them that made them worthwhile. |
| 2 | Some appetizers were tried as well and they had varying degrees of success. The pork belly buns are good but certainly there are better. The brussel spout tempura…uhhhhh not good. Like wow those were pretty bad. |
| 3 | We ordered a Shoryu Ganso Tonkotsu, their signature dish with extra meat, and also some fried gyoza. We finished our ramen and also our beer, and waited long for the gyoza. So we had to ask, and it seems they had forgotten about this order. They apologized and made the food right away when we pointed that out, so it was not the end of the world. But normally the gyoza should have come before the ramen as gyoza is considered more as an appertizer. |
| 4 | We ordered the gyoza and pork and prawn buns to share. Then I ordered the curry ramen in the others section. |
| 5 | The sides came first which I was glad about being staring. The gyoza was freshly cooked and mouth-wateringly good. The pieces of meat inside were chunky and they just went down a treat! |
You can see that the classifier did a fairly decent job except for the review #4 that I would place in the neutral category. Next, I decided to repeat the classification task using the text-davinci-003 model.This time the following labels were received for the above five reviews.
['negative', 'neutral', 'positive', 'positive', 'negative']
The results this time are not good which is not much surprising as the text-davinci-003 model is trained for instruction following and not text classification.
As we can see from the example presented here, Scikit-LLM keeps us in our familiar Sk-learn environment to create ChatGPT based tasks for natural language processing. If you would like to experiment with the reviews that I used; you can download them from the following link.
From Data to Decisions 