# Tamil 1k Tweets For Binary Sentiment Analysis

To find a labeled data for Tamil NLP task is a difficult task. Some papers talk about Tamil Neural Translation, but the article doesn't release code. If you're working part-time or possess an interest in Tamil NLP, you have a tough time finding data.

When I was looking for labeled data for simple sentiment analysis, I couldn't find any. It's understandable because there is no one working on it. So I decided to build my dataset. Twitter seemed a perfect place with lots of data. I scrapped data using [Twint Python library](https://github.com/twintproject/twint).

After spending a whole weekend annotating the data as "Happy/Sad" for 1000 tweets, I'm releasing the data in the public domain. You can find the data in [Kaggle](https://www.kaggle.com/kracekumar/tamil-binary-classification-1k-tweets-labels-v1). The dataset has two columns, `tweet and sentiment.` The `tweet` column contains Tamil text, and `sentiment` column includes relevant `sentiment`.

```
 $head -10 tamil_binary_sentiment_1k_tweets_v1.csv
tweet,sentiment
உன்னைத்தொட்டால் உன்னுள்ளத்தை நொருக்கமாட்டியோ!! என்னைப் போல பெண்ணைப்ப் பார்த்து மயங்க மாட்டியோ!!  #RaOne #chammakChallo #tamilLyrics,Happy
"நதியா நதியா நயில் நதியா
…
இடை தான் கொடியா
கொடி மேல் கனியா
#RDBurnam #HindMusic #TamilLyrics",Happy
"உறக்கம் விற்று கனவுகள் வாங்கலையா?! #TamilLyrics RT @JanuShath: கனவுகள் விற்றுக் கவிதைகள் வாங்குவதும், கவிதைகள் விற்றுக் காதலை வாங்குவதுமாய்.",Sad
மீண்டும் உன்னை காணும் மனமே ... வேண்டும் எனக்கே மனமே மனமே !!! #TamilLyrics,Sad
உயிரை தொலைத்தேன் அது உன்னில் தானோ ... இது நான் காணும் கனவோ நிஜமோ...அன்பே உயிரை தொடுவேன் உன்னை தாலாட்டுதே பார்வைகள் ! #TamilLyrics,Sad
```

The dataset includes 1011 tweets. If you do sentiment analysis on the dataset, consider uploading the kernel to Kaggle. If you're using it in your research work, mention the DOI.

```
Kracekumar, “Tamil Binary Classification 1K tweets Labels V1.” Kaggle, doi: 10.34740/KAGGLE/DSV/1226691.
```

You can download the data from [GitHub](https://github.com/kracekumar/tamil-dataset) as well.

### Points to remember
- The sentiment is labeled based on the tweet and not on the multi-media or hyperlink attachment in the tweet.
- While creating the dataset, I have not looked at the image attached or the user handle for adult content. There may be NSFW attachment.
- The attached link or URL in the tweet may or may not exist.
- The content of the tweet may contain English sentences, words, emojis, etc.


### Labeling process

It took the entire weekend to label the tweets. Google Sheets was the annotation tool. Even though there are only a thousand tweets, I had to read two to three thousand tweets to assign the label. Roughly, the whole process took 17 hours(~1 tweet per minute). It's even hard to read some hundred tweets in a stretch and label. Fun news, I was targeting to label 10K tweets minimum :-) I don't know whether I'll do more labeling any time soon. There are close to 4 lakh tweets in the DB :-)

Happy NLP!

### Important Links in the blog post

- Kaggle Dataset - https://www.kaggle.com/kracekumar/tamil-binary-classification-1k-tweets-labels-v1
- Twint - https://github.com/twintproject/twint
- Github Repo - https://github.com/kracekumar/tamil-dataset