To search, Click below search items. |
| All Published Papers Search Service |
|
|
|
Title |
Lexical normalization of roman Urdu text |
|
Author |
Zareen Sharf, Dr Saif Ur Rahman |
| Citation |
Vol. 17 No. 12 pp. 213-221 |
|
Abstract |
Social media text usually comprises of short length messages, which typically contain a high percentage of abbreviations, typos, phonetic substitutions and other informal ways of writing. The inconsistent manner of text representation poses challenges in performing Natural Language Processing and other forms of analysis on the available data. Therefore, to overcome these issues the text requires to be normalized for effective processing and analysis. In this work, we have performed a comparative study of how social media text in different languages like Chinese, Arabic, Japanese, Polish, Bangla, Dutch and Roman Urdu has been normalized to achieve consistency. We have discussed in detail the normalization methods proposed, their success rate and their shortcomings. Based on our analysis we have also proposed a model for achieving lexical normalization of text in Roman Urdu. |
|
Keywords |
Normalization, Standardization, Transliteration, Roman Urdu. |
|
URL |
