balkian/sitc
1
0
Fork 0
mirror of https://github.com/gsi-upm/sitc synced 2025-01-08 20:11:27 +00:00
Code Issues Releases Wiki Activity

Compare commits

base: balkian:75f08ea170340ede65eb14736048e8157919be09
Branches Tags
balkian:master
balkian:dveni-patch-2
balkian:dveni-patch-1
..
compare: balkian:e70689072f697a541c4d196f9933864db95c23b2
Branches Tags
balkian:master
balkian:dveni-patch-2
balkian:dveni-patch-1

No commits in common. "75f08ea170340ede65eb14736048e8157919be09" and "e70689072f697a541c4d196f9933864db95c23b2" have entirely different histories.
75f08ea170 ... e70689072f

1 changed files with 1 additions and 1 deletions
Show Stats Expand all files Collapse all files

2
nlp/4_1_Lexical_Processing.ipynb
View File

@ -326,7 +326,7 @@
"def preprocess(words, type='doc'):\n",
" if (type == 'tweet'):\n",
" tknzr = TweetTokenizer(strip_handles=True, reduce_len=True)\n",
" tokens = tknzr.tokenize(words)\n",
" tokens = tknzr.tokenize(tweet)\n",
" else:\n",
" tokens = nltk.word_tokenize(words.lower())\n",
" porter = nltk.PorterStemmer()\n",