balkian/sitc
1
0
Fork 0
mirror of https://github.com/gsi-upm/sitc synced 2025-11-03 17:08:17 +00:00
Code Issues Releases Wiki Activity

Compare commits

base: balkian:dveni-patch-1
Branches Tags
balkian:master balkian:dveni-patch-2 balkian:dveni-patch-1
..
compare: balkian:dveni-patch-2
Branches Tags
balkian:master balkian:dveni-patch-2 balkian:dveni-patch-1

2 Commits
dveni-patc ... dveni-patc

Author SHA1 Message Date
Dani Vera
19ea5dff09 Update 4_1_Lexical_Processing.ipynb 2019-11-26 15:14:40 +01:00
Carlos A. Iglesias
e70689072f Merge pull request #4 from gsi-upm/dveni-patch-1 
Update 3_3_Data_Munging_with_Pandas.ipynb
 2019-09-19 10:46:19 +02:00
1 changed files with 1 additions and 1 deletions
Expand all files Collapse all files

2
nlp/4_1_Lexical_Processing.ipynb
View File

@@ -326,7 +326,7 @@
"def preprocess(words, type='doc'):\n", "def preprocess(words, type='doc'):\n",
" if (type == 'tweet'):\n", " if (type == 'tweet'):\n",
" tknzr = TweetTokenizer(strip_handles=True, reduce_len=True)\n", " tknzr = TweetTokenizer(strip_handles=True, reduce_len=True)\n",
" tokens = tknzr.tokenize(tweet)\n", " tokens = tknzr.tokenize(words)\n",
" else:\n", " else:\n",
" tokens = nltk.word_tokenize(words.lower())\n", " tokens = nltk.word_tokenize(words.lower())\n",
" porter = nltk.PorterStemmer()\n", " porter = nltk.PorterStemmer()\n",