twitter - Train corpus of Tweets for Sentiment Analysis, using NLTK for Python -

-

i'm trying train own corpora sentiment analysis, using nltk python. have 2 text files: 1 has 25k positive tweets, separated per line, other 1 25k negative tweets.

i use stackoverflow article, method 2

when run code create corpora:

import string itertools import chain  nltk.corpus import stopwords nltk.probability import freqdist nltk.classify import naivebayesclassifier nbc nltk.corpus import categorizedplaintextcorpusreader import nltk  mydir = 'c:\users\gerbuiker\desktop\sentiment analyse\my_movie_reviews'  mr = categorizedplaintextcorpusreader(mydir, r'(?!\.).*\.txt', cat_pattern=r'(neg|pos)/.*', encoding='ascii') stop = stopwords.words('english') documents = [([w w in mr.words(i) if w.lower() not in stop , w.lower() not in string.punctuation], i.split('/')[0]) in mr.fileids()]  word_features = freqdist(chain(*[i i,j in documents])) word_features = word_features.keys()[:100]  numtrain = int(len(documents) * 90 / 100) train_set = [({i:(i in tokens) in word_features}, tag) tokens,tag in documents[:numtrain]] test_set = [({i:(i in tokens) in word_features}, tag) tokens,tag  in documents[numtrain:]]  classifier = nbc.train(train_set) print nltk.classify.accuracy(classifier, test_set) classifier.show_most_informative_features(5)

i receive error message:

c:\users\gerbuiker\anaconda\python.exe "c:/users/gerbuiker/desktop/sentiment analyse/corpus_pos_neg/createcorpus.py" traceback (most recent call last):   file "c:/users/gerbuiker/desktop/sentiment analyse/corpus_pos_neg/createcorpus.py", line 23, in <module>     documents = [([w w in mr.words(i) if w.lower() not in stop , w.lower() not in string.punctuation], i.split('/')[0]) in mr.fileids()]   file "c:\users\gerbuiker\appdata\roaming\python\python27\site-packages\nltk\corpus\reader\util.py", line 336, in iterate_from     assert self._len not none assertionerror  process finished exit code 1

does know how fix this?

i'm not 100% positive i'm not on windows machine test @ moment, think may catching difference between path slash direction in @alvas original example , adaptation windows.

specifically, use: 'c:\users\gerbuiker\desktop\sentiment analyse\my_movie_reviews' while example uses '/home/alvas/my_movie_reviews'. part fine, attempt re-use cat_pattern regex: r'(neg|pos)/.*' match slash in paths reject 1 in yours.


Comments

Post a Comment

Popular posts from this blog

java - Custom OutputStreamAppender not run: LOGBACK: No context given for <MYAPPENDER> -

-
i wish write custom appender on basis of outputstreamappender . wrote following class package tests; import java.io.ioexception; import java.io.outputstream; import ch.qos.logback.core.outputstreamappender; public class myappender<e> extends outputstreamappender<e> { public myappender() { system.out.println("myappender created"); setoutputstream(new outputstream() { @override public void write(int b) throws ioexception { throw new unsupportedoperationexception(); } }); } } then wrote following runner: package tests; import org.slf4j.loggerfactory; import ch.qos.logback.classic.logger; public class runner { private static final logger log = (logger) loggerfactory.getlogger(runner.class); public static void main(string[] args) { //log.addappender(new myappender<iloggingevent>()); log.info("hello world"); } } finall...
Read more

java - UML - How would you draw a try catch in a sequence diagram? -

-
Image
is there standard way of drawing try catch sequence diagrams? i've made attempt based on this feel end result doesn't feel right. code diagram based around: public static void save () { try { filehandle filehandle = gdx.files.external(file); filehandle.writestring(boolean.tostring(constantshandler.soundenabled)+"\n", false); (int = 0; < 5; i++) { filehandle.writestring(integer.tostring(constantshandler.highscores[i])+"\n", true); } } catch (throwable e) { } } my attempt note: know still need add loop. that's ok. fragments meant show conditional control flow in sequence diagram. superstructures puts limitations on use of fragments (see pp. 467 of ss2.1.1). should use critical region try part above. , option catch . however, keep telling can take quite freedom in using uml. it's language , such changes in life time. long reader ge...
Read more

c++ - No viable overloaded operator for references a map -

-
i trying use map can make tag names reference number. when try , use it, in code error (6 in total every time reference map): src/main.cpp:25:45: error: no viable overloaded operator[] type 'std::map<std::string, std::string>' const char* idcs = node.child_value(tagmap[3]); this code: #include "pugi/pugixml.hpp" #include <iostream> #include <string> #include <map> int main() { pugi::xml_document doca, docb; std::map<std::string, pugi::xml_node> mapa, mapb; std::map<std::string, std::string> tagmap {{"1", "data"}, {"2", "entry"}, {"3", "id"}, {"4", "content"}}; if (!doca.load_file("a.xml") || !docb.load_file("b.xml")) { std::cout << "can't find input files"; return 1; } (auto& node: doca.child(tagmap[1]).children(tagmap[2])) { co...
Read more