node.js - Mongodb, can i trigger secondary replication only at the given time or manually? -

-

i'm not mongodb expert, i'm little unsure server setup now.

i have single instance running mongo3.0.2 wiredtiger, accepting both read , write ops. collects logs client, write load decent. once day want process logs , calculate metrics using aggregation framework, data set process logs last month , calculation takes 5-6 hours. i'm thinking splitting write , read avoid locks on collections (server continues write logs while i'm reading, newly written logs may match queries, can skip them, because don't need 100% accuracy).

in other words, want make setup secondary read, replication not performing continuously, starts in configured time or better triggered before read operations started.

i'm making processing node.js 1 option see here export data created in period [yesterday, today] , import read instance myself , make calculations after import done. looking on replica set , master/slave replication possible setups didn't how config achieve described scenario. maybe wrong , miss here? there other options achieve this?

your idea of using replica-set flawed several reasons.

first, replica-set replicates whole mongod instance. can't enable individual collections, , not specific documents of collection.

second, deactivating replication , enabling before start report generation not idea either. when enable replication, new slave not up-to-date. take while until has processed changes since last contact master. there no way tell how long take (you can check how far secondary behind primary using rs.status() , comparing secondaries optimedate lastheartbeat date).

but when want perform data-mining on subset of documents selected timespan, there solution.

transfer documents want analyze new collection. can aggregation pipeline consisting of $match matches documents last month followed $out. out-operator specifies results of aggregation not sent application/shell, instead written new collection (which automatically emptied before happens). can perform reporting on new collection without locking actual one. has advantage operating on smaller collection, queries faster, can't use indexes. also, data won't change between aggregations, reports won't have inconsistencies between them due data changing between them.

when need second server report generation, can still use replication , perform aggregation on secondary. however, recommend build proper replica-set (consisting of primary, secondary , arbiter) , leave replication active @ times. not make sure data isn't outdated when generate reports, gives important benefit of automatic failover should primary go down reason.


Comments

Post a Comment

Popular posts from this blog

java - Custom OutputStreamAppender not run: LOGBACK: No context given for <MYAPPENDER> -

-
i wish write custom appender on basis of outputstreamappender . wrote following class package tests; import java.io.ioexception; import java.io.outputstream; import ch.qos.logback.core.outputstreamappender; public class myappender<e> extends outputstreamappender<e> { public myappender() { system.out.println("myappender created"); setoutputstream(new outputstream() { @override public void write(int b) throws ioexception { throw new unsupportedoperationexception(); } }); } } then wrote following runner: package tests; import org.slf4j.loggerfactory; import ch.qos.logback.classic.logger; public class runner { private static final logger log = (logger) loggerfactory.getlogger(runner.class); public static void main(string[] args) { //log.addappender(new myappender<iloggingevent>()); log.info("hello world"); } } finall...
Read more

java - UML - How would you draw a try catch in a sequence diagram? -

-
Image
is there standard way of drawing try catch sequence diagrams? i've made attempt based on this feel end result doesn't feel right. code diagram based around: public static void save () { try { filehandle filehandle = gdx.files.external(file); filehandle.writestring(boolean.tostring(constantshandler.soundenabled)+"\n", false); (int = 0; < 5; i++) { filehandle.writestring(integer.tostring(constantshandler.highscores[i])+"\n", true); } } catch (throwable e) { } } my attempt note: know still need add loop. that's ok. fragments meant show conditional control flow in sequence diagram. superstructures puts limitations on use of fragments (see pp. 467 of ss2.1.1). should use critical region try part above. , option catch . however, keep telling can take quite freedom in using uml. it's language , such changes in life time. long reader ge...
Read more

c++ - No viable overloaded operator for references a map -

-
i trying use map can make tag names reference number. when try , use it, in code error (6 in total every time reference map): src/main.cpp:25:45: error: no viable overloaded operator[] type 'std::map<std::string, std::string>' const char* idcs = node.child_value(tagmap[3]); this code: #include "pugi/pugixml.hpp" #include <iostream> #include <string> #include <map> int main() { pugi::xml_document doca, docb; std::map<std::string, pugi::xml_node> mapa, mapb; std::map<std::string, std::string> tagmap {{"1", "data"}, {"2", "entry"}, {"3", "id"}, {"4", "content"}}; if (!doca.load_file("a.xml") || !docb.load_file("b.xml")) { std::cout << "can't find input files"; return 1; } (auto& node: doca.child(tagmap[1]).children(tagmap[2])) { co...
Read more