mysql - How to improve wind data SQL query performance -

-

i'm looking on how optimize (if possible) performance of sql query used reading wind information (see below) changing e.g. database structure, query or else?

i use hosted database store table more 800,000 rows wind information (speed , direction). new data added each minute anemometer. database accessed using php script creates web page plotting data using google's visualization api.

the web page takes approximately 15 seconds load. i've added time measurements in both php , javascript part profile code , find possible areas improvements.

one part hope improve following query takes approximately 4 seconds execute. purpose of query group 15 minutes of wind speed (min/max/mean) , calculate mean value , total min/max during period of measurements.

select  avg(d_mean) group_mean,          max(d_max) group_max,          min(d_min)         group_min,          dir,          from_unixtime(max(dt),'%y-%m-%d %h:%i') group_dt     (     select  @i:=@i+1,              floor(@i/15) group_id,              cast(mean decimal(3,1)) d_mean,              cast(min decimal(3,1)) d_min,              cast(max decimal(3,1)) d_max,              dir,              unix_timestamp(str_to_date(dt, '%y-%m-%d %h:%i')) dt              table, (select @i:=-1) var_init              order id desc ) t  group group_id limit 0, 360  ...  $oresult = mysql_query($ssql);

the table has following structure:

1   id      int(11)     auto_increment 2   mean    varchar(5)  utf8_general_ci 3   max     varchar(5)  utf8_general_ci 4   min     varchar(5)  utf8_general_ci 5   dt      varchar(20) utf8_general_ci    // date , time 6   dir     varchar(5)  utf8_general_ci

the following setup used:

  • database: mariadb, 5.5.42-mariadb-1~wheezy
  • database client version: libmysql - 5.1.66
  • php version: 5.6
  • php extension: mysqli

i agree comments far -- cleanse data put table.

once have done cleansing, let's avoid subquery doing...

select  min(dt) 'start of 15 mins',         format(avg(mean), 1) 'avg wind speed',         ...     table     group floor(unix_timestamp(dt) / 900)     order floor(unix_timestamp(dt) / 900);

i don't understand purpose of limit. i'll guess want few days @ time. that, recommend add (after cleansing) between from , group by.

    dt >= '2015-04-10'       , dt  < '2015-04-10' + interval 7 day

that show 7 days, starting '2015-04-10' morning.

in order handle table of 800k, decidedly need (again, after cleansing):

index(dt)

to cleanse 800k rows, there multiple approaches. suggest creating new table, copy data in, test, , swap over. like...

create table new (     dt datetime,      mean float,     ...     primary key(dt)  -- assuming have 1 row per minute? ) engine=innodb;  insert new (dt, mean, ...)     select str_to_date(...),            mean, -- suspect cast not needed            ...;

write new select , test it.

by new missing newer rows. can either rebuild , hope finish in 1 minute window, or play other game. let know if want there.


Comments

Post a Comment

Popular posts from this blog

c++ - No viable overloaded operator for references a map -

-
i trying use map can make tag names reference number. when try , use it, in code error (6 in total every time reference map): src/main.cpp:25:45: error: no viable overloaded operator[] type 'std::map<std::string, std::string>' const char* idcs = node.child_value(tagmap[3]); this code: #include "pugi/pugixml.hpp" #include <iostream> #include <string> #include <map> int main() { pugi::xml_document doca, docb; std::map<std::string, pugi::xml_node> mapa, mapb; std::map<std::string, std::string> tagmap {{"1", "data"}, {"2", "entry"}, {"3", "id"}, {"4", "content"}}; if (!doca.load_file("a.xml") || !docb.load_file("b.xml")) { std::cout << "can't find input files"; return 1; } (auto& node: doca.child(tagmap[1]).children(tagmap[2])) { co
Read more

java - Custom OutputStreamAppender not run: LOGBACK: No context given for <MYAPPENDER> -

-
i wish write custom appender on basis of outputstreamappender . wrote following class package tests; import java.io.ioexception; import java.io.outputstream; import ch.qos.logback.core.outputstreamappender; public class myappender<e> extends outputstreamappender<e> { public myappender() { system.out.println("myappender created"); setoutputstream(new outputstream() { @override public void write(int b) throws ioexception { throw new unsupportedoperationexception(); } }); } } then wrote following runner: package tests; import org.slf4j.loggerfactory; import ch.qos.logback.classic.logger; public class runner { private static final logger log = (logger) loggerfactory.getlogger(runner.class); public static void main(string[] args) { //log.addappender(new myappender<iloggingevent>()); log.info("hello world"); } } finall
Read more

java - Cannot secure connection using TLS -

-
i'm trying establish secure socket connection between java client applet (built jdk 1.7.0_75-b13) , vc++ server application. as test vehicle, used vc++ client/server sample found in msdn forums, modified use schannel , able establish socket using cipher suite tls_rsa_with_aes_128_cbc_sha. works of tls 1.0/1.1/1.2. when try opening socket java applet same server application, connection rejected server reporting following: tls 1.0 acceptsecuritycontext failed: 0x80090327 tls 1.1 acceptsecuritycontext failed: 0x80090331 tls 1.2 acceptsecuritycontext failed: 0x80090331 this java code used create socket: debugprint("setting secure connection"); sslsocketfactory sslsocketfactory = (sslsocketfactory) sslsocketfactory.getdefault(); sslsocket sslsocket = (sslsocket) sslsocketfactory.createsocket("127.0.0.1", socketnumber); debugprint("starting handshake"); sslsocket.settcpnodelay(true); sslsocket.setsolinger(false, 0); sslsocket.s
Read more