/[Sack]/trunk
This is repository of my old source code which isn't updated any more. Go to git.rot13.org for current projects!
ViewVC logotype

Log of /trunk

View Directory Listing Directory Listing


Sticky Revision:

Revision 278 - Directory Listing
Modified Mon Jun 21 13:47:05 2010 UTC (13 years, 9 months ago) by dpavlin
open details in new window


Revision 277 - Directory Listing
Modified Fri May 7 09:15:36 2010 UTC (13 years, 10 months ago) by dpavlin
display currnet and total shards while bulk_load


Revision 276 - Directory Listing
Modified Fri Apr 30 14:34:49 2010 UTC (13 years, 11 months ago) by dpavlin
pass sack_port as second argument to client

bin/split.sh can again start two nodes on localhost

Revision 275 - Directory Listing
Modified Wed Apr 28 20:20:35 2010 UTC (13 years, 11 months ago) by dpavlin
details about node memory usage


Revision 274 - Directory Listing
Modified Wed Apr 28 19:09:54 2010 UTC (13 years, 11 months ago) by dpavlin
collect /proc statistics from nodes


Revision 273 - Directory Listing
Modified Wed Apr 28 08:08:47 2010 UTC (13 years, 11 months ago) by dpavlin
push favicon.ico so we can distinguinsh Sack tabs in browser


Revision 272 - Directory Listing
Modified Wed Apr 28 07:54:48 2010 UTC (13 years, 11 months ago) by dpavlin
convert -colors 16 sack-onion-logo.svg sack.ico


Revision 271 - Directory Listing
Modified Sun Feb 21 15:14:02 2010 UTC (14 years, 1 month ago) by dpavlin
add IP addresses to node's title


Revision 270 - Directory Listing
Modified Mon Feb 15 09:40:12 2010 UTC (14 years, 1 month ago) by dpavlin
sort nodes in display


Revision 269 - Directory Listing
Modified Thu Feb 4 22:38:07 2010 UTC (14 years, 1 month ago) by dpavlin
count all object across nodes

Revision 268 - Directory Listing
Modified Thu Feb 4 22:36:05 2010 UTC (14 years, 1 month ago) by dpavlin
bulk_load all nodes in parallel (20-40Mb/s on my network)


Revision 267 - Directory Listing
Modified Thu Feb 4 20:31:34 2010 UTC (14 years, 1 month ago) by dpavlin
save data and image as hash of all keys to work-around filename limits


Revision 266 - Directory Listing
Modified Thu Feb 4 19:08:29 2010 UTC (14 years, 1 month ago) by dpavlin
- move all pending node status into $info->{pending} and ignore forked for refresh
- do cpio of new version *after* fork for massive speedup


Revision 265 - Directory Listing
Modified Thu Feb 4 18:15:59 2010 UTC (14 years, 1 month ago) by dpavlin
add pending class to nodes only if there is anything pending


Revision 264 - Directory Listing
Modified Mon Feb 1 22:03:02 2010 UTC (14 years, 1 month ago) by dpavlin
move Results to top of page (plays well with refresh)


Revision 263 - Directory Listing
Modified Sun Jan 31 18:18:38 2010 UTC (14 years, 1 month ago) by dpavlin
color nodes based on their state and refresh page if something is pending (load, view)


Revision 262 - Directory Listing
Modified Sun Jan 31 16:44:56 2010 UTC (14 years, 1 month ago) by dpavlin
send Refresh: header


Revision 261 - Directory Listing
Modified Sun Jan 31 16:44:33 2010 UTC (14 years, 1 month ago) by dpavlin
import everything from pxelator


Revision 260 - Directory Listing
Modified Sun Jan 31 16:03:50 2010 UTC (14 years, 1 month ago) by dpavlin
support loading of single shard


Revision 259 - Directory Listing
Modified Sun Jan 31 15:51:38 2010 UTC (14 years, 1 month ago) by dpavlin
graph ips by networks in time


Revision 258 - Directory Listing
Modified Sun Jan 31 15:44:50 2010 UTC (14 years, 1 month ago) by dpavlin
generate graph for each package by dates


Revision 257 - Directory Listing
Modified Tue Jan 26 17:34:19 2010 UTC (14 years, 2 months ago) by dpavlin
create shards from MongoDB


Revision 256 - Directory Listing
Modified Tue Jan 26 16:55:10 2010 UTC (14 years, 2 months ago) by dpavlin
split shard creation in Sack::Shard


Revision 255 - Directory Listing
Modified Thu Dec 24 13:08:30 2009 UTC (14 years, 3 months ago) by dpavlin
added missing ip


Revision 254 - Directory Listing
Modified Thu Dec 24 13:06:25 2009 UTC (14 years, 3 months ago) by dpavlin
fix warning


Revision 253 - Directory Listing
Modified Fri Nov 27 13:48:54 2009 UTC (14 years, 4 months ago) by dpavlin
added checkboxes to show just some keys on graph


Revision 252 - Directory Listing
Modified Fri Nov 27 13:48:38 2009 UTC (14 years, 4 months ago) by dpavlin
relaxed filter to full key name


Revision 251 - Directory Listing
Modified Fri Nov 27 13:48:01 2009 UTC (14 years, 4 months ago) by dpavlin
uri_unescape vars


Revision 250 - Directory Listing
Modified Thu Nov 26 22:36:27 2009 UTC (14 years, 4 months ago) by dpavlin
tweak nodes output


Revision 249 - Directory Listing
Modified Thu Nov 26 22:23:52 2009 UTC (14 years, 4 months ago) by dpavlin
display shard paths as title

Revision 248 - Directory Listing
Modified Thu Nov 26 22:18:19 2009 UTC (14 years, 4 months ago) by dpavlin
refresh shard allocation from view response


Revision 247 - Directory Listing
Modified Thu Nov 26 22:08:36 2009 UTC (14 years, 4 months ago) by dpavlin
refactor and extend client server configuration

- load shards only if they are not allready loaded
- collect client responses and return it in repl using . (dot)
- push all unknown repl commands to clients
- eval on client now returns result to server
- save will dump current chards on node to /tmp/sack.data and
  reload it on next startup


Revision 246 - Directory Listing
Modified Thu Nov 26 21:56:45 2009 UTC (14 years, 4 months ago) by dpavlin
filter output data by date

Revision 245 - Directory Listing
Modified Thu Nov 26 19:02:01 2009 UTC (14 years, 4 months ago) by dpavlin
support multiple url args which collapse into array and use it for filter


Revision 244 - Directory Listing
Modified Thu Nov 26 18:48:13 2009 UTC (14 years, 4 months ago) by dpavlin
cleanup title and use $param->{filter} to filter graphs with regex


Revision 243 - Directory Listing
Modified Thu Nov 26 18:14:43 2009 UTC (14 years, 4 months ago) by dpavlin
take just document from json response


Revision 242 - Directory Listing
Modified Thu Nov 26 17:32:38 2009 UTC (14 years, 4 months ago) by dpavlin
use CouchDB Bulk Document API

speedup from 800 docs/s to 1800 docs/s... not bad :-)


Revision 241 - Directory Listing
Modified Thu Nov 26 15:22:12 2009 UTC (14 years, 4 months ago) by dpavlin
show results <h1> only if there is reasults and include gnuplot for date, keys


Revision 240 - Directory Listing
Modified Thu Nov 26 14:46:46 2009 UTC (14 years, 4 months ago) by dpavlin
generate own data dumps, so we can be called with just $out


Revision 239 - Directory Listing
Modified Thu Nov 26 13:44:47 2009 UTC (14 years, 4 months ago) by dpavlin
cleanup sharding from WebPAC and allow user-defined name


Revision 238 - Directory Listing
Modified Tue Nov 24 15:21:09 2009 UTC (14 years, 4 months ago) by dpavlin
increase shard size to 5000 records


Revision 237 - Directory Listing
Modified Tue Nov 24 00:55:29 2009 UTC (14 years, 4 months ago) by dpavlin
sum of all merge counts doesn't make sense
since it contains dumplicate keys from each node


Revision 236 - Directory Listing
Modified Tue Nov 24 00:51:03 2009 UTC (14 years, 4 months ago) by dpavlin
count affected rows


Revision 235 - Directory Listing
Modified Tue Nov 24 00:46:19 2009 UTC (14 years, 4 months ago) by dpavlin
display number of merged keys from nodes


Revision 234 - Directory Listing
Modified Tue Nov 24 00:02:01 2009 UTC (14 years, 4 months ago) by dpavlin
count affected by return from view function


Revision 233 - Directory Listing
Modified Mon Nov 23 23:52:53 2009 UTC (14 years, 4 months ago) by dpavlin
display first shard leter and show/hide info


Revision 232 - Directory Listing
Modified Mon Nov 23 23:15:31 2009 UTC (14 years, 4 months ago) by dpavlin
refactor into run_view and display view overview table


Revision 231 - Directory Listing
Modified Mon Nov 23 22:08:12 2009 UTC (14 years, 4 months ago) by dpavlin
count facebook and reverse

Revision 230 - Directory Listing
Modified Mon Nov 23 22:07:49 2009 UTC (14 years, 4 months ago) by dpavlin
fix number of tests

Revision 229 - Directory Listing
Modified Mon Nov 23 20:46:30 2009 UTC (14 years, 4 months ago) by dpavlin
drop Chart::Gnuplot and drive gnuplot directly

this improved output and allowed overlay of multiple datasets on single graph


Revision 228 - Directory Listing
Modified Mon Nov 23 20:45:35 2009 UTC (14 years, 4 months ago) by dpavlin
no transport output without $debug


Revision 227 - Directory Listing
Modified Mon Nov 23 20:45:02 2009 UTC (14 years, 4 months ago) by dpavlin
always generate dump files


Revision 226 - Directory Listing
Modified Mon Nov 23 20:44:13 2009 UTC (14 years, 4 months ago) by dpavlin
collect hr, amazon, google domains


Revision 225 - Directory Listing
Modified Mon Nov 23 19:13:56 2009 UTC (14 years, 4 months ago) by dpavlin
don't die on client socket errors


Revision 224 - Directory Listing
Modified Mon Nov 23 19:13:33 2009 UTC (14 years, 4 months ago) by dpavlin
record key and value min/max


Revision 223 - Directory Listing
Modified Mon Nov 23 14:39:40 2009 UTC (14 years, 4 months ago) by dpavlin
create data files for gnuplot


Revision 222 - Directory Listing
Modified Sun Nov 22 22:57:15 2009 UTC (14 years, 4 months ago) by dpavlin
improve shards output


Revision 221 - Directory Listing
Modified Sun Nov 22 22:34:57 2009 UTC (14 years, 4 months ago) by dpavlin
factor out load_shard and display map in html


Revision 220 - Directory Listing
Modified Sun Nov 22 21:55:52 2009 UTC (14 years, 4 months ago) by dpavlin
fix perm

Revision 219 - Directory Listing
Modified Sun Nov 22 21:54:25 2009 UTC (14 years, 4 months ago) by dpavlin
color code nodes

red = forked
green = ping


Revision 218 - Directory Listing
Modified Sun Nov 22 20:48:08 2009 UTC (14 years, 4 months ago) by dpavlin
generate gnuplot graphs


Revision 217 - Directory Listing
Modified Sun Nov 22 20:47:20 2009 UTC (14 years, 4 months ago) by dpavlin
count ffzg separatly


Revision 216 - Directory Listing
Modified Sun Nov 22 20:36:53 2009 UTC (14 years, 4 months ago) by dpavlin
cleanup output


Revision 215 - Directory Listing
Modified Sun Nov 22 17:12:27 2009 UTC (14 years, 4 months ago) by dpavlin
added gnuplot grapher


Revision 214 - Directory Listing
Modified Sun Nov 22 16:39:45 2009 UTC (14 years, 4 months ago) by dpavlin
exaple of group by date from unix timestap


Revision 213 - Directory Listing
Modified Sun Nov 22 15:31:38 2009 UTC (14 years, 4 months ago) by dpavlin
move table generation in separate package and add sorting


Revision 212 - Directory Listing
Modified Sat Nov 21 16:34:12 2009 UTC (14 years, 4 months ago) by dpavlin
load shards and run views over web


Revision 211 - Directory Listing
Modified Sat Nov 21 16:12:18 2009 UTC (14 years, 4 months ago) by dpavlin
reogranize $info->{shard} into path => node


Revision 210 - Directory Listing
Modified Sat Nov 21 15:55:24 2009 UTC (14 years, 4 months ago) by dpavlin
simple http listener which dumps $info


Revision 209 - Directory Listing
Modified Sat Nov 21 13:59:41 2009 UTC (14 years, 4 months ago) by dpavlin
use etc/cloud as default if it exists


Revision 208 - Directory Listing
Modified Sat Nov 21 13:45:54 2009 UTC (14 years, 4 months ago) by dpavlin
document new sharding and startup


Revision 207 - Directory Listing
Modified Sat Nov 21 13:34:53 2009 UTC (14 years, 4 months ago) by dpavlin
align columns and fix calculations


Revision 206 - Directory Listing
Modified Sat Nov 21 13:21:34 2009 UTC (14 years, 4 months ago) by dpavlin
display percent, records/s and estimated time


Revision 205 - Directory Listing
Modified Sun Nov 8 23:23:45 2009 UTC (14 years, 4 months ago) by dpavlin
count domains


Revision 204 - Directory Listing
Modified Sun Nov 8 23:11:55 2009 UTC (14 years, 4 months ago) by dpavlin
count package names from pxelator

Revision 203 - Directory Listing
Modified Sun Nov 8 22:56:46 2009 UTC (14 years, 4 months ago) by dpavlin
added clean which drops all shards from clients


Revision 202 - Directory Listing
Modified Sun Nov 8 21:52:13 2009 UTC (14 years, 4 months ago) by dpavlin
use Data::Dump from /srv/Sack/lib on clients


Revision 201 - Directory Listing
Modified Sun Nov 8 20:36:23 2009 UTC (14 years, 4 months ago) by dpavlin
dump CouchDB into 1000 record shards


Revision 200 - Directory Listing
Modified Sun Nov 8 16:46:19 2009 UTC (14 years, 4 months ago) by dpavlin
merge view results on shards from clients on server


Revision 199 - Directory Listing
Modified Sun Nov 8 16:29:36 2009 UTC (14 years, 4 months ago) by dpavlin
call Sack::View from client

Revision 198 - Directory Listing
Modified Sun Nov 8 16:28:47 2009 UTC (14 years, 4 months ago) by dpavlin
extract Sack::Merge from Sack::Lorry


Revision 197 - Directory Listing
Modified Sun Nov 8 16:19:49 2009 UTC (14 years, 4 months ago) by dpavlin
document cloud startup


Revision 196 - Directory Listing
Modified Sun Nov 8 16:17:59 2009 UTC (14 years, 4 months ago) by dpavlin
send views to clients and collect loaded shards


Revision 195 - Directory Listing
Modified Sun Nov 8 16:17:21 2009 UTC (14 years, 4 months ago) by dpavlin
cleanup output and some asserts


Revision 194 - Directory Listing
Modified Sun Nov 8 16:14:53 2009 UTC (14 years, 4 months ago) by dpavlin
shard size as argument


Revision 193 - Directory Listing
Modified Sun Nov 8 14:47:48 2009 UTC (14 years, 4 months ago) by dpavlin
extracted Sack::View from old Sack::Node and test it


Revision 192 - Directory Listing
Modified Sun Nov 8 14:12:38 2009 UTC (14 years, 4 months ago) by dpavlin
load shard by shard whole dataset into clients as
they request it


Revision 191 - Directory Listing
Modified Sun Nov 8 13:45:14 2009 UTC (14 years, 4 months ago) by dpavlin
remove experiments directory


Revision 190 - Directory Listing
Modified Sun Nov 8 13:44:24 2009 UTC (14 years, 4 months ago) by dpavlin
start server and repl using splitvt


Revision 189 - Directory Listing
Modified Sun Nov 8 13:44:10 2009 UTC (14 years, 4 months ago) by dpavlin
report unknown commands from repl


Revision 188 - Directory Listing
Modified Sun Nov 8 13:40:58 2009 UTC (14 years, 4 months ago) by dpavlin
extract to_all

Revision 187 - Directory Listing
Modified Sun Nov 8 13:38:11 2009 UTC (14 years, 4 months ago) by dpavlin
cleanup sharding and move shards to /tmp/sack/name/


Revision 186 - Directory Listing
Modified Sun Nov 8 13:21:01 2009 UTC (14 years, 4 months ago) by dpavlin
shart input data into 1000 record shards


Revision 185 - Directory Listing
Modified Sun Nov 8 13:18:04 2009 UTC (14 years, 4 months ago) by dpavlin
use Time::HiRes for duration


Revision 184 - Directory Listing
Modified Sun Nov 8 13:11:38 2009 UTC (14 years, 4 months ago) by dpavlin
rename Sack::From to Sack::Shards

Revision 183 - Directory Listing
Modified Sun Nov 8 13:03:54 2009 UTC (14 years, 4 months ago) by dpavlin
tweak output and put into package


Revision 182 - Directory Listing
Modified Sun Nov 8 12:55:53 2009 UTC (14 years, 4 months ago) by dpavlin
rename repl

Revision 181 - Directory Listing
Modified Sun Nov 8 12:54:51 2009 UTC (14 years, 4 months ago) by dpavlin
start clients from lib/Sack/


Revision 180 - Directory Listing
Modified Sun Nov 8 12:53:32 2009 UTC (14 years, 4 months ago) by dpavlin
rename client

Revision 179 - Directory Listing
Modified Sun Nov 8 12:53:15 2009 UTC (14 years, 4 months ago) by dpavlin
move to package, show usage and rename fork_node to fork_ssh


Revision 178 - Directory Listing
Modified Sun Nov 8 12:45:43 2009 UTC (14 years, 4 months ago) by dpavlin
begin integration of protocol v3


Revision 177 - Directory Listing
Modified Fri Nov 6 11:12:59 2009 UTC (14 years, 4 months ago) by dpavlin
added last node back from repair

Revision 176 - Directory Listing
Modified Fri Nov 6 10:48:19 2009 UTC (14 years, 4 months ago) by dpavlin
push whole /srv/Sack/ to nodes, find absolute client.pl path


Revision 175 - Directory Listing
Modified Tue Nov 3 18:02:33 2009 UTC (14 years, 4 months ago) by dpavlin
added info command


Revision 174 - Directory Listing
Modified Tue Nov 3 17:51:27 2009 UTC (14 years, 4 months ago) by dpavlin
ping all clients


Revision 173 - Directory Listing
Modified Tue Nov 3 17:28:57 2009 UTC (14 years, 4 months ago) by dpavlin
deduce client path from server's


Revision 172 - Directory Listing
Modified Tue Nov 3 17:28:08 2009 UTC (14 years, 4 months ago) by dpavlin
start server and repl using splitvt


Revision 171 - Directory Listing
Modified Tue Nov 3 17:27:53 2009 UTC (14 years, 4 months ago) by dpavlin
retry connecting to server to give it time to start cloud


Revision 170 - Directory Listing
Modified Tue Nov 3 17:19:12 2009 UTC (14 years, 4 months ago) by dpavlin
take cloud path from CLOUD


Revision 169 - Directory Listing
Modified Mon Nov 2 16:09:40 2009 UTC (14 years, 4 months ago) by dpavlin
copy files using cpio, use ssh config and read cloud nodes from file


Revision 168 - Directory Listing
Modified Mon Nov 2 16:08:56 2009 UTC (14 years, 4 months ago) by dpavlin
we need Data::Dump somewhere (temp fix)


Revision 167 - Directory Listing
Modified Mon Nov 2 16:08:30 2009 UTC (14 years, 4 months ago) by dpavlin
use absolute path for IdentityFile


Revision 166 - Directory Listing
Modified Sun Nov 1 21:17:36 2009 UTC (14 years, 4 months ago) by dpavlin
collect interesting $info and send it to repl


Revision 165 - Directory Listing
Modified Sun Nov 1 21:03:26 2009 UTC (14 years, 4 months ago) by dpavlin
make server resiliant to repl disconnect


Revision 164 - Directory Listing
Modified Sun Nov 1 21:00:30 2009 UTC (14 years, 4 months ago) by dpavlin
added external read-eval-loop readline interface


Revision 163 - Directory Listing
Modified Sun Nov 1 20:30:48 2009 UTC (14 years, 4 months ago) by dpavlin
new client-server protocol based on IO::Select


Revision 162 - Directory Listing
Modified Fri Oct 30 14:53:22 2009 UTC (14 years, 5 months ago) by dpavlin
cleanup scoping of $out and $rec


Revision 161 - Directory Listing
Modified Fri Oct 30 14:52:34 2009 UTC (14 years, 5 months ago) by dpavlin
force node result to reference


Revision 160 - Directory Listing
Modified Fri Oct 30 14:49:39 2009 UTC (14 years, 5 months ago) by dpavlin
make default view produce *something*


Revision 159 - Directory Listing
Modified Fri Oct 23 13:08:45 2009 UTC (14 years, 5 months ago) by dpavlin
fix output


Revision 158 - Directory Listing
Modified Fri Oct 9 19:44:41 2009 UTC (14 years, 5 months ago) by dpavlin
add callback to get_from_all in Sack::Lorry [0.12]

Revision 157 - Directory Listing
Modified Fri Oct 9 19:32:39 2009 UTC (14 years, 5 months ago) by dpavlin
add --install to update all nodes in cloud

Revision 156 - Directory Listing
Modified Fri Oct 9 19:12:28 2009 UTC (14 years, 5 months ago) by dpavlin
use Sack::Color from here


Revision 155 - Directory Listing
Modified Fri Oct 9 19:12:05 2009 UTC (14 years, 5 months ago) by dpavlin
color warn lines without lf in red


Revision 154 - Directory Listing
Modified Fri Oct 9 18:20:42 2009 UTC (14 years, 5 months ago) by dpavlin
use_ok Sack::Lorry for syntax check only

Revision 153 - Directory Listing
Modified Fri Oct 9 18:19:29 2009 UTC (14 years, 5 months ago) by dpavlin
load Sack::Node also (it should be pure perl)

Revision 152 - Directory Listing
Modified Fri Oct 9 18:09:50 2009 UTC (14 years, 5 months ago) by dpavlin
added simple Makefile.PL

Revision 151 - Directory Listing
Modified Fri Oct 9 17:11:50 2009 UTC (14 years, 5 months ago) by dpavlin
move C1_country into it's own view


Revision 150 - Directory Listing
Modified Fri Oct 9 17:10:54 2009 UTC (14 years, 5 months ago) by dpavlin
fix warning

Revision 149 - Directory Listing
Modified Thu Oct 8 23:10:35 2009 UTC (14 years, 5 months ago) by dpavlin
added screencast script

Revision 148 - Directory Listing
Modified Thu Oct 8 23:09:05 2009 UTC (14 years, 5 months ago) by dpavlin
added Sack logo in SVG

Revision 147 - Directory Listing
Modified Wed Oct 7 23:21:59 2009 UTC (14 years, 5 months ago) by dpavlin
improved C1_country view

Revision 146 - Directory Listing
Modified Wed Oct 7 23:11:03 2009 UTC (14 years, 5 months ago) by dpavlin
move start_nodes and load_nodes to Sack::Lorry [0.11]

Revision 145 - Directory Listing
Modified Wed Oct 7 23:08:53 2009 UTC (14 years, 5 months ago) by dpavlin
turn on compression for 30% reduce in loading time on 100Mb/s network


Revision 144 - Directory Listing
Modified Wed Oct 7 20:55:14 2009 UTC (14 years, 5 months ago) by dpavlin
split update_node out

Revision 143 - Directory Listing
Modified Wed Oct 7 20:52:20 2009 UTC (14 years, 5 months ago) by dpavlin
use $Sack::VERSION

Revision 142 - Directory Listing
Modified Wed Oct 7 20:44:07 2009 UTC (14 years, 5 months ago) by dpavlin
use $Sack::VERSION, less verbose without debug


Revision 141 - Directory Listing
Modified Wed Oct 7 20:42:46 2009 UTC (14 years, 5 months ago) by dpavlin
fix output without DEBUG

Revision 140 - Directory Listing
Modified Wed Oct 7 20:14:58 2009 UTC (14 years, 5 months ago) by dpavlin
don't display warn "#..." output with DEBUG enviroment set


Revision 139 - Directory Listing
Modified Wed Oct 7 18:58:17 2009 UTC (14 years, 5 months ago) by dpavlin
fix command looping for repl

Revision 138 - Directory Listing
Modified Wed Oct 7 18:56:03 2009 UTC (14 years, 5 months ago) by dpavlin
re-use existing ssh tunnels if we can connect to nodes

Revision 137 - Directory Listing
Modified Wed Oct 7 18:30:22 2009 UTC (14 years, 5 months ago) by dpavlin
move commands to Sack::Lorry

Revision 136 - Directory Listing
Modified Wed Oct 7 18:29:38 2009 UTC (14 years, 5 months ago) by dpavlin
turn compression off to preserve CPU on lorry

Revision 135 - Directory Listing
Modified Wed Oct 7 17:57:50 2009 UTC (14 years, 5 months ago) by dpavlin
extract duration into Sack, unify versioning [0.10]

Revision 134 - Directory Listing
Modified Wed Oct 7 17:47:50 2009 UTC (14 years, 5 months ago) by dpavlin
sane pid messages

Revision 133 - Directory Listing
Modified Wed Oct 7 17:39:40 2009 UTC (14 years, 5 months ago) by dpavlin
don't kill $$


Revision 132 - Directory Listing
Modified Wed Oct 7 17:27:43 2009 UTC (14 years, 5 months ago) by dpavlin
fix kill message

Revision 131 - Directory Listing
Modified Wed Oct 7 17:17:52 2009 UTC (14 years, 5 months ago) by dpavlin
write shard locally on lorry to speed up cloud bringup and re-seed nodes on restart

Revision 130 - Directory Listing
Modified Wed Oct 7 17:17:07 2009 UTC (14 years, 5 months ago) by dpavlin
re-exec node on serialization error

Revision 129 - Directory Listing
Modified Wed Oct 7 16:22:14 2009 UTC (14 years, 5 months ago) by dpavlin
make release goals external to my head :-)

Revision 128 - Directory Listing
Modified Wed Oct 7 16:21:48 2009 UTC (14 years, 5 months ago) by dpavlin
explain # better


Revision 127 - Directory Listing
Modified Wed Oct 7 16:21:33 2009 UTC (14 years, 5 months ago) by dpavlin
better recovery from node storable serialization errors,
faster cloud startup, 
added common pid handling,
added [u]pdate to lorry which push updated code to nodes,
tweaks to output,
version bump for Sack::Node [0.08]


Revision 126 - Directory Listing
Modified Wed Oct 7 14:15:56 2009 UTC (14 years, 5 months ago) by dpavlin
process each node once


Revision 125 - Directory Listing
Modified Wed Oct 7 14:15:06 2009 UTC (14 years, 5 months ago) by dpavlin
process each node just once


Revision 124 - Directory Listing
Modified Wed Oct 7 13:35:53 2009 UTC (14 years, 5 months ago) by dpavlin
added two more machines

Revision 123 - Directory Listing
Modified Wed Oct 7 12:30:25 2009 UTC (14 years, 5 months ago) by dpavlin
compress ssh tunnel to nodes

Revision 122 - Directory Listing
Modified Wed Oct 7 12:29:12 2009 UTC (14 years, 5 months ago) by dpavlin
execute command on each host using ssh


Revision 121 - Directory Listing
Modified Wed Oct 7 11:54:24 2009 UTC (14 years, 5 months ago) by dpavlin
use both cores on machines (double every host)


Revision 120 - Directory Listing
Modified Wed Oct 7 10:05:32 2009 UTC (14 years, 5 months ago) by dpavlin
don't output debug <<<< and >>>

Revision 119 - Directory Listing
Modified Tue Oct 6 23:04:45 2009 UTC (14 years, 5 months ago) by dpavlin
new commands: [r]estart, [sh]ell, [i]nfo [0.08]


Revision 118 - Directory Listing
Modified Tue Oct 6 23:02:39 2009 UTC (14 years, 5 months ago) by dpavlin
simple dual localhost cloud config for local testing

Revision 117 - Directory Listing
Modified Tue Oct 6 00:11:08 2009 UTC (14 years, 5 months ago) by dpavlin
better messages, show result from nodes,
*_all functions now send to sorted node list
return ticks from merge and show total view records


Revision 116 - Directory Listing
Modified Tue Oct 6 00:07:05 2009 UTC (14 years, 5 months ago) by dpavlin
added report to node view resposne and version it [0.07]


Revision 115 - Directory Listing
Modified Tue Oct 6 00:03:21 2009 UTC (14 years, 5 months ago) by dpavlin
no need to use digest function on nodes

Revision 114 - Directory Listing
Modified Mon Oct 5 23:55:06 2009 UTC (14 years, 5 months ago) by dpavlin
remove incompatible version 0.06


Revision 113 - Directory Listing
Modified Mon Oct 5 23:53:46 2009 UTC (14 years, 5 months ago) by dpavlin
added x for exit

Revision 112 - Directory Listing
Modified Mon Oct 5 23:53:12 2009 UTC (14 years, 5 months ago) by dpavlin
unconditinaly copy files to nodes

Revision 111 - Directory Listing
Modified Mon Oct 5 23:21:20 2009 UTC (14 years, 5 months ago) by dpavlin
implement in-memory digest internal to lorry


Revision 110 - Directory Listing
Modified Mon Oct 5 23:20:43 2009 UTC (14 years, 5 months ago) by dpavlin
implement v(iew) d(dump) commands

Revision 109 - Directory Listing
Modified Mon Oct 5 23:14:46 2009 UTC (14 years, 5 months ago) by dpavlin
remove slow node

Revision 108 - Directory Listing
Modified Mon Oct 5 22:38:58 2009 UTC (14 years, 5 months ago) by dpavlin
without params use just single localhost node

Revision 107 - Directory Listing
Modified Mon Oct 5 22:21:39 2009 UTC (14 years, 5 months ago) by dpavlin
move ping to Sack::Lorry, display duration of operations and
implement new configuration in etc


Revision 106 - Directory Listing
Modified Mon Oct 5 22:19:54 2009 UTC (14 years, 5 months ago) by dpavlin
simpliest possible push to nodes (on which we run dropbear, so no scp or rsync)


Revision 105 - Directory Listing
Modified Mon Oct 5 22:19:12 2009 UTC (14 years, 5 months ago) by dpavlin
ping all hosts in cloud


Revision 104 - Directory Listing
Modified Mon Oct 5 22:18:30 2009 UTC (14 years, 5 months ago) by dpavlin
and ssh configuration

Revision 103 - Directory Listing
Modified Mon Oct 5 22:18:13 2009 UTC (14 years, 5 months ago) by dpavlin
define cloud using /etc/hosts like file


Revision 102 - Directory Listing
Modified Mon Oct 5 20:31:28 2009 UTC (14 years, 5 months ago) by dpavlin
color [port] in warn messages


Revision 101 - Directory Listing
Modified Mon Oct 5 19:34:45 2009 UTC (14 years, 5 months ago) by dpavlin
test nodes with Net::Ping

Revision 100 - Directory Listing
Modified Sun Oct 4 14:42:48 2009 UTC (14 years, 5 months ago) by dpavlin
re-run query repl

Revision 99 - Directory Listing
Modified Sun Oct 4 14:42:35 2009 UTC (14 years, 5 months ago) by dpavlin
don't dump 

Revision 98 - Directory Listing
Modified Sun Oct 4 14:39:50 2009 UTC (14 years, 5 months ago) by dpavlin
reset $out before running view

Revision 97 - Directory Listing
Modified Sun Oct 4 13:09:28 2009 UTC (14 years, 5 months ago) by dpavlin
re-implement merge of results

Revision 96 - Directory Listing
Modified Sun Oct 4 12:50:10 2009 UTC (14 years, 5 months ago) by dpavlin
rsync latest sack to nodes on startup


Revision 95 - Directory Listing
Modified Sun Oct 4 12:17:52 2009 UTC (14 years, 5 months ago) by dpavlin
make correct shards

Revision 94 - Directory Listing
Modified Sun Oct 4 12:17:31 2009 UTC (14 years, 5 months ago) by dpavlin
fix country or US state (sic!) extraction


Revision 93 - Directory Listing
Modified Sat Oct 3 21:43:34 2009 UTC (14 years, 5 months ago) by dpavlin
run simple view

Revision 92 - Directory Listing
Modified Sat Oct 3 21:09:51 2009 UTC (14 years, 5 months ago) by dpavlin
begin refactor to lorry which will carry sacks for us
(creating nodes and communicating with them using Storable
traveling over sockets)


Revision 91 - Directory Listing
Modified Sat Oct 3 21:08:36 2009 UTC (14 years, 5 months ago) by dpavlin
use $digest_module for remote digests

Revision 90 - Directory Listing
Modified Wed Sep 30 19:54:01 2009 UTC (14 years, 6 months ago) by dpavlin
added digest implementation based on shared redis server

Revision 89 - Directory Listing
Modified Wed Sep 30 19:53:29 2009 UTC (14 years, 6 months ago) by dpavlin
exit on CTRL+D

Revision 88 - Directory Listing
Modified Wed Sep 30 19:53:12 2009 UTC (14 years, 6 months ago) by dpavlin
added sync stub

Revision 87 - Directory Listing
Modified Wed Sep 30 19:30:45 2009 UTC (14 years, 6 months ago) by dpavlin
use SACK_DIGEST to configure digest module

Revision 86 - Directory Listing
Modified Wed Sep 30 17:23:32 2009 UTC (14 years, 6 months ago) by dpavlin
extract common parts into Sack::Digest

Revision 85 - Directory Listing
Modified Tue Sep 29 18:04:29 2009 UTC (14 years, 6 months ago) by dpavlin
rename Sack::Digest to Sack::Digest::BerkeleyDB to indicate implementation

Revision 84 - Directory Listing
Modified Tue Sep 29 16:54:05 2009 UTC (14 years, 6 months ago) by dpavlin
speedup 10% by evaling one once into coderef


Revision 83 - Directory Listing
Modified Sat Sep 26 12:33:04 2009 UTC (14 years, 6 months ago) by dpavlin
start cluster after update and cleanup


Revision 82 - Directory Listing
Modified Sat Sep 26 12:32:30 2009 UTC (14 years, 6 months ago) by dpavlin
colorize ports in all messages, version bump [0.06]


Revision 81 - Directory Listing
Modified Sat Sep 26 10:40:05 2009 UTC (14 years, 6 months ago) by dpavlin
use ansi colors for port numbers in warn messages

Revision 80 - Directory Listing
Modified Sat Sep 26 10:14:03 2009 UTC (14 years, 6 months ago) by dpavlin
huge speedup using $merge_digest_mapping to keep information in-memory
between view runs


Revision 79 - Directory Listing
Modified Fri Sep 25 23:22:32 2009 UTC (14 years, 6 months ago) by dpavlin
fix digest output

Revision 78 - Directory Listing
Modified Fri Sep 25 21:32:20 2009 UTC (14 years, 6 months ago) by dpavlin
exit after update

Revision 77 - Directory Listing
Modified Fri Sep 25 21:13:25 2009 UTC (14 years, 6 months ago) by dpavlin
use separate ssh session for sacks

Revision 76 - Directory Listing
Modified Fri Sep 25 21:12:46 2009 UTC (14 years, 6 months ago) by dpavlin
wait for all nodes before sending view

Revision 75 - Directory Listing
Modified Fri Sep 25 20:31:38 2009 UTC (14 years, 6 months ago) by dpavlin
store pid file and kill existing sack, close socket to client

Revision 74 - Directory Listing
Modified Fri Sep 25 18:52:58 2009 UTC (14 years, 6 months ago) by dpavlin
record all report output (reformatted) and show it with "i r" command,
version bump [0.05]


Revision 73 - Directory Listing
Modified Fri Sep 25 18:51:42 2009 UTC (14 years, 6 months ago) by dpavlin
fix verbosity and enlarge t61p cloud

Revision 72 - Directory Listing
Modified Fri Sep 25 16:47:49 2009 UTC (14 years, 6 months ago) by dpavlin
don't pull file from node if it's here

Revision 71 - Directory Listing
Modified Fri Sep 25 16:37:11 2009 UTC (14 years, 6 months ago) by dpavlin
add hostname to xterm title before port

Revision 70 - Directory Listing
Modified Fri Sep 25 16:36:24 2009 UTC (14 years, 6 months ago) by dpavlin
trigger view from shell

Revision 69 - Directory Listing
Modified Fri Sep 25 16:29:38 2009 UTC (14 years, 6 months ago) by dpavlin
start xterms if $DISPLAY is set

Revision 68 - Directory Listing
Modified Fri Sep 25 16:29:14 2009 UTC (14 years, 6 months ago) by dpavlin
fix digest/s

Revision 67 - Directory Listing
Modified Fri Sep 25 14:35:54 2009 UTC (14 years, 6 months ago) by dpavlin
tweaks to merge output, reopen digest on node after transfer

Revision 66 - Directory Listing
Modified Fri Sep 25 14:35:05 2009 UTC (14 years, 6 months ago) by dpavlin
added open

Revision 65 - Directory Listing
Modified Fri Sep 25 14:01:48 2009 UTC (14 years, 6 months ago) by dpavlin
cleanup /dev/shm/sack.* before start

Revision 64 - Directory Listing
Modified Fri Sep 25 14:01:31 2009 UTC (14 years, 6 months ago) by dpavlin
ship files over socket with client

Revision 63 - Directory Listing
Modified Fri Sep 25 13:49:42 2009 UTC (14 years, 6 months ago) by dpavlin
force exit of running sack(s), use port only at connect


Revision 62 - Directory Listing
Modified Fri Sep 25 13:16:04 2009 UTC (14 years, 6 months ago) by dpavlin
make general cloud.sh starter

Revision 61 - Directory Listing
Modified Fri Sep 25 13:15:13 2009 UTC (14 years, 6 months ago) by dpavlin
rename

Revision 60 - Directory Listing
Modified Fri Sep 25 13:02:32 2009 UTC (14 years, 6 months ago) by dpavlin
tweaks in output, fix (add) node

Revision 59 - Directory Listing
Modified Fri Sep 25 12:24:42 2009 UTC (14 years, 6 months ago) by dpavlin
print merge ticks

Revision 58 - Directory Listing
Modified Fri Sep 25 12:24:19 2009 UTC (14 years, 6 months ago) by dpavlin
print send/receive only if smaller than 1k or with debug

Revision 57 - Directory Listing
Modified Fri Sep 25 10:07:04 2009 UTC (14 years, 6 months ago) by dpavlin
new connect with just port


Revision 56 - Directory Listing
Modified Fri Sep 25 10:05:45 2009 UTC (14 years, 6 months ago) by dpavlin
define node just by port, we are using ssh anyway

Revision 55 - Directory Listing
Modified Fri Sep 25 09:38:48 2009 UTC (14 years, 6 months ago) by dpavlin
added node command to add node

Revision 54 - Directory Listing
Modified Thu Sep 24 22:23:30 2009 UTC (14 years, 6 months ago) by dpavlin
added lookup

Revision 53 - Directory Listing
Modified Thu Sep 24 20:14:27 2009 UTC (14 years, 6 months ago) by dpavlin
re-added $digest->sync so we see data in storableedit.pl

Revision 52 - Directory Listing
Modified Thu Sep 24 20:11:27 2009 UTC (14 years, 6 months ago) by dpavlin
tweak case


Revision 51 - Directory Listing
Modified Thu Sep 24 19:45:08 2009 UTC (14 years, 6 months ago) by dpavlin
simplify and fix prefix extraction

Revision 50 - Directory Listing
Modified Thu Sep 24 19:21:55 2009 UTC (14 years, 6 months ago) by dpavlin
merge digests from other nodes, version bump [0.04]


Revision 49 - Directory Listing
Modified Thu Sep 24 17:44:25 2009 UTC (14 years, 6 months ago) by dpavlin
killall sack.pl before start

Revision 48 - Directory Listing
Modified Thu Sep 24 17:43:51 2009 UTC (14 years, 6 months ago) by dpavlin
simplify and remove warning about sub redefine

Revision 47 - Directory Listing
Modified Thu Sep 24 17:11:17 2009 UTC (14 years, 6 months ago) by dpavlin
extract part of repeatable field

Revision 46 - Directory Listing
Modified Thu Sep 24 13:46:16 2009 UTC (14 years, 6 months ago) by dpavlin
display just unique part of filename, sync digest to disk on end of view


Revision 45 - Directory Listing
Modified Thu Sep 24 13:37:53 2009 UTC (14 years, 6 months ago) by dpavlin
turn Sack::Digest into normal perl object with new instead of open


Revision 44 - Directory Listing
Modified Thu Sep 24 12:48:08 2009 UTC (14 years, 6 months ago) by dpavlin
Sack::Digest now uses md5 sum of value for btree and array, so
we store full value only once


Revision 43 - Directory Listing
Modified Thu Sep 24 12:13:41 2009 UTC (14 years, 6 months ago) by dpavlin
check undigested hash

Revision 42 - Directory Listing
Modified Thu Sep 24 00:00:39 2009 UTC (14 years, 6 months ago) by dpavlin
fix examples for new # syntax

Revision 41 - Directory Listing
Modified Thu Sep 24 00:00:24 2009 UTC (14 years, 6 months ago) by dpavlin
better output

Revision 40 - Directory Listing
Modified Wed Sep 23 23:54:38 2009 UTC (14 years, 6 months ago) by dpavlin
display real position in ticks

Revision 39 - Directory Listing
Modified Wed Sep 23 23:52:40 2009 UTC (14 years, 6 months ago) by dpavlin
leave to interactive lorry second part of set so it can have less records than
node because it needs also to merge them


Revision 38 - Directory Listing
Modified Wed Sep 23 23:26:08 2009 UTC (14 years, 6 months ago) by dpavlin
easy to start configuration for dual-core machine


Revision 37 - Directory Listing
Modified Wed Sep 23 23:10:50 2009 UTC (14 years, 6 months ago) by dpavlin
added unmungling to storableedit.pl

Revision 36 - Directory Listing
Modified Wed Sep 23 22:22:18 2009 UTC (14 years, 6 months ago) by dpavlin
undigest_out, document digest key names, version bump [0.03]

Revision 35 - Directory Listing
Modified Wed Sep 23 21:55:08 2009 UTC (14 years, 6 months ago) by dpavlin
dump output if smaller than 4k, extracted info_tabs and added files


Revision 34 - Directory Listing
Modified Wed Sep 23 21:47:32 2009 UTC (14 years, 6 months ago) by dpavlin
move digest to tie interface which works for me


Revision 33 - Directory Listing
Modified Wed Sep 23 20:28:21 2009 UTC (14 years, 6 months ago) by dpavlin
extract Sack::Digest into own package and test it


Revision 32 - Directory Listing
Modified Wed Sep 23 18:52:56 2009 UTC (14 years, 6 months ago) by dpavlin
http://www.lothosoft.ch/thomas/perl-storableedit/storableedit-1.5.pl

Revision 31 - Directory Listing
Modified Wed Sep 23 18:51:32 2009 UTC (14 years, 6 months ago) by dpavlin
fix SAVE output

Revision 30 - Directory Listing
Modified Wed Sep 23 18:44:08 2009 UTC (14 years, 6 months ago) by dpavlin
- added external B-tree using BerkeleyDB::Btree on /dev/shm for efficiat digest to integer
- version bump [0.02]
- cleanup output to always start with $port for easy overview
- display ticks every 1000 and 10000 records while processing view
- don't serialize output with Data::Dump, use Storable instead to save it on disk
- extracted info into sub


Revision 29 - Directory Listing
Modified Wed Sep 23 18:41:10 2009 UTC (14 years, 6 months ago) by dpavlin
don't dump shell debug

Revision 28 - Directory Listing
Modified Tue Sep 22 22:28:40 2009 UTC (14 years, 6 months ago) by dpavlin
use md5 digest to solve first memory problem with C1, but not with CR yet

Revision 27 - Directory Listing
Modified Tue Sep 22 21:58:12 2009 UTC (14 years, 6 months ago) by dpavlin
shell script to startup sack cloud

Revision 26 - Directory Listing
Modified Tue Sep 22 21:45:04 2009 UTC (14 years, 6 months ago) by dpavlin
no backtrace from warn

Revision 25 - Directory Listing
Modified Tue Sep 22 21:38:31 2009 UTC (14 years, 6 months ago) by dpavlin
display number of records loaded

Revision 24 - Directory Listing
Modified Tue Sep 22 21:27:13 2009 UTC (14 years, 6 months ago) by dpavlin
don't parse subfields in WebPAC::Input::ISI for nice memory usage
check if we got anything from client before unserializing it


Revision 23 - Directory Listing
Modified Tue Sep 22 17:40:26 2009 UTC (14 years, 6 months ago) by dpavlin
Sack overview

Revision 22 - Directory Listing
Modified Tue Sep 22 17:19:25 2009 UTC (14 years, 6 months ago) by dpavlin
fix output

Revision 21 - Directory Listing
Modified Tue Sep 22 17:15:03 2009 UTC (14 years, 6 months ago) by dpavlin
- added $VERSION [0.01]
- exit connected nodes on SIGINT
- fix merge when shards produce array of values
- make code display debug option
- listen on 127.0.0.1 (we are using ssh tunnels anyway)
- changed commands a bit and added interactive help on h or ?


Revision 20 - Directory Listing
Modified Tue Sep 22 16:02:49 2009 UTC (14 years, 6 months ago) by dpavlin
abort on first error

Revision 19 - Directory Listing
Modified Tue Sep 22 15:14:00 2009 UTC (14 years, 6 months ago) by dpavlin
added --debug and move some output to it

Revision 18 - Directory Listing
Modified Tue Sep 22 13:46:41 2009 UTC (14 years, 6 months ago) by dpavlin
add + to name of field to make sum agregation from shards


Revision 17 - Directory Listing
Modified Tue Sep 22 13:40:02 2009 UTC (14 years, 6 months ago) by dpavlin
fix warning

Revision 16 - Directory Listing
Modified Tue Sep 22 13:32:04 2009 UTC (14 years, 6 months ago) by dpavlin
- support send_nodes without content playload
- warn about nodes which go away or can't be connected to (instead of die)
- get_node to fetch data over $connected socket to node (opened by send_nodes)
- send_sock to send data to node over socket
- added info command to get overwiew of sharding


Revision 15 - Directory Listing
Modified Tue Sep 22 12:09:59 2009 UTC (14 years, 6 months ago) by dpavlin
Debian dependencies

Revision 14 - Directory Listing
Modified Tue Sep 22 10:40:32 2009 UTC (14 years, 6 months ago) by dpavlin
fix prefix

Revision 13 - Directory Listing
Modified Tue Sep 22 10:32:59 2009 UTC (14 years, 6 months ago) by dpavlin
ship results from node using Storable, merge results
(with support of + in key name to make sum)


Revision 12 - Directory Listing
Modified Tue Sep 22 10:00:22 2009 UTC (14 years, 6 months ago) by dpavlin
find $prefix of our installation and use it


Revision 11 - Directory Listing
Modified Mon Sep 21 23:21:34 2009 UTC (14 years, 6 months ago) by dpavlin
implement --listen port and --connect host:port to create master-slave cluster
(err, cloud) of nodes and implement send_nodes to send view to each node

added info to see connected views to commands and allow abbrivitation to single character


Revision 10 - Directory Listing
Modified Mon Sep 21 20:32:51 2009 UTC (14 years, 6 months ago) by dpavlin
reset time on each call of report

Revision 9 - Directory Listing
Modified Mon Sep 21 20:05:41 2009 UTC (14 years, 6 months ago) by dpavlin
fix offset and limit, rename last out/ dump to out/*.last


Revision 8 - Directory Listing
Modified Mon Sep 21 19:36:32 2009 UTC (14 years, 6 months ago) by dpavlin
make in-memory cache local


Revision 7 - Directory Listing
Modified Mon Sep 21 15:45:21 2009 UTC (14 years, 6 months ago) by dpavlin
get first rec in database

Revision 6 - Directory Listing
Modified Mon Sep 21 15:41:30 2009 UTC (14 years, 6 months ago) by dpavlin
added --view which can be repeated

Revision 5 - Directory Listing
Modified Mon Sep 21 15:36:47 2009 UTC (14 years, 6 months ago) by dpavlin
display affected records and warning about empty $out

Revision 4 - Directory Listing
Modified Sun Sep 20 20:08:11 2009 UTC (14 years, 6 months ago) by dpavlin
configurable --path, --offset and --limit


Revision 3 - Directory Listing
Modified Sun Sep 20 18:55:34 2009 UTC (14 years, 6 months ago) by dpavlin
open out/* in vi with any of commands: vi \e out


Revision 2 - Directory Listing
Modified Sun Sep 20 18:54:50 2009 UTC (14 years, 6 months ago) by dpavlin
count SC

Revision 1 - Directory Listing
Added Sun Sep 20 18:42:07 2009 UTC (14 years, 6 months ago) by dpavlin
Sack - in-memory interactive object database in pure perl


  ViewVC Help
Powered by ViewVC 1.1.26