Contents of /trunk/README

Sack - sharding memory hash in perl

Main design goal is to have interactive environment to query
perl hashes which are bigger than memory on single machine.

It implemented using TCP sockets between perl processes.
This allows horizontal scalability both on multi-core machines
as well as across the network to additional machines.

Reading data into hash is done using any perl module which
returns perl hash and supports offset and limit to select just
subset of data (this is required to create disjunctive shards).

Views are small perl snippets which are called for each record
on each shard with $rec. Views create data in $out hash which
is automatically merged in output.

You can influence default shard merge by adding " +" (space plus)
to name of your key to indicate that values should be summed
after collecting them from from shards.

1	Sack - sharding memory hash in perl
2
3	Main design goal is to have interactive environment to query
4	perl hashes which are bigger than memory on single machine.
5
6	It implemented using TCP sockets between perl processes.
7	This allows horizontal scalability both on multi-core machines
8	as well as across the network to additional machines.
9
10	Reading data into hash is done using any perl module which
11	returns perl hash and supports offset and limit to select just
12	subset of data (this is required to create disjunctive shards).
13
14	Views are small perl snippets which are called for each record
15	on each shard with $rec. Views create data in $out hash which
16	is automatically merged in output.
17
18	You can influence default shard merge by adding " +" (space plus)
19	to name of your key to indicate that values should be summed
20	after collecting them from from shards.
21