Remove 2003 Remove Aggregator Remove Algorithm Remove Product Development
article thumbnail

Lessons Learned: Sharding for startups

Startup Lessons Learned

More common is to use a one-way hashing algorithm to map the data to be accessed to one of the shards that store it. Using memcacheds multiget, which allows the fetching of many keys in parallel, I have written code to aggregate all the shard lookups for a given page and prefetch them, reducing the overhead even further.