I have a social network
The users table is around 60,000 rows
The friends table is around 1 million rows (used to determine who is your
friend)
I am wanting to do a friend feed, wall, whatever you like to call it, it will show things like user status post (twitter type posts), it will show a few different items but for the start it will just be friend status and maybe blog post.
Basicly you will only see content published with a user ID that is in your friend list.
I have been trying to come up with the best way and haven't gotten very far but here is my latest idea.
Currently to build this feed, I have to
Get the list of friend ID's on the large friend table
Get the stream data from the friend ids from the above result
JOIN the user table to get the publishers picture URL and username
Then JOIN the comments table to get comments posted on the feed items
That is one big task to build that feed
I have 3 ideas so far, this is where your help can come in.
Memcache Option:
File cache Option:
Do the same as the memcache option does but save this data as an array to a cache file instead of memory, then include this cache file into the page.
I am not sure which is the best method for performance I understand memcache stores everything in memory so friends that have like 20,000 friends that could use a lot of memory and a file cache would only put it in memory when the users needs it if I am correct. Also if I did the file method, when a user logs out of the site, I would delete there cache file so the cache folder would never be too large of files
Session cache Option:
Please give me your opinions or any advice or info you have on this as I don't have much knowledge of caching, I have read a lot but sometimes other peoples ideas help a lot
Session data is stored at the user level but caching data is stored at the application level and shared by all the users. Sessions may not improve performance whereas Cache will improve site performance. Items in cache can expire after given time to cache while items in session will stay till session expires.
Redis uses a single core and shows better performance than Memcached in storing small datasets when measured in terms of cores. Memcached implements a multi-threaded architecture by utilizing multiple cores. Therefore, for storing larger datasets, Memcached can perform better than Redis.
When storing data, Redis stores data as specific data types, whereas Memcached only stores data as strings. Because of this, Redis can change data in place without having to re-upload the entire data value. This reduces network overhead.
Although they are both easy to use and offer high performance, there are important differences to consider when choosing an engine. Memcached is designed for simplicity while Redis offers a rich set of features that make it effective for a wide range of use cases.
Memcache is your best bet for a lot of reasons:
... and many more ;)
As to some of your concerns about memory footprint of individual cached items you've got a few options. My initial thought is to just give it a whirl, see how big these cache items really get (you can find several open-source things to monitor the actual cache usage, such as cacti). I think they'll be smaller than you'd think.
If they're not, I'd suggest re-thinking your cache strategy as far as what you actually cache, for how long, etc. Maybe you could build the feed from several things already in the cache (i.e. cache individual user data, and then build the feed for a person from all those individual items in cache). There are a lot of good articles out there on that front, just search 'em out :)
The default maximum object size that is allowed in Memcache is 1MB.
@jsaondavis : "session data is saved into a file".
Your above statement is wrong. Session can be configured to store in database. Default session hadndler is file.
If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With