How to Store Events in SimpleFeed

Understanding the Score and the Value

From the library perspective, the only thing you can store as a value: is of data-type String, which is necessarily associated with a score of type Double. You can provide a custom score in the at: argument of #store, but without this argument the current timestamp will be used by the default.

• While the most common use for the score is as a (current) timestamp, other uses are possible.

• The value is always custom — based on the implementation. It represents your way of serializing the events.

This is because how you store your events is up to you, the developer, to decide.

Having said that, here are some considerations:

Considerations for Event Serialization

• the less you store the more a single-instance Redis backend can scale up without having to be sharded

• the less you store the faster network interactions will be with the feed, and less likely it will become a point of contention

• the less you store in SimpleFeed, the more you, probably, will need to fetch in addition to what SimpleFeed already returns, i.e. in order to show the user their activity full with the name of the authors, comments, etc.

With that in mind, imagine that we are building an application where most objects in the system have a database identifier field. If we must store various types of models as events, we can choose a custom serialization scheme where we reserve one or two letters for the model name, then a dot, then the database ID — as a long integer that's converted to a base of 64 string.

For example,

require 'base64'
class Post
  def simple_feed_id
    @simple_feed_id ||= 'po.' + Base64.encode64(Array(id).pack('C*'))

  end
  def self.from_simple_feed_id(encoded)
    if encoded.start_with?('po.')
      self.find_by_id(Base64.decode64(encoded[3..-1]).unpack("C*"))
    end
  end 
end

Then use #simple_feed_id to save as a Value, in #store(value: ...). Upon retrieving a value from the SimpleFeed, you would read the result and instantiate an object by calling eg.

class Post
  def self.news_feed_for(user)
    activity = SimpleFeed.feed(:news).activity(user.id)
    events = activity.paginate(page: 1)
    events.map do |event|
      self.class.from_simple_feed_id(event)
    end
  end
end

Persisting Events: Using a Polymorphic EVENTS Table with SimpleFeed

We created this polymorphic EVENTS table to records all interesting events that happened, associated only with their producer and not with the consumers (i.e. followers). This is easy and relatively cheap (up to a point) implement, and ensures you have an authoritative list of events in your system, and you can always re-generate user activity in SimpleFeed, should you need to.

Now, all you have to store in the Value is the database ID of the event. You can even store it as a base64-encoded number, for additional compactness.

Then, to render a given user's feed, you'd first call #paginate to get a page-worth of event IDs, and then fetch the actual data from the database, using primary-key lookup against an array of IDs. This will scale very well, and is easily cached, because all lookups are always by the primary key. Caching libraries like cache-object might offer a drop-in write-through caching solution.