Confused about Meteor: how to send data to all clients without writing to the database?

Question

I've actually been toying with Meteor for a little bit now, but I realized that I still lack some (or a lot!) comprehension on the topic.

For example, here is a tutorial that uses node.js/express/socket.io to make a simple real-time chat: http://net.tutsplus.com/tutorials/javascript-ajax/real-time-chat-with-nodejs-socket-io-and-expressjs/

In that above example, through socket.io, the webserver receives some data and passes it onto all of the connected clients -- all without any database accesses.

With Meteor, in all the examples that I've seen, clients are updated by writing to the mongodb, which then updates all the clients. But what if I don't need to write data to the database? It seems like an expensive step to pass data to all clients.

I am sure I am missing something here. What would be the Meteor way of updating all the clients (say, like with a simple chat app), but without needing the expense of writing to a database first?

Thank you!

Answer 1

At the moment there isn't an official way to send data to clients without writing it to a collection. Its a little tricker in meteor because the step to send data to multiple clients when there isn't a place to write to comes from when multiple meteor's are used together. I.e items sent from one meteor won't come to clients subscribed on the other.

There is a temporary solution using Meteor Streams (http://meteorhacks.com/introducing-meteor-streams.html) that can let you do what you want without writing to the database in the meanwhile.

There is also a pretty extensive discussion about this on meteor-talk (https://groups.google.com/forum/#!topic/meteor-talk/Ze9U9lEozzE) if you want to understand some of the technical details. This will actually become possible when the linker branch is merged into master, for a single server

Here's a bit of way to have a 'virtual collection, its not perfect but it can do until Meteor has a more polished way of having it done.

Meteor.publish("virtual_collection", function() {
    this.added("virtual_coll", "some_id_of_doc", {key: "value"});

    //When done
    this.ready()
});

Then subscribe to this on the client:

var Virt_Collection = new Meteor.Collection("virtual_coll");
Meteor.subscribe("virtual_collection");

Then you could run this when the subscription is complete:

Virt_Collection.findOne();
=> { _id: "some_id_of_doc", key: "value"}

This is a bit messy but you could also hook into it to update or remove collections. At least this way though you won't be using any plugins or packages.

See : https://www.eventedmind.com/posts/meteor-how-to-publish-to-a-client-only-collection for more details and a video example.

Answer 2

The publish function on the server sends data to clients. It has some convenient shortcuts to publish query results from the database but you do not have to use these. The publish function has this.added(), this.removed(), and this.changed() that allow you to publish anything you choose. The client then subscribes and receives the published data.

For example:

if ( Meteor.isClient ){
  var someMessages = new Meteor.Collection( "messages" );  //messages is name of collection on client side
  Meteor.subscribe ( "messagesSub" );  //messagesSub tells the server which publish function to request data from

  Deps.autorun( function(){
    var message = someMessages.findOne({});
    if ( message )  console.log( message.m ); // prints This is not from a database
  });
}

if (Meteor.isServer ) {
  Meteor.publish( "messagesSub", function(){
    var self = this;
    self.added ( "messages", "madeUpId1", { m: "This is not from a database"} ); //messages is the collection that will be published to
    self.ready();
  });
}

There is an example in meteor docs explained here and another example here. I also have an example that shares data between clients without ever using a database just to teach myself how the publish and subscribe works. Nothing used but basic meteor.

Answer 3

It's possible to use Meteor's livedata package (their DDP implementation) without the need of a database on the server. This was demoed by Avital Oliver and below I'll point out the pertinent part.

The magic happens here:

if (Meteor.isServer) {
    TransientNotes = new Meteor.Collection("transientNotes", {connection: null});
    Meteor.publish("transientNotes", function () {
        return TransientNotes.find();
    });
}

if (Meteor.isClient) {
    TransientNotes = new Meteor.Collection("transientNotes");
    Meteor.subscribe("transientNotes");
}

Setting connection: null specifies no connection (see Meteor docs).

---

Akshat suggested using streams. I'm unable to reply to his comment due to lack of reputation, so I will put this here. The package he links to is no longer actively maintained (see author's tweet). I recommend using the yuukan:streamy (look it up on Atmosphere) package instead or use the underlying SockJS lib used in Meteor itself—you can learn how to do this by going through the Meteor code and look how Meteor.server.Stream_server, and Meteor.connection._stream are used, which is what the Streamy package does.

I tested an implementation of the Streamy chat example and found performance to be negligibly different but this was only on rudimentary tests. Using the first approach you get the benefits of the minimongo implementation (e.g. finding) and Meteor reactivity. Reactivity is possible with Streamy, though is does through things like using ReactiveVar's.

Answer 4

There is a way! At least theoretically The protocol used by Meteor to sync between client and server is called DDP. The spec is here

And although there some examples here and here of people implementing their own DDP clients, I'm afraid haven't seen examples of implementations of DDP servers. But I think the protocol is straightforward and would guess it wouldn't be so hard to implement.