Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

PubNub: What is the right way to log all published message to my db

Tags:

pubnub

What is the right way to log every published message and save it to my server db.

There are two options which I can think of:

  1. Use PubNub function After Publish event and forward the message to a dedicated logger channel. the server will subscribe to the channel and save the arrived messages to db. Here rise another question: when I forward the message to another channel in the PubNub function will it also trigger the PubNub function?
  2. use PubNub XHR request/response function and call to a rest API on my server with the published message and save it to db

What is the best practice concerning performance and cost?

like image 460
tomn Avatar asked Feb 07 '18 13:02

tomn


1 Answers

Save PubNub Messages to your Private Database

We wrote an article that talks about the right way to log JSON Messages to a Private Database.

While many approaches exist. One is best. Using PubNub Functions. You will asynchronously save messages reliably to your Database. Using an OnAfter Publish Event. Your database needs to be accessible via a secured HTTPS endpoint.

PubNub does not index your messages using FTS Indexing; at time of writing. You may want Full Text Search Indexing using your database or use an API provider like https://www.algolia.com/ for full text searching.

Data is valuable. AI and ML allows you to create insight from your data using Tensorflow. You may want to run data analysis for the message content. Using EMR / Hadoop or other big data analysis software.

You will use PubNub Functions to save your message asynchronously into your database system easily by following these steps.

PubNub Save Message to Database

To get started is easy. Assuming you already have a stream of messages being published to a PubNub data channel. Follow these easy steps. Successfully you will create a realtime function that is triggered every Publish event.

  1. Collect a list of channel(s). You may be interested in ALL channels *.
  2. In your Account Dashboard find your app, then click Functions.
  3. Create a new Module for your specified API key set.
  4. Create a new OnAfter event handler on * channel.
  5. Use the following example code to save, asynchronously, your messages to your database.
  6. Make sure to modify the URL/Parameters to fit your needs.

Asynchronous HTTPS Save Message Function

// Request Handler
export default request => { 
    return save(request).then( () => request.ok() );
}

// Async Logging/Save of JSON Messages
function save( data, retry=3 ) {
    const xhr  = require('xhr');
    const post = { method : "POST", body : request.message };
    const url  = "https://my.company.com/save"; // <-- CHANGE URL HERE

    // save message asynchronously
    return xhr.fetch( url, post ).then( serverResponse => {
        // Save Success! 
    }).catch( err => {
        // Retry
        if (retry > 0) save( data, --retry );
    });
}
like image 62
Stephen Blum Avatar answered Oct 15 '22 12:10

Stephen Blum