RSS subscription implementation on express server

Question

I'm setting up a node.js server that subscribes to an RSS feed. When a new item is published to the feed, I want the server to parse that information and pass it to an API which will alert end users. Can I use feedparser as my subscriber?

I understand that this library creates an EventEmitter that fires actions. Is it possible to export this function and make it run in parallel with my Express application?

Taken from the parser examples:

const FeedParser = require('feedparser')
const request = require('request')

const subscriber = async () => {
  const req = request('https://www.reddit.com/.rss')
  const feedparser = new FeedParser()
  req.on('error', (error) => {
    console.log(error)
  })
  req.on('response', (r) => {
    const stream = this
    if (r.statusCode !== 200) {
      this.emit('error', new Error('bad status code'))
    } else {
      stream.pipe(feedparser)
    }
  })
  feedparser.on('readable', () => {
    // This is where the action is!
    const stream = this
    var item = ''
    while (item = stream.read()) {
      console.log(item)
    }
  })
  return feedparser
}

module.exports = {
   subscriber
 }

When I call this function, I am expecting the console to log new items but I am not getting any items. The reason why is unclear.

Bonus Question: Can I validate the subscription is working without having to wait for new items to be published? Is there an RSS tool or website that can simulate the behavior?

Answer 1

Couple of things I learned:

1. RSS Feeds don't send information. If you want to subscribe to one, you need to make the requests on an interval and parse new information: Subscribe to an RSS Feed
2. Any functionality that doesn't require handling incoming requests probably doesn't need to be used in your Express middleware, you can just start the RSS subscription interval immediately before/after starting your HTTP server:

var app = require('../app');
var http = require('http');
var port = normalizePort(process.env.PORT || '3000');
app.set('port', port);

/**
 * Create HTTP server.
 */

var server = http.createServer(app);
/**
 * Listen on provided port, on all network interfaces.
 */

server.listen(port);
server.on('error', onError);
server.on('listening', onListening);
/**
 * Start the subscriber
 */
var feed = new RssFeed({your rss feed url}, 60000); // 60000 = 60 second refresh rate
feed.start();

Also including an incomplete snippet of my implementation:

const Parser = require('rss-parser')
class RssSubscriber extends Parser {
  constructor (url, interval) {
    super()
    this.url = url
    this.interval = interval
    this.latestPublishTime = null
  }
  start () {
    // execute on an interval
    setInterval(async () => {
      try {
        console.log('Fetching feed...')
        const news = await this.parseFeed()
        await Promise.all(news.map(n => { return this.alert(n) }))
      } catch (err) {
        console.log('WARN: Error Encountered when fetching subscription items.')
      }
    }, this.interval)
  }