Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

How to deal with streaming data in PHP?

There is a family of methods (birddog, shadow, and follow)in the Twitter API that opens a (mostly) permanent connection and allows you to follow many users. I've run the sample connection code with cURL in bash, and it works nicely: when a user I specify writes a tweet, I get a stream of XML in my console.

My question is: how can I access data with PHP that isn't returned as a direct function call, but is streamed? This data arrives sporadically and unpredictably, and it's not something I've ever dealt with nor do I know where to begin looking for answers. Any advice and descriptions of libraries or pitfalls would be appreciated.

like image 289
Alex Mcp Avatar asked Jul 31 '09 18:07

Alex Mcp


People also ask

How do you handle streaming data?

There are three ways to deal with streaming data: batch process it at intervals ranging from hours to days, process the stream in real time, or do both in a hybrid process. Batch processing has the advantage of being able to perform deep analysis, including machine learning, and the disadvantage of having high latency.

What is data stream in PHP?

PHP Stream Introduction Streams are the way of generalizing file, network, data compression, and other operations which share a common set of functions and uses. In its simplest definition, a stream is a resource object which exhibits streamable behavior.

Which database is best for streaming data?

Apache Kafka It is the practice to capture data in real-time in a stream of events from sources like sensors, mobile devices, databases, and software applications. Apache Kafka combines the following capabilities for event streaming in a highly scalable, fault-tolerant, flexible, and secure manner.

How is streaming data stored?

Streaming data processing requires two layers: a storage layer and a processing layer. The storage layer needs to support record ordering and strong consistency to enable fast, inexpensive, and replayable reads and writes of large streams of data.


1 Answers

fopen and fgets

<?php
$sock = fopen('http://domain.tld/path/to/file', 'r');
$data = null;
while(($data = fgets($sock)) == TRUE)
{
    echo $data;
}
fclose($sock);

This is by no means great (or even good) code but it should provide the functionality you need. You will need to add error handling and data parsing among other things.

like image 104
UnkwnTech Avatar answered Oct 05 '22 00:10

UnkwnTech