I'm trying to timestamp data from stdin (into a log file) using a Windows Batch file

Question

I'm collecting data from an API using a DOS port of wget to generate data for a log file (which will be analysed at a later date). The API provides all the information I need except a current time (it provides a time at the start of the stream of data but not again after that).

The API provides, typically 10 lines of data initially and then a line every 20-30 seconds.

I'm trying to timestamp this output and copy it to a log file - I don't mind if the timestamp is on the same line as the rest of the output or the line before.

I first started with this batchfile:

addtimes.bat:

@echo off >nul
:start
set /p input="":

echo %time% 
echo %input%

goto:start

(called as "wget..... | addtimes.bat > log.log")

However this dropped data comping in - the beginning of many lines of data were lost.

I've looked on here and realised I should use a for loop.

addtimes2.bat:

@echo off
cls
setlocal EnableDelayedExpansion
for /F "tokens=*" %%a in ('more') do ( 
echo !time! %%a )
)

I've tried with and without Enabling Delayed Expansion.

I don't seem to be able to pass information one line at a time with a different timestamp - all my lines get identical timestamps once I close the datastream.

Typical input data is of the form:

[1,"219","265",14528,1359031137000,1359031137000]
[1,"6594","358",18188,1359031019000,1359031019000]
[1,"690","94",15920,1359031534000,1359031534000]
[1,"25164","102",2129,1359031457000,1359031457000]


[1,"3488","329",2109,1359030868000,1359030868000]

[1,"37247","6",11506,1359031223000,1359031223000]

You may notice there are UTC times in the data but they are not the current time.

Answer 1

I don't believe you can get the result you want using pure native batch. The reason why all your times are the same is that the FOR /F loop does not process any lines of input until the entire input stream has been buffered (the command on the left of the pipe has finished). The FOR /F command waits until all the input has been received, and then dumps every line in one mad rush. If the input stream is large enough, you will get slight variations in time, but nothing that comes close to representing when the original command generated each line.

Here is a hybrid JScript/batch script that does what you want. It works fine as a straight JScript file, but then you need to explictly use CSCRIPT. The hybrid approach makes the utility more convenient.

Call it addtimes.bat and use it just as you were before.

@if (@X)==(@Y) @end /* Harmless hybrid line that begins a JScript comment

::************ Batch portion ***********
@echo off
cscript //E:JScript //nologo "%~f0"
exit /b 0

************* JScript portion **********/
while (!WScript.StdIn.AtEndOfStream) {
  var ts = new Date()
  var ms = (ts.getTime() % 1000)
  WScript.Stdout.WriteLine(
    ts.getFullYear() + "-" +
    ((ts.getMonth()<9)?"0":"") + (ts.getMonth()+1) + "-" +
    ((ts.getDate()<10)?"0":"") + ts.getDate() + " " +
    ((ts.getHours()<10)?"0":"") + ts.getHours() + ":" +
    ((ts.getMinutes()<10)?"0":"") + ts.getMinutes() + ":" +
    ((ts.getSeconds()<10)?"0":"") + ts.getSeconds() + "." +
    ((ms<10)?"00":((ms<100)?"0":"")) + ms + " - " +
    WScript.StdIn.ReadLine()
  );
}

EDIT

wmz has a very clever and dangerous solution. That solution can be simplified - There is no need to muck with Autorun.

Warning - as wmz said, the solution below can have very bad consequences if any line in the output starts with an executable command or program name! I do not recommend actually using this solution, but I find it very interesting.

(echo @prompt $D $T -$S & YourCommandHere )|cmd 2>nul|findstr /rbc:"../../.... ..:..:..\... - " >log.log

The FINDSTR pipe is added to strip out the CMD header info, the initial PROMPT command, and the unwanted blank line that CMD inserts after each "command". The FINDSTR regex may need to change to match the specifics of your chosen prompt and your locale.