How can I get data from 'ravi' file?

Question

What ravi file is:

A RAVI file is a video file created by thermal imaging software, such as Micro-Epsilon TIM Connect or Optris PIX Connect. It contains video captured by thermal cameras and is saved in a format similar to the Audio Video Interleave (.AVI) format. RAVI files also store radiometric information, such as temperature and measurement area information collected by the thermal camera.

My issue:

I have to work with data from the ravi file. I need the temperature value for the pixels (Or the maximum temperature of the frame is enough for me). I would like to check the maximum temperature on a certain frame. The final result would be a report which contains the maximum temperature values on frames (It would be a graph). It is easy to check and process with Micro-Epsilon TIM Connect or Optris PIX Connect tools but I am not able to use them (I have to write an own one).

My questions:

• How can I get the data from ravi file (Actually I need only the temperature values)?
• Is there any converter to convert ravi file to another (It is not relevant if I can get the data from ravi file)?

Note:

• The Python language is the preferred but I am open for every idea.
• I have to work with the ravi files and I am not able to record new files or modify the recording.
• I have found a site which provides a SDK for this type of camera but it is not clear for me that get data from ravi file is possible. Link to libirimager2 documentation: libirimager2
• If I play the ravi file with a media player then it says the used codeck is: Uncompressed packed YUV 4:2:2 (You can see the getting stream below)

If I parse it with OpenCV or play with a media player, I can see something stream. But I am not sure how I can get the temperature...

CV2 code:

import cv2

cap = cv2.VideoCapture("my_test.ravi")

if not cap.isOpened():
    print("Error opening video stream or file")

while cap.isOpened():
    ret, frame = cap.read()
    if ret:
        cv2.imshow('Frame', frame)
    if cv2.waitKey(25) & 0xFF == ord('q'):
        break

cap.release()
cv2.destroyAllWindows()

Getting stream:

(I see the same "pink and green" stream in a simple media player as well.)

Stream in the official software:

ravi file in HexEditor:

I have found a site about AVI video format. You can see below some lines from begging of my file, perhaps it can help.

00000000  52 49 46 46  F8 B1 C6 3F   41 56 49 20  4C 49 53 54     RIFF...?AVI LIST
00000010  CC 7F 00 00  68 64 72 6C   61 76 69 68  38 00 00 00     ....hdrlavih8...
00000020  12 7A 00 00  44 FF DD 00   00 02 00 00  10 08 00 00     .z..D...........
00000030  44 6D 00 00  00 00 00 00   01 00 00 00  08 65 09 00     Dm...........e..
00000040  80 02 00 00  E1 01 00 00   00 00 00 00  00 00 00 00     ................
00000050  00 00 00 00  00 00 00 00   4C 49 53 54  74 7E 00 00     ........LISTt~..
00000060  73 74 72 6C  73 74 72 68   38 00 00 00  76 69 64 73     strlstrh8...vids
00000070  59 55 59 32  00 00 00 00   00 00 00 00  00 00 00 00     YUY2............
00000080  B4 C4 04 00  80 96 98 00   00 00 00 00  A4 50 00 00     .............P..
00000090  08 65 09 00  00 00 00 00   00 00 00 00  00 00 00 00     .e..............
000000A0  00 00 00 00  73 74 72 66   28 00 00 00  28 00 00 00     ....strf(...(...
000000B0  80 02 00 00  E1 01 00 00   01 00 10 00  59 55 59 32     ............YUY2
000000C0  00 65 09 00  60 00 00 00   60 00 00 00  00 00 00 00     .e..`...`.......
000000D0  00 00 00 00  69 6E 64 78   F8 7D 00 00  04 00 00 00     ....indx.}......
000000E0  06 00 00 00  30 30 64 62   00 00 00 00  00 00 00 00     ....00db........

Testing materials:

If you download the PIX Connect Rel. 3.6.3046.0 Software from http://infrarougekelvin.com/en/optris-logiciel-eng/ site, you can find several ravi files in the "Samples" folder inside zip.

Additional info from an official documentation:

Software for thermoIMAGER TIM Infrared camera documentation

Video sequences can both be saved as a radiometric file (RAVI) or as a non-radiometric file (AVI). RAVI files contain all temperature as well as measure area information.

If Radiometric Recording, see Chap. 5.6.2, is not activated the images will be saved as standard AVI file only containing color information. A later conversion of a RAVI file into an AVI file and vice versa is not possible

Update:

I have tried to use the PyAV module to get data. This module is able to handle the yuyv422 format. I got the same "green-pink" stream and I was not able to get the temperature from it...

Used code:

# coding=utf-8
import av
import os


ravi_path = "Brake disc.ravi"
container = av.open(ravi_path)
stream = container.streams.video[0]
stream.codec_context.skip_frame = 'NONKEY'
tgt_path = "frames"
if not os.path.isdir(tgt_path):
    os.makedirs(tgt_path)
for frame in container.decode(stream):
    tgt_filename = os.path.join(tgt_path, 'frame-{:09d}.jpg'.format(frame.pts))
    print(frame, tgt_filename)
    frame.to_image().save(tgt_filename, quality=80)

The output of script:

>>> python ravi_test2.py 
(<av.VideoFrame #0, pts=0 yuyv422 160x121 at 0x7f501bfa8598>, 'frames/frame-000000000.jpg')
(<av.VideoFrame #1, pts=1 yuyv422 160x121 at 0x7f501bfa8600>, 'frames/frame-000000001.jpg')
(<av.VideoFrame #2, pts=2 yuyv422 160x121 at 0x7f5018e0fdb8>, 'frames/frame-000000002.jpg')
(<av.VideoFrame #3, pts=3 yuyv422 160x121 at 0x7f501bfa8598>, 'frames/frame-000000003.jpg')
(<av.VideoFrame #4, pts=4 yuyv422 160x121 at 0x7f501bfa8600>, 'frames/frame-000000004.jpg')
(<av.VideoFrame #5, pts=5 yuyv422 160x121 at 0x7f5018e0fdb8>, 'frames/frame-000000005.jpg')

Answer 1

I don't know your camera make, but expect the video file to contain raw sensor values as 16-bit unsigned int, which is maybe just named YUV422 in the video header, because they fit the same 16 bits per pixel.

These values you can convert to real valued temperature via a particular non-linear calibration curve. If the RAVI-format is a single file format (as opposed to some legacy IR-cameras with raw-AVI + calibration table) then you should find the location of the few floating point constants and/or table, which make up the equation.

It's possible to reverse engineer the logic, but better ask the correct equation from the manufacturer. For example, what you find on the internet, could be just a legacy version of the calibration curve. Most manufacturers offer calibration libraries together with their devices. Some, out-of-product cycle devices could be a pain to negotiate, but you should get at least a white paper on the topic.

If you use OpenCV, you need to read YUV422-frames raw (16bpp, not 24bpp) and just reinterpret their context as uint16 before applying look up table.

// sample C++ code employing private content of OpenCV library
// Particularly container_avi.private.hpp and container_avi.cpp
void mainactual()
{
    cv::AVIReadContainer reader;
    reader.initStream(cv::String("C:/tello/intro2.avi"));
    cv::frame_list frames;
    // initializes the stream
    reader.parseRiff( frames );
    std::cout << "Number of frames: " << frames.size() << std::endl;
    int w=reader.getWidth();
    int h=reader.getHeight();
    std::cout << "size " << cv::Size(w,h) << std::endl;
    // a frame in the middle
    cv::frame_iterator it=frames.begin() + frames.size()/2;

    std::vector< char> data = reader.readFrame( it );
    // In your case, data here is supposed to be 
    // uncompressed YUV422 which is w * h * 2 bytes per frame
    // You might need to modify 
    // bool AVIReadContainer::parseStrl(char stream_id, Codecs codec_)
    // to accept your FCC
    //
    //
    //if ( data.size()!=w*h*2 )
    //{
    //  // error
    //}

    // My video is MJPEG, so I'm confident to just to decode it
    cv::Mat img = cv::imdecode( data, cv::IMREAD_UNCHANGED );
    cv::imshow("image", img ); // looks fine
    cv::waitKey( 0 );
    reader.close();
}

EDIT: Tested brake disk.ravi, looks like below. Modified the parser to accept uncompressed YUV2 format and added a hack according to https://learn.microsoft.com/en-us/previous-versions/windows/desktop/api/Aviriff/ns-aviriff-avioldindex

dwOffset

Specifies the location of the data chunk in the file. The value should be specified as an offset, in bytes, from the start of the 'movi' list; however, in some AVI files it is given as an offset from the start of the file.

Not sure what the scrabble is, but looks like a brake disc.

    cv::Mat img;
    if ( data.size()==w*h*2 )
    {
        std::cout << data.size() << " " << w*h*2 << std::endl;
        cv::Mat t( h, w, CV_16UC1, &data[0] );
        // img(y,x) = (float)t(y,x)/10.0 - 100.0
        t.convertTo( img, CV_32F, 0.1, -100.0 );
    }else
        return;
    double mi,ma;
    cv::minMaxLoc( img, &mi, &ma );
    std::cout << "range: [" << mi << ", " << ma << "]" << std::endl;
    cv::Mat gray;
    img.convertTo( gray, CV_8U ); // [0, 255] range saturated
    cv::Mat bigger;
    cv::resize(gray,bigger,cv::Size(4*w,4*h),0,0,cv::INTER_LINEAR );
    cv::Mat jet;
    cv::applyColorMap( bigger, jet, cv::COLORMAP_JET );
    cv::imshow("image", jet ); // looks fine
    cv::waitKey( 0 );
    reader.close();

Answer 2

What I have found out on some footage from a IR-only camera:

• The frames have an extra first line which holds per-frame metadata. It should be discarded for display.
• The image data buffer frames hold 16-bit values (two bytes per pel, low byte first) related to the temperature. I could fit a 4-order polynom with <0.01C residuals, but it's not exactly linear. It is probably logarithmic, not polynomial.
• Note that according to micro-epsilon documentation, there could be up to 3 lines of metadata. The format is probably camera-dependent, and I was not able to interpret the first-line metadata.

I do not know if all frames from the same .ravi have the same calibration. Calibration metadata line is very similar from one frame to the next, only three 16-bit values differ slightly (takes two different values), and one increments by one at each frame.

The float t = (float)data[i] / 10.f - 100.f formula does not really yield matching temperatures.

Conclusion: only using the provided libirimager can work out the calibration metadata. It could be vendor-depedent too...