How to assign memory images to a DLIB array2d or image?

Question

This is likely more of a c++ question, but it does relate to DLIB. I am using another library to load images into memory (Leadtools). I want to replace the "load_image" function with assignments, but I am having trouble figuring out how to do so. I have all the information and memory allocated I need from my load, such as pixel bit depth, width, height, width_step, rgb/bgr order, and a pointer to the data.

So in the sample, there is:

array2d<bgr_pixel> img;

I can do this:

img.set_size(mywidth, myheight);

But (assuming the data mypointer is bgr contiguous char * bytes):

img.data = mydatapointer;  

does not work - there is no img.data, plus I can't figure out how to use image_view.

What is the right way to setup that data pointer? Note that I do NOT want to duplicate memory :-)

Is there anything else that needs setting in order to simulate the load_image and have the correct structure to use within dlib?

TIA

Answer 1

Based on Davis King's comment.

To avoid copying data from your own image structure to dlib internal format, dlib allows you to implement a generic image interface that will allows dlib to use directly your image. The interface that needs to be implemented looks like this:

    In dlib, an "image" is any object that implements the generic image interface.  In
    particular, this simply means that an image type (let's refer to it as image_type
    from here on) has the following seven global functions defined for it:
        - long        num_rows      (const image_type& img)
        - long        num_columns   (const image_type& img)
        - void        set_image_size(      image_type& img, long rows, long cols)
        - void*       image_data    (      image_type& img)
        - const void* image_data    (const image_type& img)
        - long        width_step    (const image_type& img)
        - void        swap          (      image_type& a, image_type& b)
     And also provides a specialization of the image_traits template that looks like:
        namespace dlib
        {
            template <> 
            struct image_traits<image_type>
            {
                typedef the_type_of_pixel_used_in_image_type pixel_type;
            };
        }

You can find more details here: http://dlib.net/dlib/image_processing/generic_image.h.html

Answer 2

I found the answer by spending a lot more time with DLIB. Has nothing to do with C++. I rarely use templates so it was overwhelming at first.

What one needs to do is create some headers that follow the generic image model, as suggested. I chose to use array2d as the guide. By creating similar headers, along with adding a function to map to the image I have in memory, I was able to use the image data without duplicating it.

BTW the one gotcha, be careful about width_step (number of bytes per line). I had to set that since many images I had were padded. See code below.

To save someone a lot of time, here's what I did, naming the new structure arrayEd. Just used arrayEd instead of array3d.

If anyone has any comments on this code, it's welcomed :-)

arrayEd.h:

#pragma once
#include "arrayEd_kernel.h"
#include "serialize_pixel_overloads_arrayEd.h"
#include "arrayEd_generic_image.h"

For serialize_pixel_overloads_arrayEd.h, just replaced array3d with arrayEd.

arrayEd_kernel.h: (the bulk of changes). Added function setup and passed it my memory image class, along with a couple of variables to keep track of things.

#pragma once
// Copyright (C) 2006  Davis E. King ([email protected])
// License: Boost Software License   See LICENSE.txt for the full license.

#include "C:\VStudioProjects\dlib-master\dlib/algs.h"
#include "C:\VStudioProjects\dlib-master\dlib/interfaces/enumerable.h"
#include "C:\VStudioProjects\dlib-master\dlib/serialize.h"
#include "C:\VStudioProjects\dlib-master\dlib/geometry/rectangle.h"

namespace dlib
{
template <
    typename T,
    typename mem_manager = default_memory_manager
>
    class arrayEd : public enumerable<T>
{

    /*!
    INITIAL VALUE
    - nc_ == 0
    - nr_ == 0
    - data == 0
    - at_start_ == true
    - cur == 0
    - last == 0

    CONVENTION
    - nc_ == nc()
    - nr_ == nc()
    - if (data != 0) then
    - last == a pointer to the last element in the data array
    - data == pointer to an array of nc_*nr_ T objects
    - else
    - nc_ == 0
    - nr_ == 0
    - data == 0
    - last == 0


    - nr_ * nc_ == size()
    - if (cur == 0) then
    - current_element_valid() == false
    - else
    - current_element_valid() == true
    - *cur == element()

    - at_start_ == at_start()
    !*/


    class row_helper;
public:

    // These typedefs are here for backwards compatibility with older versions of dlib.
    typedef arrayEd kernel_1a;
    typedef arrayEd kernel_1a_c;

    typedef T type;
    typedef mem_manager mem_manager_type;

    // -----------------------------------
    bool bIsAllocatedHere;  //ec this is used to tell whether or not the memory came from here - if so, we clean it up.
    long lWidthStep;   // number of bytes in a row. This comes from leadtools and is set when we setup the image in setup()

    class row
    {
        /*!
        CONVENTION
        - nc_ == nc()
        - for all x < nc_:
        - (*this)[x] == data[x]
        !*/

        friend class arrayEd<T, mem_manager>;
        friend class row_helper;

    public:
        long nc(
        ) const {
            return nc_;
        }

        const T& operator[] (
            long column
            ) const
        {
            // make sure requires clause is not broken
            DLIB_ASSERT(column < nc() && column >= 0,
                "\tconst T& arrayEd::operator[](long column) const"
                << "\n\tThe column index given must be less than the number of columns."
                << "\n\tthis:    " << this
                << "\n\tcolumn:  " << column
                << "\n\tnc(): " << nc()
            );

            return data[column];
        }

        T& operator[] (
            long column
            )
        {
            // make sure requires clause is not broken
            DLIB_ASSERT(column < nc() && column >= 0,
                "\tT& arrayEd::operator[](long column)"
                << "\n\tThe column index given must be less than the number of columns."
                << "\n\tthis:    " << this
                << "\n\tcolumn:  " << column
                << "\n\tnc(): " << nc()
            );

            return data[column];
        }

    private:

        row(T* data_, long cols) : data(data_), nc_(cols) {}

        T* data;
        long nc_;


        // restricted functions
        row() {}
        row& operator=(row&);
    };

    // -----------------------------------

    arrayEd(
    ) :
        data(0),
        nc_(0),
        nr_(0),
        cur(0),
        last(0),
        at_start_(true),
        bIsAllocatedHere(false),
        lWidthStep(0)
    {
    }

    arrayEd(
        long rows,
        long cols
    ) :
        data(0),
        nc_(0),
        nr_(0),
        cur(0),
        last(0),
        at_start_(true),
        bIsAllocatedHere(false),
        lWidthStep(0)
    {
        // make sure requires clause is not broken
        DLIB_ASSERT((cols >= 0 && rows >= 0),
            "\t arrayEd::arrayEd(long rows, long cols)"
            << "\n\t The arrayEd can't have negative rows or columns."
            << "\n\t this: " << this
            << "\n\t cols: " << cols
            << "\n\t rows: " << rows
        );

        set_size(rows, cols);
    }

#ifdef DLIB_HAS_RVALUE_REFERENCES
    arrayEd(arrayEd&& item) : arrayEd()
    {
        swap(item);
    }

    arrayEd& operator= (
        arrayEd&& rhs
        )
    {
        swap(rhs);
        return *this;
    }
#endif

    virtual ~arrayEd(
    ) {
        clear();
    }

    long nc(
    ) const {
        return nc_;
    }

    long nr(
    ) const {
        return nr_;
    }

    row operator[] (
        long row_
        )
    {
        // make sure requires clause is not broken
        DLIB_ASSERT(row_ < nr() && row_ >= 0,
            "\trow arrayEd::operator[](long row_)"
            << "\n\tThe row index given must be less than the number of rows."
            << "\n\tthis:     " << this
            << "\n\trow_:      " << row_
            << "\n\tnr(): " << nr()
        );

        return row(data + row_*nc_, nc_);
    }

    const row operator[] (
        long row_
        ) const
    {
        // make sure requires clause is not broken
        DLIB_ASSERT(row_ < nr() && row_ >= 0,
            "\tconst row arrayEd::operator[](long row_) const"
            << "\n\tThe row index given must be less than the number of rows."
            << "\n\tthis:     " << this
            << "\n\trow_:      " << row_
            << "\n\tnr(): " << nr()
        );

        return row(data + row_*nc_, nc_);
    }

    void swap(
        arrayEd& item
    )
    {
        exchange(data, item.data);
        exchange(nr_, item.nr_);
        exchange(nc_, item.nc_);
        exchange(at_start_, item.at_start_);
        exchange(cur, item.cur);
        exchange(last, item.last);
        pool.swap(item.pool);
    }

    void clear(
    )
    {
        if (data != 0)
        {
            // ec we might manage this memory at the leadtools level 
            if (bIsAllocatedHere)
                pool.deallocate_array(data);
            nc_ = 0;
            nr_ = 0;
            data = 0;
            at_start_ = true;
            cur = 0;
            last = 0;
            bIsAllocatedHere = false;
        }
    }

    void set_size(
        long rows,
        long cols
    );

    bool at_start(
    ) const {
        return at_start_;
    }

    void reset(
    ) const {
        at_start_ = true; cur = 0;
    }

    bool current_element_valid(
    ) const {
        return (cur != 0);
    }

    const T& element(
    ) const
    {
        // make sure requires clause is not broken
        DLIB_ASSERT(current_element_valid() == true,
            "\tconst T& arrayEd::element()()"
            << "\n\tYou can only call element() when you are at a valid one."
            << "\n\tthis:    " << this
        );

        return *cur;
    }

    T& element(
    )
    {
        // make sure requires clause is not broken
        DLIB_ASSERT(current_element_valid() == true,
            "\tT& arrayEd::element()()"
            << "\n\tYou can only call element() when you are at a valid one."
            << "\n\tthis:    " << this
        );

        return *cur;
    }

    bool move_next(
    ) const
    {
        if (cur != 0)
        {
            if (cur != last)
            {
                ++cur;
                return true;
            }
            cur = 0;
            return false;
        }
        else if (at_start_)
        {
            cur = data;
            at_start_ = false;
            return (data != 0);
        }
        else
        {
            return false;
        }
    }

    unsigned long size(
    ) const {
        return static_cast<unsigned long>(nc_ * nr_);
    }

    long width_step(
    ) const
    {
        if (lWidthStep == 0)  // inc ase image not allocated by leadtools
            return nc_ * sizeof(T);
        else
            return lWidthStep;
    }

    // eds setup fcn
    int setup(CpwImage& img)
    {
        data = (T *)img.AccessBitmap();
        nc_ = img.GetWidth();
        nr_ = img.GetHeight();
        cur = last = 0;
        at_start_ = (true);
        lWidthStep = img.GetBytesPerLine();
        if (data != 0)
            return 1;
        return 0;
    }

private:


    T* data;
    long nc_;
    long nr_;

    typename mem_manager::template rebind<T>::other pool;
    mutable T* cur;
    T* last;
    mutable bool at_start_;

    // restricted functions
    arrayEd(arrayEd&);        // copy constructor
    arrayEd& operator=(arrayEd&);    // assignment operator

};

// ----------------------------------------------------------------------------------------

template <
    typename T,
    typename mem_manager
>
    inline void swap(
        arrayEd<T, mem_manager>& a,
        arrayEd<T, mem_manager>& b
    ) {
    a.swap(b);
}


template <
    typename T,
    typename mem_manager
>
    void serialize(
        const arrayEd<T, mem_manager>& item,
        std::ostream& out
    )
{
    try
    {
        // The reason the serialization is a little funny is because we are trying to
        // maintain backwards compatibility with an older serialization format used by
        // dlib while also encoding things in a way that lets the arrayEd and matrix
        // objects have compatible serialization formats.
        serialize(-item.nr(), out);
        serialize(-item.nc(), out);

        item.reset();
        while (item.move_next())
            serialize(item.element(), out);
        item.reset();
    }
    catch (serialization_error e)
    {
        throw serialization_error(e.info + "\n   while serializing object of type arrayEd");
    }
}

template <
    typename T,
    typename mem_manager
>
    void deserialize(
        arrayEd<T, mem_manager>& item,
        std::istream& in
    )
{
    try
    {
        long nr, nc;
        deserialize(nr, in);
        deserialize(nc, in);

        // this is the newer serialization format
        if (nr < 0 || nc < 0)
        {
            nr *= -1;
            nc *= -1;
        }
        else
        {
            std::swap(nr, nc);
        }

        item.set_size(nr, nc);

        while (item.move_next())
            deserialize(item.element(), in);
        item.reset();
    }
    catch (serialization_error e)
    {
        item.clear();
        throw serialization_error(e.info + "\n   while deserializing object of type arrayEd");
    }
}

// ----------------------------------------------------------------------------------------
// ----------------------------------------------------------------------------------------
// member function definitions
// ----------------------------------------------------------------------------------------
// ----------------------------------------------------------------------------------------

template <
    typename T,
    typename mem_manager
>
    void arrayEd<T, mem_manager>::
    set_size(
        long rows,
        long cols
    )
{
    // make sure requires clause is not broken
    DLIB_ASSERT((cols >= 0 && rows >= 0),
        "\tvoid arrayEd::set_size(long rows, long cols)"
        << "\n\tThe arrayEd can't have negative rows or columns."
        << "\n\tthis: " << this
        << "\n\tcols: " << cols
        << "\n\trows: " << rows
    );

    // set the enumerator back at the start
    at_start_ = true;
    cur = 0;

    // don't do anything if we are already the right size.
    if (nc_ == cols && nr_ == rows)
    {
        return;
    }

    if (data != 0)
    {
        DLIB_ASSERT(0, "Something wrong - this csoi image type should not be REsetting size");
        return;
    }

    nc_ = cols;
    nr_ = rows;

    // free any existing memory
    if (data != 0)
    {
        pool.deallocate_array(data);
        data = 0;
    }

    // now setup this object to have the new size
    try
    {
        if (nr_ > 0)
        {
            data = pool.allocate_array(nr_*nc_);
            last = data + nr_*nc_ - 1;
            bIsAllocatedHere = true;
        }
    }
    catch (...)
    {
        if (data)
            pool.deallocate_array(data);

        data = 0;
        nc_ = 0;
        nr_ = 0;
        last = 0;
        throw;
    }
}
// ----------------------------------------------------------------------------------------

template <typename T>
struct is_arrayEd : public default_is_kind_value
{
    /*!
    - if (T is an implementation of array2d/array2d_kernel_abstract.h) then
    - is_array2d<T>::value == true
    - else
    - is_array2d<T>::value == false
    !*/
};

// ----------------------------------------------------------------------------------------

template <typename T, typename MM>
struct is_arrayEd <arrayEd<T, MM> >
{
    const static bool value = true;
};

// ----------------------------------------------------------------------------------------

}