How can I work with a GLib.Array in Python?

Question

I am writing a plugin for Rhythmbox, wherein a signal raised is passing in an object of type GArray. The documentation for GLib Arrays shows me a few methods I am interested in, but am unable to access.

For example, g_array_index can get me the nth item in a GArray, but I am unable to call it. The GArray object doesn't show me any useful methods either.

To see what I mean, do this in a Python console:

from gi.repository.GLib import Array
x = Array()
dir(x)

Here is the output of dir(x)

['__class__', '__delattr__', '__dict__', '__doc__', '__eq__', '__format__', '__ge__', '__getattribute__', '__gt__', '__gtype__', '__hash__', '__info__', '__init__', '__le__', '__lt__', '__module__', '__ne__', '__new__', '__reduce__', '__reduce_ex__', '__repr__', '__setattr__', '__sizeof__', '__str__', '__subclasshook__', '__weakref__', '_free_on_dealloc', 'copy', 'data', 'len']

I see no methods in there read from the array, and nothing about the g_array_index or any other methods mentioned on the GLib Arrays documentation page. I also tried

    for a in x:
        print a

And also

list(x)

But I receive an error:

TypeError: 'Array' object is not iterable

Attempting x[0] gives this:

TypeError: 'Array' object does not support indexing

The len property gives the length of the array as is expected.

The data property gives this

How can I work with this GLib.Array that I am being passed?

I am running Python 2.7.4

Answer 1

Introduction

In this answer, I'll attempt to summarize what I've learned about this problem and provide a solution that can address some situations in which this problem arises. Unfortunately, in some cases, there seems to be no ready solution, though I think the Python GI module could be modified to enable workarounds (see Closing Thoughts, below).

Background

The core issue is that GArray is only a rather thin wrapper around malloc(), realloc(), and free(). Beyond that, it adds a couple features, such as ref-counting and built-in support for zero-termination. However, a notable feature it lacks is any knowledge of the array's element type! What this means is that the Python GI (GObject Introspection) module is unable to adapt an arbitrary GArray to work as a Python sequence type, without further information about what the GArray contains.

Official Method

The method of using GArrays that's supported by the gi module is to generate a .typelib file, which contains the additional information it needs, in order to adapt each specific GArray instance. Fortunately, there's a toolchain that exists to help you generate these files directly from your sourcecode.

To use this method, start by documenting your source code with comment blocks according to the format defined here:

• https://wiki.gnome.org/Projects/GObjectIntrospection/Annotations

Next, run the g-ir-scanner tool, in order to generate a .gir file. The docs for this tool can be found here:

• https://gi.readthedocs.io/en/latest/tools/g-ir-scanner.html

Finally, the g-ir-compiler tool can be used to create the .typelib file. It's documented here:

• https://gi.readthedocs.io/en/latest/tools/g-ir-compiler.html

A walk-through of this process is written up here:

• https://storageapis.wordpress.com/2014/07/25/minimalistic-example-of-the-glibs-gboxedtype-usage/

Here's one with a Javascript focus:

• http://helgo.net/simon/introspection-tutorial/index.xhtml

And I can vouch for the fact that it works. Once you generate a .typelib file providing the necessary details of your GArray, the gi module will provide a familiar sequence-style interface to it, so you can use it like a list.

Workarounds

Unfortunately, what you cannot do is use the gi framework to expose functions for working with under-specified GArrays that you might get from other APIs! If you try to pass one of these GArrays into your function, Python complains about it not being a sequence type.

In my case, I was writing a GStreamer app, in which a particular pipeline element was generating GstMessages that contained a couple GArray members. While I couldn't write accessor functions to directly read the contents of these members, I found I could write functions which took the GstStructure, then read the desired member and returned it as a fully-specified GArray that gi could adapt as a proper Python sequence.

References

For more details on GArray, see:

• https://developer.gnome.org/glib/stable/glib-Arrays.html
• https://github.com/GNOME/glib/blob/master/glib/garray.h
• https://github.com/GNOME/glib/blob/master/glib/garray.c

In particular, note that while garray.h defines a GArray as a struct containing only a data and len member, you can see in garray.c that this interface type is backed by struct _GRealArray, which contains an additional 6 members.

For further information about the GObject Introspection framework and the Python gi module, see:

• https://gi.readthedocs.io/en/latest/
• https://developer.gnome.org/gi/unstable/
• https://pygobject.readthedocs.io/en/latest/

Closing Thoughts

Finally, what PyGObject could do to enable workarounds for libraries you don't control, is to expose the data member as a Python bytes object, with a length equal to GArray.len * GRealArray.elt_size.