Python 3.5, GTK+ 3, Glade and OpenCV

I’ve just spent an hour or so figuring out how to display an OpenCV image in a GTK+ 3 window that’s created through a Glade UI using Python 3.  Since it’s not at all obvious even where to find the documentation, I’m writing it down here.

Background – Python 3 and GTK+

Time was, to use GTK in Python you installed PyGTK.  Those days are gone.  What we have now is called GObject Introspection – or ‘gi’.  What it does is pretty cool – it can expose any GObject-based library in Python.  Any new GObject-based library that’s written is immediately available in Python.  Just like that.

What’s really really dumb about it is calling it ‘gi’.  Try Googling that!

So, here’s where the documentation is: https://lazka.github.io/pgi-docs/.  Once you’ve found the documentation, it’s pretty easy to use.  Finding it is the hard part.

So, show me how to do it

Here’s code that takes an OpenCV feed from a webcam and displays it in a Glade UI.  First, the Glade file:

<?xml version="1.0" encoding="UTF-8"?>
<!-- Generated with glade 3.18.3 -->
<interface>
  <requires lib="gtk+" version="3.12"/>
  <object class="GtkWindow" id="window1">
    <property name="can_focus">False</property>
    <signal name="delete-event" handler="onDeleteWindow" swapped="no"/>
    <child>
      <object class="GtkBox" id="box1">
        <property name="visible">True</property>
        <property name="can_focus">False</property>
        <property name="orientation">vertical</property>
        <child>
          <object class="GtkToggleButton" id="greyscaleButton">
            <property name="label" translatable="yes">Greyscale</property>
            <property name="visible">True</property>
            <property name="can_focus">True</property>
            <property name="receives_default">True</property>
            <signal name="toggled" handler="toggleGreyscale" swapped="no"/>
          </object>
          <packing>
            <property name="expand">False</property>
            <property name="fill">True</property>
            <property name="position">0</property>
          </packing>
        </child>
        <child>
          <object class="GtkImage" id="image">
            <property name="visible">True</property>
            <property name="can_focus">False</property>
            <property name="stock">gtk-missing-image</property>
          </object>
          <packing>
            <property name="expand">False</property>
            <property name="fill">True</property>
            <property name="position">1</property>
          </packing>
        </child>
      </object>
    </child>
  </object>
</interface>

The main thing to note here is that we’re using a GtkImage object to display the video. Each frame, we’ll replace the GtkImage’s image data with the frame from the camera. I’ve also added a button to switch between greyscale and colour. Note that the developers are all Americans and so spell ‘grey’ and ‘colour’ wrong.

And here’s the Python code:

import cv2
import numpy as np
import gi

gi.require_version('Gtk', '3.0')
from gi.repository import Gtk, Gdk, GLib, GdkPixbuf

cap = cv2.VideoCapture(1)

builder = Gtk.Builder()
builder.add_from_file("test.glade")

greyscale = False

class Handler:
    def onDeleteWindow(self, *args):
        Gtk.main_quit(*args)

    def toggleGreyscale(self, *args):
        global greyscale
        greyscale = ~ greyscale

window = builder.get_object("window1")
image = builder.get_object("image")
window.show_all()
builder.connect_signals(Handler())

def show_frame(*args):
    ret, frame = cap.read()
    frame = cv2.resize(frame, None, fx=2, fy=2, interpolation = cv2.INTER_CUBIC)
    if greyscale:
        frame = cv2.cvtColor(frame, cv2.COLOR_BGR2GRAY)
        frame = cv2.cvtColor(frame, cv2.COLOR_GRAY2RGB)
    else:
        frame = cv2.cvtColor(frame, cv2.COLOR_BGR2RGB)

    frame = cv2.cvtColor(frame, cv2.COLOR_BGR2RGB)

    pb = GdkPixbuf.Pixbuf.new_from_data(frame.tostring(),
                                        GdkPixbuf.Colorspace.RGB,
                                        False,
                                        8,
                                        frame.shape[1],
                                        frame.shape[0],
                                        frame.shape[2]*frame.shape[1])
    image.set_from_pixbuf(pb.copy())
    return True

GLib.idle_add(show_frame)
Gtk.main()

Things to note here:

• It’s quite important to handle the window’s delete_event signal. Otherwise it can be quite difficult to kill the program (Ctrl+C doesn’t work; try Ctrl+Z and then kill -9 %1).
• I’m resizing the video to twice it’s native resolution.
• To convert to greyscale, I first convert BGR to greyscale and then greyscale to RGB. GTK+ can apparently only handle the RGB colourspace, so you need to end up there one way or another. OpenCV natively generates BGR, not RGB, so even to display colour you need to do a conversion.
• To get the data into a form that GtkImage understands, we first convert the numpy ndarray to a byte array using .tostring(). We then use GdkPixbuf.Pixbuf.new_from_data to convert this to a pixbuf. The False argument is to say there is no alpha channel. 8 is the only bit depth supported. frame.shape[1] is the image width and frame.shape[2] is the image height, and the last argument is the number of bytes in one row of the image (ie. the number of channels times the width in pixels).
• We don’t display the pixbuf directly but instead display a copy of it. This gets around a wrinkle in the memory management which would otherwise require us to manually clean up the pixbuf object when we’re done with it.
• The function gets called by the GTK idler; GLib.idle_add(show_frame) is adding the function to the list of functions called when idle.
• You have to return True from idle functions or they don’t get called again.

That’s it!