dbus tutorial - part 2

Implementation overview

The first part introduced the dbus components and how to access them using Python.

The goal is to write a program that lists and stores the names of the files on a CD after it has been inserted.  The basic steps are:

1. Initialize dbus
2. Wait for the notification when a CD is inserted into the drive.
3. Wait for the notification when the volume is mounted by the (Gnome) automounter
4. Read the file list below that mount point and store the info.
5. Then perhaps eject the CD.
6. Go back to step 2

Initialize dbus: Starting dbus threads

In order to get the dbus communication running in the python application, the following threads have to be started before you connect to any bus:

import gobject
import dbus
import dbus.glib
...
gobject.threads_init()
dbus.glib.threads_init()

This piece of code uses Python threads in the background. 


HAL implementation specifics

In the first part we've seen how callbacks are hooked into the system:

bus = dbus.SystemBus()
bus.add_signal_receiver(device_added_callback,
                                 "DeviceAdded",
                                 "org.freedesktop.Hal.Manager",
                                 "org.freedesktop.Hal",
                                 "/org/freedesktop/Hal/Manager")
bus.add_signal_receiver(device_removed_callback,
                                 "DeviceRemoved",
                                 "org.freedesktop.Hal.Manager",
                                 "org.freedesktop.Hal",
                                 "/org/freedesktop/Hal/Manager")

When a CD is inserted DeviceAdded receives a string like this:

/org/freedesktop/Hal/devices/volume_label_backupcd1________

When the CD is ejected, DeviceRemoved is executed, receiving the same information.  I'll refer to this string as udi in the following code.

This string refers to an HAL object, and describes a volume i.e. the medium you inserted, not the actual hardware (i.e. the CDROM drive).  To get more information about this volume you have to get the HAL object via bus.get_object (as described before):

d_object = bus.get_object('org.freedesktop.Hal',udi)

Objects starting with /org/freedesktop/Hal/devices/... implement the interface org.freedesktop.Hal.Device, which in turn exposes (among others) the function GetAllProperties which returns a dictionary that contains further useful information:

d_interface = dbus.Interface(d_object,'org.freedesktop.Hal.Device')
properties = d_interface.GetAllProperties()

Interesting keys in this dictionary are:

• volume.label
  description: CD label
  typical value: empty, backup123
• volume.disc.has_data
  description: data disk
  typical value: 0, 1
  
• volume.disc.is_blank
  description: blank media
  typical value: 0, 1
• volume.mount_point
  description: CD mount point
  typical value: /media/backup123
• block.device
  description: linux device descriptor
  typical value: /dev/sr0
  
• storage.drive_type
  typical value: cdrom
  

See also HAL specification

To get a quick overview, type

lshal | more

in a shell.  (You will need the more command.)

The interface org.freedesktop.Hal.Device (implemented by all .../Hal/devices/ ) also contains the signal PropertyModified - which will be fired when properties change (in our example: the info volume.mount_point is empty when the device is added.  It becomes available later when the auto-mounter of the desktop framework has mounted the device.).  You can add a callback for this signal:

def device_added_hook(udi):
    ....
    bus.add_signal_receiver(property_modified_callback,
           "PropertyModified",
           "org.freedesktop.Hal.Device",
           "org.freedesktop.Hal",
           udi)

def property_modified_callback(numupdates,updates):
    ...

numupdates are the number of entries in a list of updates.  An "update" is a tuple with the following signature: (key_name, was_added, was_removed)

If a CD/DVD was mounted, the list of updates contains (among others) the following tuples:

('volume.mount_point', False, False)
('volume.is_mounted', False, False)

This means that the property was neither added (1st False) nor removed (2nd False), it was merely modified.  We don't get any information about the new value of the property and more importantly we don't get any information about the object (i.e. the device) that triggered it.

For this reason the method add_signal_receiver can be extended:

...
    bus.add_signal_receiver(property_modified_callback,
           "PropertyModified",
           "org.freedesktop.Hal.Device",
           "org.freedesktop.Hal",
           udi,
           path_keyword = "sending_device" )

The parameter path_keyword defines the name of an additional parameter (in this example sending_device) for the property_modified_callback which will contain the “path” - or in my parlance “the object” - that triggered the signal.  Now the property_modified_callback looks like this:

def property_modified_callback(numupdates, updates, sending_device = None):
    ...


Apart from path_keyword, add_signal_receiver can define even more additional parameters for the callback function:

• sender_keyword - for the connection name (usually in the internal numeric format)
• destination_keyword - None for broadcasts
• interface_keyword - interface name
• member_keyword - signal name
• path_keyword - object name

each followed by a string containing the name of the parameter in the callback function.  The variable will contains the appropriate information.

HAL implementation: Filter out events that are not CD related

DeviceAdded is called for every new device in the system (network devices, bluetooth devices, etc.), and the property_modified_callback is invoked if any property of any device changes.

In order to filter out the CD related calls, we have to check the properties for suitable keys.

First, we have to ensure that the object describes a volume.  For this we have to check if the property block.is_volume is present, and if so, that it is set to True.
The object will then have a property named info.parent, pointing back to the physical drive.

After retrieving the properties of that parent object, we first check block.is_volume, which has to be set to False, and then obtain the value of storage.drive_type, which is set to "cdrom", if the device is an optical drive.

...
# get object of the device that triggered the property_modified_callback
d_object = self.system_bus.get_object('org.freedesktop.Hal',sending_device)
d_interface = dbus.Interface(d_object,'org.freedesktop.Hal.Device')

# get their properties
d_properties = d_interface.GetAllProperties()

if not d_properties.has_key("block.is_volume"): return
if not d_properties["block.is_volume"]: return

parent_device = d_properties["info.parent"]
p_object = system_bus.get_object('org.freedesktop.Hal',parent_device)
p_interface = dbus.Interface(p_object,'org.freedesktop.Hal.Device')
p_properties = p_interface.GetAllProperties()
if not p_properties["block.is_volume"]:
   if p_properties["storage.drive_type"] == "cdrom":
       # further code here


Another approach is to get the list of all optical drives when the utility is launched and check the parent udi against this list:

system_bus = dbus.SystemBus()
hal_object = system_bus.get_object('org.freedesktop.Hal', '/org/freedesktop/Hal/Manager')
hal_manager = dbus.Interface(hal_object, 'org.freedesktop.Hal.Manager')
...
installed_optical_drives = hal_manager.FindDeviceStringMatch("storage.drive_type","cdrom")
...
if parent in installed_optical_drives:
   ...

Mount point is empty
At the time when the device is added volume.mount_point is empty.  That's why the signals DeviceAdded and DeviceRemoved are not important in our application.  You might want to use them anyway to display the state of your drive (e.g. disk inserted or nor) in the GUI.

The import part is the property_modified_callback.  After the checks described above we know the label of the CD and its mount point.  With this information we can obtain a file list and store it in any form we want.



The next part will show how DeviceKit.Disks handles these problems.

dbus tutorial - part 1

Basic dbus concepts

What is dbus?

dbus is part of most modern Linux distros.  It handles interprocess communication, either between desktop applications or between systems daemons and applications (e.g. a desktop framework like Gnome or KDE, or user applications).


Overview

The goal of this series of postings is the creation of a small desktop example application written in Python which reads the name of the files on a CD or DVD after the media has been inserted, and to store these names together with the CD label in a file.  The graphical user interface will be based on wxWidgets. 

The first part contains an introduction to the basic dbus components.
The second part discusses the implementation using the Hardware Abstraction Layer (HAL).
The following part shows the differences in implementation between HAL to the DeviceKit.Disks.


Prerequisites

Python: which is part of any current Linux distribution. (I'm using Python 2.6).
python-dbus: the dbus bindings for Python
python-wxgtk2.8: wxWidgets and its Python bindings (for the GUI)

Useful additions

python-wxglade: GUI builder for wxWidgets, creates Python source code (and C source code...)

d-feet: graphical display of dbus components

dbus components

In our example dbus is used to catch the notifications sent by the kernel when a CD or DVD has been inserted, as well as the notification when Gnome has mounted this medium. (There were times when you had to do this manually.)

In some places I will differ from the official dbus designations for these components to make their distinction easier.  In some cases different components share the same name.  This makes their close relation quite obvious, but it becomes difficult to determine if a given name refers to a connection or an interface, for example.


Buses

In dbus “buses” are at the root of a hierarchy.  In normal applications you encounter mostly the following two buses:

    bus = dbus.SessionBus()

for communication between desktop applications, and

    bus = dbus.SystemBus()

for communication with system programs and daemons like the NetworkManager, the ModemManager, the Hardware Abstraction Layer daemon (HAL) or DeviceKit, to name but a few.

When you insert a disc, a notification is routed via HAL or DeviceKit, depending on which daemon is installed.

Note: Ubuntu Karmic uses HAL and DeviceKit simultaneously.  However, the Hardware Abstraction Layer is being phased out from most Linux distributions, and the upcoming version of Ubuntu 10.04 (Lucid Lynx) will drop it completely.



Connections

“Connections” are the layer below the buses.  You can use a "connection" to “connect” to a specific daemon or application.  Please note the dotted notation:
The NetworkManager is available via a connection named org.freedesktop.NetworkManger. 

The connection org.freedesktop.Hal will connect you to the HAL daemon. 
The "disks" part of DeviceKit is available via the connection named org.freedesktop.DeviceKit.Disks.


Objects

These programs (NetworkManager, HAL, DeviceKit.Disks, etc.) will offer their services via one or more "objects".  Object names contain slashes and look like file paths, which might be the reason that they are sometimes called "paths". 

Examples:
/org/freedesktop/Hal/devices

/org/freedesktop/Hal/Manager

/org/freedesktop/Hal/devices/volume_label_backupcd1________
/org/freedesktop/DeviceKit/Disks/devices/sr0


Interfaces

Objects export methods and and signals grouped into one or more "interfaces".  The names of these interfaces again contain dots, e.g. org.freedesktop.Hal.Manager or org.freedesktop.DeviceKit.Disks.Device. 


Methods

Methods are functions you can invoke.  One method of the org.freedesktop.Hal.Manager interface for example is DeviceExists.  The interface org.freedesktop.DeviceKit.Disks implements e.g. EnumerateDevices, etc.


Signals

An interface may also export "signals".  Signals of the org.freedesktop.Hal.Manager interface that we are going to use are DeviceAdded and DeviceRemoved, which are triggered  when you put a new CD into the drive or eject it, but also if any other device is added (an USB stick, a new network interface, etc.), as well as the signal PropertiesModified.



Introspect

One interface that is implemented by all objects is org.freedesktop.DBus.Introspectable which offers one method: Introspect(). 
This function returns a description of the object's interfaces, i.e. what methods and signals are available and what parameters they use.  The above mentioned utility d-feet uses this method to display the information in a nice graphical way.  (Note: In d-feet a "Connection" is called "Bus Name").


If you want to see the dbus messaging at work (for the SystemBus) open a shell and start:

dbus-monitor --system


Summary

• bus: top level (SessionBus or SystemBus)


• connection: from the bus to a daemon; names with dots


• object: logical part within that daemon; names with slashes


• interface: one or more within an object (same interface on different objects means same API (methods and signals)); names with dots


• methods and signals (none or more within an object)

Examples

How to call a method (HAL)

In order to call a method like DeviceExists, you have to know, that this particular function is

1. part of the interface org.freedesktop.Hal.Manager


2. that this interface is implemented by the object named /org/freedesktop/Hal/Manager


3. which in turn can be reached via the connection org.freedesktop.Hal


4. on the SystemBus

In Python:
# (4)
bus = dbus.SystemBus()
# (3 + 2)
object = bus.get_object("org.freedesktop.Hal","/org/freedesktop/Hal/Manager")
# (1)
interface = dbus.Interface(object,"org.freedesktop.Hal.Manager")
#
result = interface.DeviceExists(...)

As mentioned earlier, you have to know which parameter of bus.get_object and dbus.Interface is the object name, the connection name, and the interface name.  The names themselves are usually not a great help.


Signal callbacks (HAL)

In order to make dbus call a method in your program when a signal is triggered you have to add a signal receiver:

bus.add_signal_receiver(
   handler_name_in_your_program,
   "signal_name",
   "interface_name",
   "connection_name",
   "object_name")


The handler method looks like this:

def handler(...):
   ...

The parameters used when this handler is executed (the so called "signature") differ from signal to signal.  (This info can be obtained via the Introspect interface.)  Here the example for the methods DeviceAdded and DeviceRemoved:


bus = dbus.SystemBus()
bus.add_signal_receiver(device_added_callback,
                                 "DeviceAdded",
                                 "org.freedesktop.Hal.Manager",
                                 "org.freedesktop.Hal",
                                 "/org/freedesktop/Hal/Manager")
bus.add_signal_receiver(device_removed_callback,
                                 "DeviceRemoved",
                                 "org.freedesktop.Hal.Manager",
                                 "org.freedesktop.Hal",
                                 "/org/freedesktop/Hal/Manager")

....

def device_added_callback(udi):
    print udi

def device_removed_callback(udi):
   print udi


(More on add_signal_receiver in the next post)


How to call a function (DeviceKit.Disks)

The fundamental dbus principles described in the HAL example still apply:

# (4)
bus = dbus.SystemBus()
# (3 + 2)
devkit_object = bus.get_object("org.freedesktop.DeviceKit.Disks",
                       "/org/freedesktop/DeviceKit/Disks")
# (1)
devkit_disks = dbus.Interface(devkit_object, 'org.freedesktop.DeviceKit.Disks')
#
all_devices = devkit_disks.EnumerateDevices()


Note: This time the connection/object/interface names are nearly identical.  Keep in mind which parameter describes what


Signal callbacks (DeviceKit.Disks)
Even tough a few handler names are identical to HAL and their functions are similar, their parameters are different.  Also note that the way DeviceKit.Disks adds callbacks is different:


devkit_disks.connect_to_signal('DeviceAdded', device_added_callback)
devkit_disks.connect_to_signal('DeviceRemoved', device_removed_callback)
devkit_disks.connect_to_signal('DeviceChanged', device_changed_callback)

def device_added_callback(device):
    # not really needed
    print 'Device %s was added' % (device)

def device_removed_callback(device):
    # not really needed
    print 'Device %s was removed' % (device)
 
def device_changed_callback(device):
    print 'Device %s was changed' % (device)



 
Other daemons

There are other interesting daemons, like the ModemManager, which not only can control modems, but also allows access to your mobile phone, SMS etc.

Ein Tag ohne E-Mail

Nach einem Sicherheitsupdate in Ubuntu Karmic meldete sich Thunderbird beim Start mit:

Die Sicherheitskomponente der Anwendung konnte nicht
initialisiert werden. Der wahrscheinlichste Grund dafür sind
Probleme mit Dateien im Profil-Ordner Ihrer Anwendung.
Bitte stellen Sie sicher, dass der Ordner keine Lese- und
Schreibbeschränkung hat und Ihre Festplatte nicht voll oder
fast voll ist. Es wird empfohlen,  dass Sie die Anwendung
jetzt beenden und das Problem beheben. Wenn Sie diese Sitzung
weiter verwenden, können Sie unkorrektes Verhalten der Anwendung
beim Zugriff auf Sicherheitsfunktionen feststellen.

und sämtliche mit SSL gesicherten Verbindungen zu POP- und SMTP-Servern schlugen fehl... und das sind bei mir alle.

I hate it when this happens.

Gegenmaßnahmen, die ich bei einer ersten Google-Suche nach dieser Fehlermeldung gefunden hatte (Neuinstallation von Thunderbird, Löschen von cert8.db, Anlegen eines neuen Profils) waren jedoch wirkungslos.

Beim Sicherheitsupdate wurde unter anderem die Bibliothek libnss aktualisiert, die für Verschlüsselung zuständig ist.

Ein Eintrag im Ubuntu-Form bestätigte diesen Verdacht. In meinem Fall reichte es, die veraltete Bibliothek libnss3-0d in Synaptic zu deinstallieren.