How to set the active camera ... when Blender renders in the background

Background rendering, i.e. rendering without the UI and based on command line parameters alone, is usually the domain of render farms.

But there are situations where this functionality is useful for normal users as well: scripts.

In my case the blend file contains a complex architectural model and various cameras (which you can conveniently set-up and switch between with the “Stored Views” add-on).

Some of those cameras are (virtual) panoramic cameras.  Their output will be fed into a generator script to calculate multi-res images for use with the web panorama viewer pannellum (https://pannellum.org) to create a virtual tour.

Wouldn’t it be nice to have a script producing the rendered pictures for all those cameras; it might even call the pannellum generator script afterwards to calculate the multi-res images needed for the virtual tour...

When Blender is started in background mode, various parameters can be set from the command line - among others the frame and scene to be rendered, the render engine and the output path.

Missing however are some settings I would like for my workflow: the active camera and the dimension percentage (Output Properties > Dimensions > Resolution > %). Those values are stored in the blend file and used in the render process.

If you want rendered images from multiple cameras you usually open the blend file, switch the camera, save the blend file and re-render…

As for the other setting: I sometimes forget to switch the resolution from my usual 50% to 100% before starting to render. While rendering without the UI you don’t notice this right away.  Something that sets this value to 100% would be nice.

The solution

Blender’s command line option --python-expr lets you enter a short Python snippet which gives you access to all settings in your scene.

To switch the active camera in the current screen use

import bpy
bpy.context.scene.camera = bpy.context.scene.objects.get('XXXXXX')

where XXXXXX is the name of the camera.

The command for setting the resolution percentage value is

bpy.context.scene.render.resolution_percentage = 100

You can test these commands in Blender’s Python console.

Note: It does matter where you put the --python-expr command on the command line. If used at the end of the command it will be ineffective because it changes the parameter after the render process.

My typical Blender background render command looks like this:

blender -b myblend.blend --python-expr "import bpy; bpy.context.scene.camera = bpy.context.scene.objects.get('Camera XXXX'); bpy.context.scene.render.resolution_percentage = 100" -o /tmp/pic_#### -F PNG -x 1 -f 1 -E CYCLES

• -b: background render of myblend.blend.
• --python-expr: use camera ‘Camera XXXX’ and render at 100%
• -o: store under /tmp/pic_####
• -F: store as PNG
• -x: add the PNG extension
• -f: render frame 1
• -E: use cycles render engine

You might use this technique to change other values on the fly without them being saved back to disk.

Accessing servers with self-signed certificates in Python

As long as I can remember Python was always capable of retrieving web pages from encrypted servers.  In the early days it didn't bother verifying the ssl certificate.  In newer version it does so by default - which is good - and you usually don't have any problems. And if you do it should merit your attention.

However there are situations where this verification breaks things: self-signed certificates. E.g. the ones you use in your local network or as in my case a web cam which actually offers https.  It uses an self-signed certificate - probably the same in all cameras of this type - but hey... beggars can't be choosers.

To access the cam in Firefox you would create a security exception to access the cam, in Python life is not that simple.

The following post shows:

• how to disable the verification
• how to pull the server certificate
• how to use it in Python3
• how to install it in the system

Please note: The following description works on Ubuntu 16.04 LTS. On your distro the directory paths may vary. Change IP addresses, hostnames, filenames, etc. to your needs.

I'm using a small script pulling images from the above mentioned web cam:

import urllib.request
...
hp = urllib.request.urlopen("https://192.168.0.100/pic.jpg")
pic = hp.read()
...

which now results in

urllib.error.URLError: < urlopen error [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed

The following "context" disables the certificate verification

import ssl
...
ctx = ssl.create_default_context()
ctx.check_hostname = False
ctx.verify_mode = ssl.CERT_NONE

hp = urllib.request.urlopen("https://192.168.0.100/pic.jpg", context=ctx)

That works, but having a certificate verification would be nice. To do that we need the server certificate:

cert = ssl.get_server_certificate( ('192.168.0.100', 443) )
open('/tmp/ipcamera.crt','w').write(cert)

The cert looks like this

-----BEGIN CERTIFICATE-----
MIIClDCCAf2gAwIBAgIJAIMQZ+Ua/bkXMA0GCSqGSIb3DQEBBQUAMGIxCzAJBgNV
...
4XAVFCBZOPwflj9Ug0YNSIgcSfDOxha06C9hwZ0+ZuafkjXv16sGEA==
-----END CERTIFICATE-----


Now you can create a context which uses that cert:

ctx2 = ssl.create_default_context()
ctx2.load_verify_locations("/tmp/ipcamera.crt")

hp = urllib.request.urlopen("https://192.168.0.100", context=ctx2)
...

Which results in:
 
ssl.CertificateError: hostname '192.168.0.100' doesn't match 'IPC'

Well, that didn't work, at least the error message has changed.

Let's have a look at the cert:

>>> ctx2.get_ca_certs()
[{'issuer': ((('countryName', 'ch'),), (('stateOrProvinceName', 'guangdong'),), (('localityName', 'zhenzhen'),), (('organizationName', 'IPCam'),), (('organizationalUnitName', 'IPCam'),), (('commonName', 'IPC'),)), 'notBefore': 'Mar  7 01:24:16 2013 GMT', 'subject': ((('countryName', 'ch'),), (('stateOrProvinceName', 'guangdong'),), (('localityName', 'zhenzhen'),), (('organizationName', 'IPCam'),), (('organizationalUnitName', 'IPCam'),), (('commonName', 'IPC'),)), 'notAfter': 'Feb 23 01:24:16 2063 GMT', 'serialNumber': '831067E51AFDB917', 'version': 3}]

As you can see the commonName for this cert is IPC and we're trying to access the server using the hostname 192.168.0.100. They don't match.

You can fix this in two ways. Either tell Python to ignore the hostname:

ctx3 = ssl.create_default_context()
ctx3.load_verify_locations("/tmp/ipcamera.crt")
ctx3.check_hostname = False

hp = urllib.request.urlopen("https://192.168.0.100", context=ctx3)

or put an entry into /etc/hosts (you need root privileges for that)

192.168.0.100   IPC

System wide integration
Using contexts is fine, but I have to change every piece of code: create the context and use it. It would be nice to have it "simply" work.
For this you need root access. Then you can put the cert into the system wide certificate store and Python will use it like any normal cert - including the one from the Hongkong Post Office :-)

First create the above mentioned entry in /etc/hosts to get the hostname check right.

Then create a the directory /etc/ssl/mycerts and copy ipcamera.crt into it.

The system wide certs are stored in /etc/ssl/certs. In order for your certificate to be found, it must be renamed. Calculate its hash using openssl:

$ openssl x509 -noout -hash -in /etc/ssl/mycerts/ipcamera.crt
ab0cd04d

Now goto /etc/ssl/certs and create the appropriate named link (you must append .0 to the hash).

sudo ln -s ../mycerts/ipcamera.crt ab0cd04d.0

Now it simply works:

w = urllib.request.urlopen("https://IPC")

If there are easier ways to do it, please let me know.



Links:

• https://docs.python.org/2/library/ssl.html
• http://gagravarr.org/writing/openssl-certs/others.shtml#ca-openssl

Accessing library materials, groups and other stuff via the Blender Python API

Information valid for Blender 2.72

The API documentation for the equivalent of the Link and Append command is a bit cryptic regarding what values to use for the parameters of this operator.  After some Googleing I found the solution in a few snippets of code here.

The main parameters of the Blender Python append command are:

• filepath
• filename
• directory

The values of these parameters combine the file system path of the blend file and its "inner" structure.

You can see this structure if you use the Link/Append command of the Blender GUI.  Once you click on the blend file its inner structure with materials, groups, objects etc. becomes visible.


In order to append for example the Material with the name white55 from a file named test.blend located in my home directory /home/mike, I have to use:

bpy.ops.wm.append(
   # // + name of blend file + \\Material\\
   filepath="//test.blend\\Material\\",

   # name of the material
   filename="white55",

   # file path of the blend file + name of the blend file + \\Material\\
   directory="/home/mike/test.blend\\Material\\", 
  
   # append, don't link
   link = False
)

Check the API entry for the other parameters.

This is an example from a Unix system, where the file path separator is a normal slash; on Windows systems you have to use a backslash.

However, the backslash is also the Python escape character, i.e. for one \ in the path name, you have to type \\

You can modify this example using \\Object\\ or \\Group\\ ...

Note: The operator append has recently (v2.71) been renamed.  It's earlier name was link_append.

This operator doesn't return an error code if the material, object etc. couldn't be loaded.  You have to be sure that it is present in the blend file.

You can iterate over the material names in a blend library file using the following snippet:

with bpy.data.libraries.load("/home/mike/test.blend") as (data_from, data_to):
   print(data_from.materials)

data_from.materials is a list of strings with the material names. The list is empty, if there aren't any materials available.

The command dir(data_from) returns
['actions', 'armatures', 'brushes', 'cameras', 'curves', 'fonts', 'grease_pencil', 'groups', 'images', 'ipos', 'lamps', 'lattices', 'linestyles', 'masks', 'materials', 'meshes', 'metaballs', 'movieclips', 'node_groups', 'objects', 'paint_curves', 'palettes', 'scenes', 'sounds', 'speakers', 'texts', 'textures', 'worlds']

Three guesses what data_from.groups or data_from.textures will return and how to modify the append command...

Blender: Change Cycles render seed automatically

Here is a small Blender add-on to change the seed value for the Cycles render engine automatically before rendering a frame.

The seed value determines the noise pattern of a cycles render.

If you only render one image, this is of no concern for you.

If you render an animation, you get the same pattern for each frame which is very obvious for the viewer.

To counter this, you can enter #frame as seed value (creating a so called driver) which returns the frame number as seed value. This gives you a different noise pattern per frame which makes it look like film grain.

This obviously only works if you have an animation to render, but not with a single frame.

After installing the add-on you see an additional checkbox in the render samplig panel where you can switch this feature on or off.



Why to go the trouble to write an add-on for this?

Lately I have used image stacking a lot.

This technique allows you to reduce noise in pictures created by Cycles rendering engine by rendering the same frame multiple times - provided you change the render seed. You then can calculate the "average" of these pictures (e.g. using ImageMagick) cancelling out some noise.

If you want to a clearer version of an image that has just rendered over an hour, you save it and render it again, then stack the images. This is much faster than scrapping the first image and re-rendering it with a larger sample count.

After forgetting to change the seed value a couple of times, the level of suffering was high enough to make an add-on. :-)

Hacking the Foscam FI9821W V2

The Foscam FI9821W V2 is a low end HD colour  IP camera.

The feature list is impressive.

• The unit has Ethernet and Wifi connectivity.
• It can stream video in various resolutions and formats (H.264 and MJPEG) as well as audio from the built-in microphone.
• It comes with multiple built-in infrared LEDs to get a B&W picture even in the dark.
• The lens can pan and tilt.  You can set preset points, and define cruises from point to point. 
• The built-in software supports motion detection with a grid of active and inactive areas.
• With the talk-back function you can speak through the cameras loud-speaker.
• And more

OS support

The description at Amazon promised a browser plugin for Explorer, Firefox, and Chrome, to control the camera as well as support for Windows, Mac and Linux.
As I had to find out, this didn't mean a browser plugin for Firefox that runs under Linux.  The plugin is Windows only.  It is part of the cameras firmware and can be downloaded via the cameras web interface.  It is recommended to delete the plugin after a firmware update and install the new one.

While the unit has a web interface which can be accessed from any current browser on any platform, this interface only allows you to set the basic configuration (user accounts, Ethernet and Wifi parameters, firmware upgrade, etc.).  The interface for the "advanced" features is implemented in the plugin, i.e. it is Windows only.  These advanced feature include video display, audio playback, pan and tilt, motion detection, etc.

The camera provides a rtsp video stream, which can be viewed using vlc - even under Linux.

Fortunately, Foscam provides a SDK with the description of the CGI interface, meaning that with a handful of specially crafted URLs most of the above "advanced" functions are available.  In fact, the browser plugin uses the CGI interface as well.

pyFosControl

I've started to write a Python interface to access those functions (GitHub).

One function the CGI interface can not provide is talk-back, which involves sending an audio stream to the camera.  As it turns out, the browser plug-in also uses a low-level protocol.  Even though Foscam provides a description of this protocol for its older cam, the newer ones (HD, H.264) are not (yet?) documented.

The low-level protocol

I reverse-engineered a part of this protocol. The results can be found in the "lowlevel" folder in the above mentioned pyFosControl project.  You'll also find the python program I'm using to "tickle" my camera.  If you find out more, please let me know.

Firmware and bugs

The firmware of this camera is still in development, and there have been a couple of firmware updates.  Further updates are badly needed as there are still bugs in even basic functions:

•  You can define "privacy areas" - black masks in the video stream.  However, these only work in the video stream not for snapshot pictures.
• Pictures and alarm videos stored on the internal SD RAM are accessible via FTP.  In the last firmware you didn't need any credentials to access them.  Currently you need admin rights. However, the credentials for the FTP server these are stored in RAM after you create an account and are gone when the unit is powered down.  You have to recreate the still existing account in order to get credential into memory. (The login to the web interface is not affected).
• The web interface can be securely accessed using https:// thus encrypting user name and password.  However, even under these circumstances the user name and password are transmitted in the clear in various situations, making it unsuitable for accessing the camera from the Internet without the protection of a VPN.

User forum

The Foscam user forum is quite active.  Many other problems of the various Foscam cameras are discussed there, but keep in mind that forums usually attracts those who have problems.  A few friendly souls trying to help you where they can.  It's worth a visit.

[Update 2014-03-02]

Faulty IR LEDs

After a few months illumination by the IR LEDs has become erratic. The problem seems to be intermittent contact of the power supply connectors to the pcb carrying the LEDs. According to the forums, this is a common problem – which can't be solved. And as most Foscam models look similar, it is very likely that all these units are affected. :(

How to store images in mp3 files using eyeD3

(This post refers to version 0.7.1 of eyeD3)

eyeD3 is a powerful python utility for manipulating id3 tags from the command line, but it also exposes these function as a python module.

Unfortunately the documentation on the site on how to use it as a python module is sparse, although it covers most use cases:

import eyed3

audiofile = eyed3.load("song.mp3")
audiofile.tag.artist = u"Nobunny"
audiofile.tag.album = u"Love Visions"
audiofile.tag.title = u"I Am a Girlfriend"
audiofile.tag.track_num = 4

audiofile.tag.save()

Information on how to access images are not readily available. But being open source, you can look at the source code to find out:

Example on how to append an image

import eyed3

# load tags
audiofile = eyed3.load("song.mp3")

# read image into memory
imagedata = open("test.jpg","rb").read()

# append image to tags
audiofile.tag.images.set(3,imagedata,"image/jpeg",u"you can put a description here")

# write it back
audiofile.tag.save()

The constant 3 means Front Cover, 4 means Back Cover, 0 for other.
The complete list can found at eyed3/id3/frames.py

"image/jpeg" is the mime type.
u"...” is a description, which must be provided in a unicode string. EasyTAG e.g. stores the original file name in this field.


Example on how to access the images

import eyed3

# load tags
audiofile = eyed3.load("song.mp3")

# iterate over the images
for imageinfo in audiofile.tag.images:
   ...

Amoung the infos available via imageinfo are:
 .image_data - image binary data
 .mime_type  - e.g. “image/jpeg”
 .picture_type - 3, 4, 0, see above
 .description - the description field

You can access the imageinfo also like this:

audiofile.tag.images[x]

Where x is the index into an array starting with 0. Eventually you will get an out of range exception.

dbus tutorial - part 1

Basic dbus concepts

What is dbus?

dbus is part of most modern Linux distros.  It handles interprocess communication, either between desktop applications or between systems daemons and applications (e.g. a desktop framework like Gnome or KDE, or user applications).


Overview

The goal of this series of postings is the creation of a small desktop example application written in Python which reads the name of the files on a CD or DVD after the media has been inserted, and to store these names together with the CD label in a file.  The graphical user interface will be based on wxWidgets. 

The first part contains an introduction to the basic dbus components.
The second part discusses the implementation using the Hardware Abstraction Layer (HAL).
The following part shows the differences in implementation between HAL to the DeviceKit.Disks.


Prerequisites

Python: which is part of any current Linux distribution. (I'm using Python 2.6).
python-dbus: the dbus bindings for Python
python-wxgtk2.8: wxWidgets and its Python bindings (for the GUI)

Useful additions

python-wxglade: GUI builder for wxWidgets, creates Python source code (and C source code...)

d-feet: graphical display of dbus components

dbus components

In our example dbus is used to catch the notifications sent by the kernel when a CD or DVD has been inserted, as well as the notification when Gnome has mounted this medium. (There were times when you had to do this manually.)

In some places I will differ from the official dbus designations for these components to make their distinction easier.  In some cases different components share the same name.  This makes their close relation quite obvious, but it becomes difficult to determine if a given name refers to a connection or an interface, for example.


Buses

In dbus “buses” are at the root of a hierarchy.  In normal applications you encounter mostly the following two buses:

    bus = dbus.SessionBus()

for communication between desktop applications, and

    bus = dbus.SystemBus()

for communication with system programs and daemons like the NetworkManager, the ModemManager, the Hardware Abstraction Layer daemon (HAL) or DeviceKit, to name but a few.

When you insert a disc, a notification is routed via HAL or DeviceKit, depending on which daemon is installed.

Note: Ubuntu Karmic uses HAL and DeviceKit simultaneously.  However, the Hardware Abstraction Layer is being phased out from most Linux distributions, and the upcoming version of Ubuntu 10.04 (Lucid Lynx) will drop it completely.



Connections

“Connections” are the layer below the buses.  You can use a "connection" to “connect” to a specific daemon or application.  Please note the dotted notation:
The NetworkManager is available via a connection named org.freedesktop.NetworkManger. 

The connection org.freedesktop.Hal will connect you to the HAL daemon. 
The "disks" part of DeviceKit is available via the connection named org.freedesktop.DeviceKit.Disks.


Objects

These programs (NetworkManager, HAL, DeviceKit.Disks, etc.) will offer their services via one or more "objects".  Object names contain slashes and look like file paths, which might be the reason that they are sometimes called "paths". 

Examples:
/org/freedesktop/Hal/devices

/org/freedesktop/Hal/Manager

/org/freedesktop/Hal/devices/volume_label_backupcd1________
/org/freedesktop/DeviceKit/Disks/devices/sr0


Interfaces

Objects export methods and and signals grouped into one or more "interfaces".  The names of these interfaces again contain dots, e.g. org.freedesktop.Hal.Manager or org.freedesktop.DeviceKit.Disks.Device. 


Methods

Methods are functions you can invoke.  One method of the org.freedesktop.Hal.Manager interface for example is DeviceExists.  The interface org.freedesktop.DeviceKit.Disks implements e.g. EnumerateDevices, etc.


Signals

An interface may also export "signals".  Signals of the org.freedesktop.Hal.Manager interface that we are going to use are DeviceAdded and DeviceRemoved, which are triggered  when you put a new CD into the drive or eject it, but also if any other device is added (an USB stick, a new network interface, etc.), as well as the signal PropertiesModified.



Introspect

One interface that is implemented by all objects is org.freedesktop.DBus.Introspectable which offers one method: Introspect(). 
This function returns a description of the object's interfaces, i.e. what methods and signals are available and what parameters they use.  The above mentioned utility d-feet uses this method to display the information in a nice graphical way.  (Note: In d-feet a "Connection" is called "Bus Name").


If you want to see the dbus messaging at work (for the SystemBus) open a shell and start:

dbus-monitor --system


Summary

• bus: top level (SessionBus or SystemBus)


• connection: from the bus to a daemon; names with dots


• object: logical part within that daemon; names with slashes


• interface: one or more within an object (same interface on different objects means same API (methods and signals)); names with dots


• methods and signals (none or more within an object)

Examples

How to call a method (HAL)

In order to call a method like DeviceExists, you have to know, that this particular function is

1. part of the interface org.freedesktop.Hal.Manager


2. that this interface is implemented by the object named /org/freedesktop/Hal/Manager


3. which in turn can be reached via the connection org.freedesktop.Hal


4. on the SystemBus

In Python:
# (4)
bus = dbus.SystemBus()
# (3 + 2)
object = bus.get_object("org.freedesktop.Hal","/org/freedesktop/Hal/Manager")
# (1)
interface = dbus.Interface(object,"org.freedesktop.Hal.Manager")
#
result = interface.DeviceExists(...)

As mentioned earlier, you have to know which parameter of bus.get_object and dbus.Interface is the object name, the connection name, and the interface name.  The names themselves are usually not a great help.


Signal callbacks (HAL)

In order to make dbus call a method in your program when a signal is triggered you have to add a signal receiver:

bus.add_signal_receiver(
   handler_name_in_your_program,
   "signal_name",
   "interface_name",
   "connection_name",
   "object_name")


The handler method looks like this:

def handler(...):
   ...

The parameters used when this handler is executed (the so called "signature") differ from signal to signal.  (This info can be obtained via the Introspect interface.)  Here the example for the methods DeviceAdded and DeviceRemoved:


bus = dbus.SystemBus()
bus.add_signal_receiver(device_added_callback,
                                 "DeviceAdded",
                                 "org.freedesktop.Hal.Manager",
                                 "org.freedesktop.Hal",
                                 "/org/freedesktop/Hal/Manager")
bus.add_signal_receiver(device_removed_callback,
                                 "DeviceRemoved",
                                 "org.freedesktop.Hal.Manager",
                                 "org.freedesktop.Hal",
                                 "/org/freedesktop/Hal/Manager")

....

def device_added_callback(udi):
    print udi

def device_removed_callback(udi):
   print udi


(More on add_signal_receiver in the next post)


How to call a function (DeviceKit.Disks)

The fundamental dbus principles described in the HAL example still apply:

# (4)
bus = dbus.SystemBus()
# (3 + 2)
devkit_object = bus.get_object("org.freedesktop.DeviceKit.Disks",
                       "/org/freedesktop/DeviceKit/Disks")
# (1)
devkit_disks = dbus.Interface(devkit_object, 'org.freedesktop.DeviceKit.Disks')
#
all_devices = devkit_disks.EnumerateDevices()


Note: This time the connection/object/interface names are nearly identical.  Keep in mind which parameter describes what


Signal callbacks (DeviceKit.Disks)
Even tough a few handler names are identical to HAL and their functions are similar, their parameters are different.  Also note that the way DeviceKit.Disks adds callbacks is different:


devkit_disks.connect_to_signal('DeviceAdded', device_added_callback)
devkit_disks.connect_to_signal('DeviceRemoved', device_removed_callback)
devkit_disks.connect_to_signal('DeviceChanged', device_changed_callback)

def device_added_callback(device):
    # not really needed
    print 'Device %s was added' % (device)

def device_removed_callback(device):
    # not really needed
    print 'Device %s was removed' % (device)
 
def device_changed_callback(device):
    print 'Device %s was changed' % (device)



 
Other daemons

There are other interesting daemons, like the ModemManager, which not only can control modems, but also allows access to your mobile phone, SMS etc.

Python: cStringIO vs. StringIO

If a function expects a file handle as an input parameter, but the data you want to process is stored in a string variable, you could store its content in a temporary file on disk, then open that file and use its file handle for the above mentioned function.

A nicer way to do this is the python function StringIO in the StringIO library. It takes the string as an argument and returns a file handle, from which the function can read(). The conversion happens in memory, i.e. no tedious creation of temporary files.

Depending on how often you need this functionality and how large the data is, the StringIO.StringIO - which is programmed purely in Python - may be too slow.

The library cStringIO offers a function with the same name with the same functionality carried out by a faster C-implementation. This function should be a drop-in replacement for the Python version.

However, since more and more strings are stored in unicode this function has an unexpected side effect. This side effect is detailed in the library description, if you look close enough.

cStringIO.StringIO does not return the original encoded text, but the "representation of the Unicode string", which may differ from machine to machine:

>>>import StringIO
>>> a = StringIO.StringIO(u"test")
>>> a.read()
u'test'

>>>import cStringIO
>>> a = cStringIO.StringIO(u"test")
>>> a.read()
't\x00\x00\x00e\x00\x00\x00s\x00\x00\x00t\x00\x00\x00'

A patch to fix this was proposed but rejected due to backwards compatibility.

So be aware...

wxPython-Versionen auswählen

Wenn mehrere wxPython-Versionen gleichzeitig auf einem System installiert sind, so lädt ein einfaches

import wx

natürlich immer die Version, mit der man gerade nicht arbeiten will.

import wxversion
wxversion.select('2.8')
import wx

Die Version des gerade geladenen Moduls erhält man mit:

print wx.__version__

Linux, Python und die MS-Access-Datenbank

Hier lag sie nun - ein Überbleibsel aus einer anderen Zeit und Welt. Eine Bilderdatenbank, die ich vor Jahren unter Windows mit dem bekannten Programm "Daumen Plus" erstellt hatte und deren Daten ich nun gerne nutzen wollte. Das Problem war das Format: MS-Access

Dieses Format ist unter Linux nicht gerade gängig, der Zugriff ist aber - wenn auch nicht "out of the box" - durchaus möglich.

Die hier beschriebenen Schritte beziehen sich auf eine Standardinstallation von Ubuntu Jaunty - sie dürften sich aber so oder ähnlich auch auf anderen Systemen nachvollziehen lassen.

Der Treiber

Der eigentliche Treiber findet man im Paket libmdbodbc. Wie der Name schon andeutet, läuft der Treiber im ODBC-Framework.

Die Schnittstelle ODBC

ODBC ist eine standardisierte Schnittstelle, in der auf der einen Seite Treiber wie Plugins die Verbindung zu den verschiedenen Datenbankentypen herstellen, und die auf der anderen Seite von Programmen - unabhängig vom Typ der Datenbank - immer in der gleichen Weise angesprochen werden können.

ODBC ist in Jaunty nicht standardmäßig installiert. Man braucht das Paket unixodbc. Zusätzlich sollte man noch unixodbc-dev holen, das wir weiter unten brauchen.

Zunächst muss man ODBC über das Vorhandensein des Treibers informieren. Die Treiber werden in Form sog. Templates eingerichtet. Hier das Template für MS Access.

[MDB]
Description = Microsoft Access ODBC Driver
Driver = /usr/lib/libmdbodbc.so.0
Setup =
FileUsage = 1
CPTimeout =
CPReuse =
UsageCount = 1

Es besagt, dass der Treiber libmdbodbc.so.0 nun im ODBC-Framework unter dem Namen frei gewählten Namen MDB angesprochen werden kann.

Speichern Sie dieses Template (z.B. unter derm Namen msaccess.template) und installieren Sie es mit dem folgenden Befehl:

odbcinst -i -d -f msaccess.template

Das Template wird in der Datei /etc/odbcinst.ini abgelegt. Diese Datei im INI-Format könnte man natürlich auch von Hand editieren.

Die Datenquelle

Eine Datenquelle fasst mehrere Parameter unter einem Namen (Data Source Name, DSN) zusammen. Diese Parameter könnten zwar auch beim Aufruf der Datenbank angegeben werden, aber der Parameterstring wird schnell sehr lang und seine Länge ist begrenzt.
Unter einem DSN werden in diesem Beispiel der verwendete Treiber, der Server und der Pfad zur Datenbank hinterlegt. Weitere Angaben sind je nach verwendetem Treiber möglich.

Das Verfahren ähnelt dem Einrichten des Treibers. Hier ein Beispiel:

[bilderdb]
Description = Bilder-Datenbank
Driver = MDB
Database = /home/mike/fotos.mdb
Servername = localhost
UserName =
Password =
Port = 5432

Abgespeichert in einer Textdatei mit dem Namen bilder.template, erfolgt die Installation entweder systemweit mit

odbcinst -i -s -l -f bilder.template

oder benutzerspezifisch mit

odbcinst -i -s -f bilder.template

Gespeichert wird diese Information in der INI-Datei /etc/odbc.ini (systemweit) bzw. ~/.odbc.ini (benutzerspezifisch).

Zugriff via Python

Zum Zugriff braucht man das Paket pyodbc. Das wiederum gibt es leider zurzeit nicht in einem Ubuntu-Paket. Man muss es - wie früher - aus dem Quellcode kompilieren. Hierzu braucht man zum einen den Sourcecode von pyodbc sowie die oben erwähnte Datei unixodbc-dev.

Wer das erste Mal selbst kompiliert, installiert noch build-essential, letzteres ist wieder bequem über apt-get oder Synaptic aus den Ubuntu-Archiven verfügbar.

Das pyodbc-Archiv muss entpackt werden. Danach begibt man sich in das beim Auspacken angelegte Verzeichnis und gibt

sudo python setup.py build install

ein. Das war es schon.

Kurze Zusammenfassung:

Aus den Ubuntu-Repositories braucht man:
build-essential, unixodbc-dev, unix-odbc und libmdbodbc.
Aus dem Web:
pyodbc

Und los geht's

import odbc

con = pyodbc.connect("DSN=bilderdb")
cursor = con.cursor()
cursor.execute("sinnvoller SQL-Befehl")
result = cursor.fetchone()

Jedem, der in Python mit Datenbanken zu tun hatte, sollten diese Zeilen bekannt vorkommen.

Die bei connect verwendete Zeichenkette, die in diesem Beispiel nur auf eine Datenquelle (DSN) verweist, kann viele andere Parameter aufnehmen, z.B. Username und Passwort bei einer passwortgeschützten Datenbank:

con = pyodbc.connect("DSN=bilderdb;UID=name;PWD=passwort")

Die einzelnen Parameter werden, durch Semikolon getrennt, in der Form Attribut=Wert aneinandergereiht. Die Namen der Attribute hängen vom verwendeten Treiber ab und sind recht vielfältig.

Zum Schluss noch ein paar nützliche Tipps beim Erkunden einer MS-Access-Datenbank:

Liste der Tabellen in einer Datenbank

import pyodbc

con = pyodbc.connect('DSN=bilderdb')
cursor = con.cursor()

for row in cursor.table():
print row

Die Zeilen haben den Aufbau (Werte: String oder None):

table_cat
table_schem
table_name Name der Tabelle
table_type SYSTEM TABLE, VIEW, TABLE

Die eigentlichen Tabellen werden unter dem Typ TABLE aufgelistet.

Aufbau einer Tabelle

Dieser lässt sich mit dem normalen SQL-Befehl erfragen:

cursor.execute('DESCRIBE TABLE tablenname")

Das Ergebnis wird in der Form (Name, Type, Größe) zurückgegeben.