Montag, 9. August 2010

Lift + Scala, Introduction

A recent show of Floss Weekly made me curious about Lift, a web framework using Scala.  

“Scala is a general purpose programming language designed to express common programming patterns in a concise, elegant, and type-safe way.” so they promise  on the Scala website.

The Lift project leader praises his product as “a concise language, frame work with security features, very performant...”

A first glance

Lift, like any framework that you haven't written yourself, will take some time getting familiar with, and Scala, well, it's just another computer language... or so I thought.

Lift

Perhaps the most important feature of Lift is its template system.  These templates are strictly XML based text files without any programming logic.  Tags within the template link to functions in your Scala code that generates some output.  This output (also XML) then replaces the tag.  After a few more processing steps the result is returned to the enquiring web browser, your page gets rendered, end of story.  Yes, there are lots of support features within the framework – like html form generation, database connectivity, page access control – but this is the basic principle of Lift.

Scala

The designers of Scala cherry-picked the best features and concepts from other programming languages, some object-oriented stuff but they also borrowed heavily from functional programming. Its mantra: “everything is an object” and “if it's worth saying, say it with a function”.

Scala's smart compiler can in most cases infer types of variables, so you don't have to declare it manually.  The language has lots of these simplyfications, dramatical reducing boiler plate code.  However, you have to be familiar with these in order to be able to revert the expression back to its more verbose form, otherwise you will scratch your head when you encounter the “Scala smiley”:

 ( _ + _ )

Some language constructs help you avoiding Null pointer dereferences, or to catch all possible cases in a matching construct.  And being a statically typed language  the Scala compiler will be very stubborn if your types don't match.

You can write very elegant code – even though I wouldn't call deeply nested function calls simple to read, at least not yet.

Treacherous simplicity

Letting the compiler infer types looks like a good idea.  On second sight, it also means that you need a good memory or at least a helpful IDE.

Simplifying
   val a: Int = 1

to
   val a = 1

is simple and easy to understand. (The literal 1 is an integer, i.e. “a” must be integer as well). But:

val a = function_i_wrote_ealier_and_dont_remember_the_return_type()
val b = another_cool_function_from_the_web_framework(User => go look)

does work just as well. And now you either know the types or you have to look it up.

The possibility to drop dots and brackets in some situation lets you further “simplify” things:
   a.+(b)

becomes a much more friendlier
   a + b
(+ is a valid function name in Scala).

But
   a :: b

does not mean
   a.::(b)

but
   b.::(a)


(“Operators” that end with “:” bind from right to left).

Another nice feature: When defining a class how often have you assigned the values from the constructor parameter list to class variables for later use? It's usually the first thing you do.  In Scala this is done automatically.  Why hasn't somebody invented this earlier? 

Documentation + support

There is documentation available for Lift and Scala on the web.  While perhaps not exactly light reading, they will get you started.

For Scala http://programming-scala.labs.oreilly.com/index.html from O'Reilly is a good start, and on the Scala website there is http://www.scala-lang.org/docu/files/ScalaByExample.pdf.  Both publications expect that you have done some programming before.

For Lift you should check out master.pdf from http://groups.google.com/group/the-lift-book/files. More information is available at the Lift Wiki and a subscription to the Lift newsgroup is also recommended.

Montag, 28. Juni 2010

Update with caution - Firefox 3.6.6

The recent update to Firefox 3.6.6 seems to cause more problems than usual.

Over here, a Mac was the first to update - and Firefox did not re-start.  Having discovered the safe-mode option recently, I used it on the Mac:

/Applications/Firefox.app/Contents/MacOS/firefox-bin -safe-mode

After disabling all add-ons, Firefox 3.6.6 did start normally.

On my Ubuntu machine, Firefox is still on version 3.6.3.   So I decided to test FF 3.6.6 before the Ubuntu updater pushes the update.  I downloaded the Linux version of 3.6.6 from Mozilla and installed it in a separate directory. 

Multiple versions of Firefox can coexist on the same machine, but only one can run at any given time.  I.e. you have to close ALL Firefox windows of one version before starting the other.

Strangely enough, the new version worked even with the add-ons that crashed Firefox on the Mac.  But there were other issues:
  • no graphics on Google Analytics
  • no address resolution of a web server on the LAN (worked on FF 3.6.3, might be a 64 bit issue - ping is ok)
I think, I'll skip this one.

If the new versions has already been installed...

If you hover your mouse over the appropriate download button on the Mozillas website, you see a link like this one (perhaps with different o/s and language settings) in the status bar:

http://download.mozilla.org/?product=firefox-3.6.6&os=osx&lang=de

You can copy the link, modify the version number to get the older Firefox version.

[Update, June 29th]
Firefox 3.6.6 (32 bit and 64 bit) from the Ubuntu repositories run without problems.

Samstag, 26. Juni 2010

Integrating a re-branded Huawai UMTS modem into Ubuntu 10.04

The Huawai E160 is a popular UMTS stick and works with Ubuntu Lucid Lynx out-of-the-box.

It is also commonly used by European phone companies in their mobile internet starter packs.  Unfortunately these sticks are usually sim-locked to the phone company and some identify themselves with a USB id unknown to current Ubuntu setup.

When you plug the stick into the computer, all you see is a small flash drive icon containing the Windows and sometimes Macintosh drivers.  On these systems the drivers switches the stick from mass storage device mode into modem mode after the initial phase.

The Linux utility for this task is called modem-modeswitch, and can be found in /lib/udev.  The actual task is to automate the execution of this utility when the UMTS stick is plugged in.

The UMTS stick in this example is/was sold by the Belgian phone company Mobistar.  When you query the USB IDs:

sudo lsusb

it is identified as

12d1:1446 Huawei Technologies Co., Ltd. 


The two numbers at the beginning of the line are the vendor ID and product ID of the stick in the current mode.

To check if the modem switch works as expected, call the utility manually:

sudo /lib/udev/modem-modeswitch -v 0x12d1 -p 0x1446 -t option-zerocd

Wait a few seconds and check the USB IDs again.  The entry should now read:

12d1:1003 Huawei Technologies Co., Ltd. E220 HSDPA Modem / E270 HSDPA/HSUPA Modem


If nothing happens, David Santinoli suggests in a related document to unmount the device containing the drivers before attempting the switch.

You can automate the process by creating the file 62-huawai.rules in /lib/udev/rules.d:

ATTRS{idVendor}=="12d1", ATTRS{idProduct}=="1446", RUN+="modem-modeswitch -v 0x%s{idVendor} -p 0x%s{idProduct} -t option-zerocd"

(The file name is arbitrary, as long as it starts with “62-” and ends with “.rules”.) The rule itself is based on similar entries in 61-option-modem-modeswitch.rules.

Next time you boot your computer, the rule becomes active, and the modem should be recognized automatically.

Montag, 14. Juni 2010

Firefox - Ausfall der rechten Maustaste

In den letzten Wochen ist es bei mir häufiger vorgekommen, dass Firefox 3.5 (unter Ubuntu 10.04) nicht mehr auf die rechte Maustaste reagierte. Auf dem Desktop und in den übrigen Programmen funktionierte sie weiterhin, sodass ein Problem mit der Maus ausgeschlossen werden konnte.

Der eigentliche Grund des Problems ist weiterhin unklar. Mit dem folgenden Trick kann man Firefox jedoch in den Normalzustand zurücksetzen:

Hierzu beendet man Firefox vollständig und ruft es von der Kommandozeile mit

firefox -safe-mode

auf. Daraufhin erscheint der folgende Bildschirm:



Im Safe-Mode startet Firefox - ähnlich wie der Safe-Mode bei Windows - ohne Erweiterungen.

Wie man sieht kann man fast alles zurücksetzen, was man in den letzten Jahren in mühsamer Kleinarbeit zusammengetragen hat.

"Reset toolbars and controls" scheint die Option mit dem geringsten "Lösch-Potenzial" zu sein, und in meinem Fall reichte sie auch aus. Nach "Make Changes and Restart" reagierte Firefox wieder auf die rechte Maustaste.

Dienstag, 25. Mai 2010

Touch Me, Tux - die Zweite

Im Zuge der in einigen Tagen beginnenden Fußball-WM 2010 verkaufte Aldi-Süd letzte Wochen den „TEVION® Tragbarer 11 cm/4,3'' Touch Multimediaplayer“ (MD 82700) mit eingebautem DVB-T-Empfänger. So können sich die Fußballfans per „Überall-Fernsehen“ auf dem Laufenden halten - wenigstens „überall“ dort, wo es empfangbar ist.

Im Lieferumfang befinden sich neben dem Netzteil, ein USB-Kabel, ein Ohrhörer und eine kleine DVB-T-Magnetfußantenne.

Daneben gibt der Player Videos, Musik und Fotos wieder. Und auch Text kann man sich anzeigen lassen; natürlich kein Vergleich zu Kindle und Co – weshalb das Feature wahrscheinlich nicht einmal beworben wurde.

Das Gerät meldet sich unter Linux als USB-Massenspeicher (MD 82700) an. Je nachdem, ob eine zusätzlich Micro-SD Karte eingelegt ist, erscheinen ein (intern: 4 GB) bzw. zwei USB-Laufwerke. Bilder und Video lassen sich auch mit gängigen Werkzeugen unter Linux so codieren, dass sie vom Player wiedergegeben werden.

Als Wiedergabeformate werden JPEG, MP3, MPEG4, RM, RMVB, Divx und Xvid beworben. Zu RM und RMVB kann ich nichts sagen, da ich unter Linux keine Encoder dafür laufen habe.

Die Angabe von MPEG4 ist zwar nicht falsch, aber vielleicht etwas missverständlich. MPEG4 ist ähnlich wie AVI ein Container-Format. Es sagt nichts über die darin verwendeten Codecs aus.

Um normale Dateien zu  komprimieren gibt unterschiedliche Algorithmen (ZIP, RAR, LZH, usw.). Ein sehr bekannt Algorithmus für Audio ist MP3, aber er ist halt nicht der einzige. Im Open-Souce-Bereich ist auch Ogg Vorbis verbreitet, Mac-User kennen AAC usw. Und auch bei den Videocodecs sind mittlerweile nicht nur Eingeweihten mehrere Codecs bekannt: DivX (und sein Open-Source-Pendant Xvid) sowie MPEG2 von den DVDs.

Häufig genutzte Quellen für MP4-Dateien sind heutzutage Video-Portale wie Youtube und Vimeo. Die dort erhältlichen Videos werden mit dem sehr effektiven Codec H.264 für Video und häufig mit AAC für Audio komprimiert. Diese beiden Codecs versteht das Gerät allerdings nicht. Solche Dateien können nicht direkt wiedergegeben werden und müssen auf dem Computer umcodiert werden.

Wenn man einmal vom DVB-T absieht, so erinnert der Player sehr an das Technaxx-Gerät, mit dem dieses Blog vor über einem Jahr begann. Die verfügbaren Formate, Aufbau und Inhalt der Menüs, exotische Funktionen wie Liedtext-Wiedergabe mit .lirc-Dateien, ähneln sehr der Software im Technaxx-Player. Selbst der vor einem Jahr genannte Konvertier-Befehl kann so (und in leicht abgewandelter Form) verwendet werden.

Von der Auflösung her unterscheiden sich der Technaxx- (320 x 240) und der Medion-Player (480 x 272) nicht sonderlich. Die höhere Pixelzahl ist fast ausschließlich dem Wechsel vom 4:3- zum 16:9-Format zu verdanken; aber natürlich ist der Bildschirm jetzt deutlich größer.

Der Wiedergabe-Chip ist leistungsstärker geworden, so dass viele Einschränkungen weggefallen sind: Es werden höhere Bildfrequenzen (Technaxx: max. 20 Bilder pro Sekunde) und höhere Bitraten für Video und Audio wiedergegeben. Auch B-Frames werden nun unterstützt, die eine höhere Komprimierung erlauben.

Die alten für den Technaxx kodierten Dateien lassen sich daher weiter abspielen. Und da diverse Einschränkungen wegfallen, lassen sich mit Avidemux erstellte Dateien nun direkt verwenden.

Erfolgreich getestete Formate:

Container-Formate: AVI, MP4
Videocodecs: DivX, Xvid
Audiocodecs: MP2, MP3, AC3
Videofrequenz (Bilder/Sekunde): 20, 25
Bildformate: JPEG, BMP

Hier ein Konvertierbefehl im Bausatz

Dateinamen, Ausgabeformat: AVI
mencoder -noodml QUELLDATEI -of avi -o ZIELDATEI

Videocodec XVID, Bitrate
-ovc xvid -xvidencopts bitrate=VIDEOBITRATE:quant_type=h263:me_quality=6
VIDEOBITRATE: je nach Qualität, typische Werte 800 bis 1000

Audiocodec
-oac AUDIOCODEC
AUDIOCODEC: mp3lame, mp2, ac3

Beispiel:
mencoder -noodml ein.mp4 -of avi -o aus.avi -ovc xvid -xvidencopts bitrate=1000:quant_type=h263:me_quality=6 -oac mp3lame



Mittwoch, 19. Mai 2010

The wow factor book

Yesterday, a young man in Australia became victim of his own success.

Andrew Price of blenderguru.com fame had demonstrated in the past that he can produce good and easy to understand blender text and video tutorials.

He peaked the community's interest when he announced an e-book -  “the wow-factor” - about the compositor of Blender 2.5, something that hasn't got a lot of coverage so far.

During the last weeks he released a sample chapter of his book and produced a promotional video worthy of a shopping channel.  Apart from detailing the content of the e-book, it listed all the bonuses which come with it, and culminated in the display of the e-book itself and all the bonuses as CDs and booklets which, of course, all exist - in good Blender fashion - only virtually.

One day before the release he offered a chance to win a free copy of the e-book.  It got 700+ entries which should have been a warning about what was going to hit him.

With only a few hours to go, his website become slower and slower, and when the first message "error 500" appeared, it was clear that the demand crashed his server.  The move to a more powerful home took another 3+ hours which deprived a few of his costumers of the chance to be among the first hundred buyers eligible for “the vault” - the collection of source material which made up the book.

Before going to bed myself, I skimmed the e-book.  It delivers what was promised: an overview of glares, blurs and optical effects - many of which most of us have never heard of - and how to create them in Blender. 

The bonuses provide additional information which are not available in this completeness at this time anywhere else.  The “node encyclopaedia”  for example will save you hours of gathering this information and will hopefully become part of the regular Blender documentation once the final 2.5 version is released, which can still take some time.

The reason that the compositor is not the most prominent feature in most other Blender tutorials is, that it is one of the last steps of creating something in Blender used to polish your render.  This means that you must already have something to work with – a fact that has been taken care of by two video tutorials showing how to build a scene in which to use the effects.  Unfortunately they are only available in a time limited fashion.

All in all it seems to provide a lot of useful information which I can hopefully learn during the coming weeks while reading, watching and listening to the provided material.

I wonder if there were free bonus steak knives mentioned in the commercial... .blend files would suffice. :-)

Freitag, 7. Mai 2010

dbus tutorial - part 3

The last part started with the basic structure of the dbus part of the CD file listing utility and then showed the particular problems with the HAL implementation.

DeviceKit.Disks has to overcome similar problems, but due to the different concepts behind DeviceKit they have to be solved slightly differently.

DeviceKit.Disks

Like HAL, DeviceKit.Disks is another daemon handling devices and is offering similar services under different names. HAL is a jack of all trades, providing information about nearly everything in your system. This lead to its downfall, because the code base became huge and unmaintainable.  DeviceKit on the other hand is split into individual more specialised daemons.  DeviceKit.Disks is strictly focussed on disks and drives.

Differences to HAL from a programmers point of view:
  • As already seen: A different function is used to install the callbacks.
  • The string given to the callback function (e.g. /org/freedesktop/DeviceKit/Disks/devices/sr0)  is the name of a Devkit device. It points to a hardware device (e.g. the CDROM drive) not a volume (the inserted medium).  And because the string already points to the hardware device we don't have to check for any parent object.
  • Being a fixed drive, we also don't need the device_added_callback or device_removed_callback, because the physical CDROM device is not likely to be removed during the lifetime of the program and only changing the medium does not trigger these callbacks. (Note: A USB stick where the physical device itself is being unplugged does trigger these callbacks.).
  • Which leaves the device_changed_callback. However, unlike HAL, there is no indication which property has changed when it is being triggered. We have to track the appropriate properties ourselves.
Initialization
(See part 2)

import gobject
import dbus
...
gobject.threads_init()
dbus.glib.threads_init()


Adding the callback
(See part 1)

bus = dbus.SystemBus()
devkit_object = bus.get_object("org.freedesktop.DeviceKit.Disks",
    "/org/freedesktop/DeviceKit/Disks")
devkit_disks = dbus.Interface(devkit_object, 'org.freedesktop.DeviceKit.Disks')
devkit_disks.connect_to_signal('DeviceChanged', device_changed_callback)

...

def device_changed_callback(device):
    print 'Device %s was changed' % (device)


Device properties in Devicekit.Disks

DeviceKit.Disks devices implement an interface named Properties which gives access to the device properties:

device_object = bus.get_object("org.freedesktop.DeviceKit.Disks", udi)
device_prop = dbus.Interface(device_object, "org.freedesktop.DBus.Properties")
property = device_prop.Get("org.freedesktop.DeviceKit.Disks.Device","name_of_property")

# or

all_properties = device_prop.GetAll("org.freedesktop.DeviceKit.Disks.Device")


Interesting properties are:
  • DriveMediaCompatibility
    description: hardware compatibility
    typical values: optical_cd, optical_cd_r, optical_cd_rw, optical_dvd, optical_dvd_r,  optical_dvd_ram
  • DeviceIsOpticalDisc
    typical values: 0/1
    0: disk tray empty
    1: optical media has been found
  • DeviceIsMounted
    typical values: 0/1
    0: nothing mounted yet
    1: valid DeviceMountPaths
  • IdLabel
    description: label of CD/DVD
  • DeviceMountPaths
    descirption: mount point, if a CD/DVD has been mounted
  • OpticalDiscIsBlank
    typical values: 0/1
    0: media in the drive is not blank
    1: media in the drive is blank
  • DeviceIsRemovable
    typical values: 0/1
    1: device can contain removable media (-> Eject should work)
See also: http://hal.freedesktop.org/docs/DeviceKit/

You can get a quick overview with human readable property names with

devkit-disks --dump | more

How to filter out events that are not CD related

The most promising way is to check DriveMediaCompatibility for the value optical_cd or optical_dvd.

device_object = bus.get_object("org.freedesktop.DeviceKit.Disks", udi)
device_prop = dbus.Interface(device_object, "org.freedesktop.DBus.Properties")
md = device_prop.Get("org.freedesktop.DeviceKit.Disks.Device","DriveMediaCompatibility")
print "is CD-ROM:", (u"optical_cd" in md) or (u"optical_dvd" in md)


If you observe the events (using the dbus-monitor) when inserting or ejecting a CD, the following patterns emerge:
(I haven't found any authoritative documentation yet, but the steps seem logical):
  • When the CDROM tray is empty:
    DeviceIsOpticalDisc == 0
  • Blank medium inserted:
    DeviceIsOpticalDisc == 1
    OpticalDiscIsBlank == 1
  • After ejecting the blank medium:
    DeviceIsOpticalDisc == 0

  • Insert a medium containing data
    • step 1 (medium gets recognized):
      DeviceIsOpticalDisc == 1
      OpticalDiscIsBlank == 0
      DeviceIsMounted == 0
    • step 2 (medium gets auto mounted):
      DeviceIsOpticalDisc == 1
      OpticalDiscIsBlank == 0
      DeviceIsMounted == 1
  • eject medium:
    • step 1 (auto umount)
      DeviceIsOpticalDisc == 1
      OpticalDiscIsBlank == 0
      DeviceIsMounted == 0
    • step 2 (eject)
      DeviceIsOpticalDisc == 0
      OpticalDiscIsBlank == 0
      DeviceIsMounted == 0

  • If DeviceIsOpticalDisc is being set to 1, the property IdLabel becomes valid.
  • If DeviceIsMounted is being set to 1, the property DeviceMountPaths becomes valid.
When to read the file list

As DeviceKit.Disks does not provide the information which property was modified, we have to track the state of DeviceIsMounted ourselves and read the files below the mount point(s) when DeviceIsMounted changes from 0 to 1.

Eject via software

This function is located in the interface org.freedesktop.DeviceKit.Disks.Device. Note that a volume must be unmounted first before it can be ejected (see property DeviceIsMounted).

system_bus = dbus.SystemBus()
device_object = system_bus.get_object("org.freedesktop.DeviceKit.Disks", udi)

# get properties
device_prop = dbus.Interface(device_object, "org.freedesktop.DBus.Properties")
all_properties = device_prop.GetAll("org.freedesktop.DeviceKit.Disks.Device")

# call unmount/eject depending on the device state
disk_dev_func = dbus.Interface(device_object, "org.freedesktop.DeviceKit.Disks.Device")
if all_properties['DeviceIsMounted'] == 1:
    disk_dev_func.FilesystemUnmount('')
if all_properties['DeviceIsOpticalDisc'] == 1:
    disk_dev_func.DriveEject('')


With this, we have all the building blocks for DeviceKit.disks implementation of the CD file listing utility