Python cmd.exe UnicodeEncodeError

October 11th, 2010


I have written a script that calls SVN command using Popen. It looked something like this:

up = subprocess.Po­pen([SVN, „up“, file, ‚-r‘, revision], cwd=dest, stdout=subpro­cess.PIPE).com­municate()[0]

It seemed to work OK until I got a bug report including this trace:

Traceback (most recent call last):
  File "E:\zdir\", line 212, in <module>
    up = subprocess.Popen([SVN, "up", file, '-r', revision], cwd=dest,
  File "C:\Python27\lib\", line 672, in __init__
    errread, errwrite)
  File "C:\Python27\lib\", line 882, in _execute_child
UnicodeEncodeError: 'ascii' codec can't encode character u'\xfc' in position
103: ordinal not in range(128)

The string that did not get processed was Änderungen and the problem occurred only if the script was run from cmd.exe (it worked OK from Linux shell and Windows PyDev console).


Having tried many things which all ended in UnicodeEncodeError. The only solution I found working was to use encode method. locale.getpre­ferredencodin­g() seems to be the right choice to keep the program portable. If you don't provide the encode() method with this argument, it will again end up in some UnicodeEncodeError.

The result code looks like this:

up = subprocess.Po­pen([SVN, „up“, file.encode(lo­cale.getprefe­rredencoding()), ‚-r‘, revision], cwd=dest, stdout=subpro­cess.PIPE).com­municate()[0]

Rational Team Concert Workflow Visualiser

April 9th, 2010


Example workflow diagram Workflow Visualiser is a tool that takes the RTC process configuration XML and exports graphical representation of the workflows included in the configuration.


The most typical task I'm facing at new RTC deployments is analysis of their workflow and its implementation in RTC. The usual procedure is:

  1. Meet with the customer to try to understand the workflow (states, transitions)
  2. Draw the workflow in Microsoft Visio and discuss the Visio files with customer
  3. Implement the workflow in RTC

This would be all great it there wasn't number 4 – implement the changes the customer finds after the implementation in RTC, which usually repeats several times. In this moment the Visio diagrams and the RTC implementation get out of sync and it is not easy to keep them synchronized. Workflow Visualiser helps in this step as I can export the real diagram and either send it to the customer for review or redraw it in Visio.

Another case in which this tool comes handy are big workflows (with 10< states). Although the transition table in RTC workflow designer is straightforward, checking such a workflow is not an easy task. Seeing a graphical representation of the workflow makes it clear.

Running the program

The program has to be invoked from command line and the command line arguments are:

Usage: java -jar visualiser.jar [options] source-xml-file
  -h|--help Print help and exit
  --version Print help and exit
  -d|--dot Location of dot executable (defaults to /usr/bin/dot)
  -o|--out Output directory (defaults to /tmp/)
  -f|--format Output format (see the output formats of dot) (defaults to pdf)
  -s|--source Save the DOT source file in the output directory


Windows example PDF
  • JAR stored in C:\visualiser.jar
  • Process XML configuration file saved in process.xml
  • Graphviz installed in standard location
  • Export diagrams in PDF format to „C:\temp“
java -jar "C:\visualiser.jar" -o "C:\temp" -d "C:\Program Files\Graphviz2.26.3\bin\dot.exe" process.xml

Windows example PNG
  • JAR stored in C:\visualiser.jar
  • Process XML configuration file saved in process.xml
  • Graphviz installed in standard location
  • Export diagrams in PNG format to „C:\temp“
java -jar "C:\visualiser.jar" -o "C:\temp" -d "C:\Program Files\Graphviz2.26.3\bin\dot.exe" -f png process.xml

Linux example
  • JAR stored in ~/visualiser.jar
  • Process XML configuration file saved in process.xml
  • Graphviz installed in standard location (/usr/bin/dot)
  • Export diagrams in PDF format to ~/diagrams
java -jar ~/visualiser.jar -o ~/diagrams process.xml



The program uses Graphviz for diagram drawing. You must have Graphviz installed on your computer to be able to run the program. It can be freely downloaded from


Download Runnable JAR

Source code

Source code is available at GitHub:…w-visualiser. It includes pom.xml with the Maven project dependencies.


Pro program is distributed under Apache License, Version 2.0. For the license agreement of the used libraries see the appropriate license agreement included in the distribution.

Version history

  • 2010–04–09 Initial version
  • 2010–04–09.2 Resolution edges corrected

Comments, bug reports

Any comments or bug reports are welcome. Send them either in the comments under this article or via email to

RQM on 64bit Ubuntu

February 7th, 2010

I installed IBM Rational Quality Manager on the same machine as I installed Rational Requirements Composer before. Some steps from installation tips may be needed.

The official product documentation does not give the exact DB2 command line command. If you use the command from RTC manual, the table creation will fail. This is caused by the pagesize being too small. Use this command to create the database instead:

db2 create database RQM using codeset UTF-8 territory en PAGESIZE 16384

RRC on 64bit Ubuntu

February 6th, 2010

Another post, where the main audience is me :)


To install IBM Rational Requirements Composer on 64bit Linux, you need 32bit libraries installed. Strange error message is displayed otherwise when trying to run the installer:

root@rtc:/opt/rrc-install/RRCS_SETUP/InstallerImage_linux# ./install
bash: ./install: No such file or directory

To install the needed libraries run:

apt-get install ia32-libs

The installation should run successfully after this.


RRC needs swt-gtk to run. Install it by:

apt-get install libswt-gtk-3.4-jni

In addition, RRC needs a X screen to start. There is an IBM Technote with solution for RHEL and Suse. To install Xvfb on Ubuntu do:

apt-get install xvfb

The rest of the procedure (starting the server) is the same as on RHEL or SLES (see the technote).

The server is started by server.startup. This script has to be called from its directory – you need to cd first. The init script can look for example like this:

# rrc        Startup script for the RRC
# chkconfig: - 90 16
# description: Rational Requirements composer  \
# processname: java
# pidfile: /var/run/


start() {
        echo -n $"Starting $prog: "
        # export GTK_PATH=/usr/lib32/gtk-2.0
        Xvfb :1 -screen 0 800x600x24&
        cd ${rrchome}
        DISPLAY=localhost:1.0 ./server.startup
        [ $RETVAL = 0 ] && touch ${lockfile}
        return $RETVAL

stop() {
        echo -n $"Stopping $prog: "
        killall Xvfb
        [ $RETVAL = 0 ] && rm -f ${lockfile} ${pidfile}

# See how we were called.
case "$1" in
        status $prog
        echo $"Usage: $prog {start|stop|restart}"

exit $RETVAL

OS details: Ubuntu 9.04, Server edition, 64bit

IBM Installation Manager RHEL 4 64bit

January 8th, 2010

I'm currently installing Jazz CALM solution on CentOS 4 64bit (equivalent to RHEL 4 64bit). I got stuck when installing both Rational Quality Manager (RQM) and Rational Requirements Composer (RRC). Both need to be installed via IBM Installation Manager. RRC installation package includes IBM Installation Manager, whereas RQM doesn't.

To install IBM Installation manager on 64bit RHEL 4 you first need to install these:

yum install gtk2.i386 libgcc.i386

I first installed libgcc.i386 only to get rid of the error message must be installed for pthread_cancel to work

After this, there was no error message, but the installer returned exit code 13 (which I haven't found documented anywhere). Installing 32bit gtk library solved the problem.

The IBM Installation Manager itself can be downloaded from…alsdp/v7/im/

I hope this helps somebody save some time of desperate googling for solution…

RTC 2.0 on Ubuntu

October 31st, 2009


  • RTC Server version:
  • OS: Ubuntu 9.04, 64bit. Server edition
  • HW: VMWare guest, 64bit, 1GB RAM


Installation of Rational Team Concert is in general as easy as unzipping the downloaded archive. I did this recently on Ubuntu 9.04 64bit, server edition. Later I have encountered errors when generating any report (for example Floating License Report). I got this message:


The post at forums suggested, that there are missing libraries required by file */jazz/server/jre/bin/­libfontmanager­.so You can find out which libraries are missing by running

ldd jazz/server/jre/bin/­libfontmanager­.so

My output was: =>  (0x00007fff5efad000) => /lib/ (0x00007fc04eed9000) => /lib/ (0x00007fc04ecbd000) => not found => not found => not found => /lib/ (0x00007fc04ea36000) => /usr/lib/ (0x00007fc04e72f000) => /lib/ (0x00007fc04e3bd000)
/lib64/ (0x00007fc04f26f000) => /usr/lib/ (0x00007fc04e1a0000) => /usr/lib/ (0x00007fc04df9d000) => /usr/lib/ (0x00007fc04dd98000)


The system library that was missing in my system (using default installation of Ubuntu 9.04 64bit, Server edition) was This library is included in the package libstdc++5. It can be installed by running:

apt-get install libstdc++5

The RTC server has to be restarted afterwards.

jazz/server/ser­ver.shutdown jazz/server/ser­ver.startup

Ubuntu: change default encryption algorithm

July 12th, 2009

While fighting with DB2 denying logins I had to change the default hash algorithm used to store passwords in /etc/shadow.

As far as I know (and as grep -i -r "sha512" /etc/* tells me :-) ), there are two files that carry this information. Namely /etc/pam.d/common-password and /etc/login.defs.

To change the algorithm from sha512 (which shifts as the default in the new Ubuntu releases) change this:


password [success=1 default=ignore] obscure sha512


password [success=1 default=ignore] obscure md5





HOWTO: add disk to VMware Ubuntu guest without reboot using LVM

July 10th, 2009

This howto is mostly built on the great tutorial available at…hout-reboot/. My spot tries to be more Ubuntu and LVM specific. This manual was tested on Ubuntu 9.04 (jaunty) Server edition guest OS.

Add Virtual Hard Disk

add-disk1 A new hard disk can be added easily to a running virtual machine in VMWare Infrastructure Web Access by clicking Add Hardware and following the wizard.

Note: I assume that all of the commands given are run under superuser (root) rights. To become root type

sudo su root

Detect new SCSI Disk

The package scsitools provides the script, which does exacly what we need. Install it and run it:

apt-get install scsitools

This should detect the newly added hard disk and make it available in /dev/ – /dev/sdb in the case you are adding a new drive for the first time. The output of fdisk -l should include the newly detected disk.

Format the new disk

Format the newly detected drive using fdisk

fdisk /dev/sdb
    n (for new partition)
    p (for primary partition)
    1 (partition number)
    (keep the other values default)
    w (write changes)

Change the type of the partition to LVM Linux:

fdisk /dev/sdb
    t (change the partition type)
    8e (for Linux LVM)
    w (write changes)


Initialize LVM Physical Volume

pvcreate /dev/sdb1

Add Physical Volume to Volume Group

List the Volume groups available at your machine.


Replace VolGroupName with what you read from the output of the last command.

vgextend VolGroupName /dev/sdb1

You should see the newly aquired free space in the output of


Resize Logical Volumes

After adding some free space to your Volume Group, you can distribute it to the Logical Volumes. The list of Logical Volumes can be obtained by running


Run the following commands for each Logical Volume you want to resize (add space to). Again – replace /dev/VolGroupNa­me/Name with the Logical Volume name you can read from the output of the last command. The +10G means Add 10 GB

lvextend -L +10G /dev/VolGroupName/Name

The last step is to resize the ext3 filesystem (use different tool is you use different filesystem):

resize2fs /dev/VolGroupName/Name

Shibboleth Ubuntu

June 23rd, 2009

SP2 Installation

I switched one server from Fedora Core to Ubuntu yesterday. Shibboleth 2 SP installation is a bit more tricky under Ubuntu than Fedora Core. The packages are not available in the Ubuntu repositories, but Satya's blog serves a working howto.

failed to bind to socket

Shibboleth seemed to be working well after installation, but today it was down dumping

2009–06–23 12:51:53 ERROR Shibboleth.Listener : socket call resulted in error (2): no message 2009–06–23 12:51:53 CRIT Shibboleth.Listener : failed to bind to socket.

into the /var/log/shibboleth/shibd.log. I luckily found a solution in quite a short time. The problem was that the shibd was trying to create socket in /var/run/shibboleth directory, which did not exist. Creating it once does not help because Ubuntu removes content of /var/run/ at every reboot. To fix it this has to be added to /etc/init.d/shibd

if [ ! -e /var/run/shibboleth ]
   mkdir /var/run/shibboleth

DB2: SQL30082N Security processing failed with reason “15″

May 11th, 2009

If you are getting

SQL30082N Security processing failed with reason „15“ („PROCESSING FAILURE“)

when connecting to the db2 database using db2 CONNECT TO dbname USER username or

Connection authorization failure occurred. Reason: Local security service non-retryable error. ERRORCODE=-4214, SQLSTATE=28000

when connecting remotely (via JDBC driver) read on. I finally found what causes this error and a solution!




Tho whole problem is in the format of the passwords in /etc/shadow. DB2 doesn't seem to like the passwords generated when changing password using the standard passwd command. In Fedora Core 10 the password is hashed using sha-512 and the entry for user username looks like:



DB2 works fine with passwords hashed with standard crypt function. Password in the desired format can be obtained by calling openssl passwd desiredPassword. The output of openssl can be passed to usermod --password The complete command to change user's password then look like:

usermod --password `openssl passwd desiredPassword` usename

Bigger image

I tried to hand-edit /etc/shadow and insert salted MD5 password, which can be obtained by openssl passwd -1 desiredPassword and the connection was sucessfuly established. To sum it up, the whole problem in the end seems to be that DB2 doesn't like SHA-512 hashes in /etc/shadow. On the other hand, it works fine with hashes generated by crypt and MD5.

Changing default algorithm in Fedora Core

Fedora Core contains a nice tool authconfig. To change the default hashing algorithm to MD5 run

authconfig –passalgo md5 –update

All the passwords inserted in the password database will be stored in MD5 from now on. It will, of course, not change the hashes of the current passwords.


Ubuntu ships with sha512 as default hash algorithm for passwords in /etc/shadow. The easiest way to change the default algorithm is to edit the file /etc/pam.d/common-password and change the line that reads

password [success=1 default=ignore] obscure sha512


password [success=1 default=ignore] obscure md5

The information about the algorith is also included in the file /etc/login.defs, which is used by chpasswd for example. Therefore change the line