Archive for the ‘python’ Category

Python cmd.exe UnicodeEncodeError

Monday, October 11th, 2010

Problem

I have written a script that calls SVN command using Popen. It looked something like this:

up = subprocess.Po­pen([SVN, „up“, file, ‚-r‘, revision], cwd=dest, stdout=subpro­cess.PIPE).com­municate()[0]

It seemed to work OK until I got a bug report including this trace:

Traceback (most recent call last):
  File "E:\zdir\checkout.py", line 212, in <module>
    up = subprocess.Popen([SVN, "up", file, '-r', revision], cwd=dest,
stdout=subprocess.PIPE).communicate()[0]
  File "C:\Python27\lib\subprocess.py", line 672, in __init__
    errread, errwrite)
  File "C:\Python27\lib\subprocess.py", line 882, in _execute_child
    startupinfo)
UnicodeEncodeError: 'ascii' codec can't encode character u'\xfc' in position
103: ordinal not in range(128)

The string that did not get processed was Änderungen and the problem occurred only if the script was run from cmd.exe (it worked OK from Linux shell and Windows PyDev console).

Solution

Having tried many things which all ended in UnicodeEncodeError. The only solution I found working was to use encode method. locale.getpre­ferredencodin­g() seems to be the right choice to keep the program portable. If you don't provide the encode() method with this argument, it will again end up in some UnicodeEncodeError.

The result code looks like this:

up = subprocess.Po­pen([SVN, „up“, file.encode(lo­cale.getprefe­rredencoding()), ‚-r‘, revision], cwd=dest, stdout=subpro­cess.PIPE).com­municate()[0]