AES encpytion in Python using C extensions

This blog post gives example code how to do AES encryption and decryption in Python. The crypto implementations shown are written in C, but they have a Python interface. For pure Python implementations, have a look at Python_AES in tlslite or SlowAES.

Example Python code using the AES implementation in alo-aes:

#! /usr/bin/python2.4
# by pts@fazekas.hu at Sat Nov  7 11:45:38 CET 2009

# Download installer from http://www.louko.com/alo-aes/alo-aes-0.3.tar.gz
import aes

# Must be 16, 24 or 32 bytes.
key = '(Just an example for testing :-)'

# Must be a multiple of 16 bytes.
plaintext = 'x' * 16 * 3

o = aes.Keysetup(key)
# This uses ECB (simplest). alo-aes supports CBC as well (with o.cbcencrypt).
ciphertext = o.encrypt(plaintext)

assert (ciphertext.encode('hex') == 'fe4877546196cf4d9b14c6835fdeab1a' * 3)
assert len(ciphertext) == len(plaintext)
assert ciphertext != plaintext  # almost always true
decrypted = o.decrypt(ciphertext)
assert plaintext == decrypted

print 'benchmarking'
plaintext = ''.join(map(chr, xrange(237)) + map(chr, xrange(256))) * 9 * 16
assert len(plaintext) == 70992
for i in xrange(1000):
  assert plaintext == o.decrypt(o.encrypt(plaintext))

Example Python code using the AES implementation in PyCrypto:

#! /usr/bin/python2.4
# by pts@fazekas.hu at Sat Nov  7 12:04:44 CET 2009
# based on
# http://www.codekoala.com/blog/2009/aes-encryption-python-using-pycrypto/

# Download installer from
# http://ftp.dlitz.net/pub/dlitz/crypto/pycrypto/pycrypto-2.1.0b1.tar.gz   
from Crypto.Cipher import AES

# Must be 16, 24 or 32 bytes.
key = '(Just an example for testing :-)'

# Must be a multiple of 16 bytes.
plaintext = 'x' * 16 * 3

o = AES.new(key)
# This uses ECB (simplest). PyCrypto aes supports CBC (with AES.new(key,
# aes.MODE_ECB)) as well.
ciphertext = o.encrypt(plaintext)

assert (ciphertext.encode('hex') == 'fe4877546196cf4d9b14c6835fdeab1a' * 3)
assert len(ciphertext) == len(plaintext)
assert ciphertext != plaintext  # almost always true
decrypted = o.decrypt(ciphertext)
assert plaintext == decrypted

print 'benchmarking'
plaintext = ''.join(map(chr, xrange(237)) + map(chr, xrange(256))) * 9 * 16
assert len(plaintext) == 70992
for i in xrange(1000):
  assert plaintext == o.decrypt(o.encrypt(plaintext))

Please note how similar the two example codes are. (Only the import and the object creation are different.)

On an Intel Centrino T2300 1.66GHz system, PyCrypto seems to be about 8% faster than alo-aes. The alo-aes implementation is smaller, because it contains only AES.

How to use \showhyphens with beamer

When using \showhyphens with the beamer LaTeX class, the output of \showhypens doesn't appear on LaTeX's terminal output or in the .log file. The solution is to specify \rightskip=0pt, e.g.

\showhyphens{\rightskip0pt pseudopseudohypoparathyroidism}

results

Underfull \hbox (badness 10000) in paragraph at lines 9--9
[] \OT1/cmss/m/n/10.95 pseu-dopseu-do-hy-poparathy-roidism

Tiny, self-contained C compiler using TCC + uClibc

This post presents how I played with a small C compiler and a small libc on Linux. I've combined TCC (the Tiny C Compiler) 0.9.25 and uClibc 0.9.26, compressed with UPX 3.03 to a tiny, self-contained C compiler for Linux i386. You don't need any external files (apart from the compiler executable) to compile (or interpret) C code, and the compiled executable will be self-contained as well.

Here is how download and use it for compilation on a Linux x86 or x86_64 system:

$ uname
Linux
$ wget -O pts-tcc http://pts-mini-gpl.googlecode.com/svn/trunk/pts-tcc/pts-tcc-0.9.25
$ chmod +x pts-tcc
$ ls -l pts-tcc
-rwxrwxr-- 1 pts pts 241728 Nov  1 13:07 pts-tcc
$ wget -O example1.c http://pts-mini-gpl.googlecode.com/svn/trunk/pts-tcc/example1.c
$ cat example1.c
#! ./pts-tcc -run
int printf(char const*fmt, ...);
double sqrt(double x);
int main() {
  printf("Hello, World!\n");
  return sqrt(36) * 7;
}
$ ./pts-tcc example1.c
$ file a.out
a.out: ELF 32-bit LSB executable, Intel 80386, version 1 (SYSV), statically linked, not stripped
$ ls -l a.out
-rwxrwxr-x 1 pts pts 17124 Nov  1 13:17 a.out
$ ./a.out; echo "$?"
Hello, World!
42
$ strace -e open ./pts-tcc example1.c
open("/proc/self/mem", O_RDONLY)        = 3
open("/proc/self/mem", O_RDONLY)        = 3
open("example1.c", O_RDONLY)            = 3
open("/proc/self/mem", O_RDONLY)        = 3
open("/proc/self/mem", O_RDONLY)        = 3
open("a.out", O_WRONLY|O_CREAT|O_TRUNC, 0777) = 3
$ strace ./a.out
execve("./a.out", ["./a.out"], [/* 47 vars */]) = 0
ioctl(0, SNDCTL_TMR_TIMEBASE or TCGETS, {B38400 opost isig icanon echo ...}) = 0
ioctl(1, SNDCTL_TMR_TIMEBASE or TCGETS, {B38400 opost isig icanon echo ...}) = 0
write(1, "Hello, World!\n", 14Hello, World!
)         = 14
_exit(42)                               = ?

As you can see above, my version the compiler is less than 250k in size, and it doesn't need any library files (such as libc.so) for compilation (as indicated by the strace output above), and the compiled binary doesn't need library files either.

TCC can run the C program without creating any binaries:

$ rm -f a.out
$ ./pts-tcc -run example1.c; echo "$?"
Hello, World!
42
$ head -1 example1.c
#! ./pts-tcc -run
$ chmod +x example1.c
$ ./example1.c; echo "$?"
Hello, World!
42

TCC is a very fast compiler (see the speed comparisons on it web site), it is versatile (it can compile the Linux kernel), but it doesn't optimize much: the code it produces is usually larger and slower than the output of a traditional, optimizing C compiler such as gcc -O2.

The shell script http://pts-mini-gpl.googlecode.com/svn/trunk/pts-tcc/pts-tcc-0.9.25-compile.sh
builds TCC from source, adding uClibc, and compressing with UPX.

How to enable autologin in Ubuntu Jaunty with GDM

To enable automatic login in Ubuntu Jaunty running GDM (default) without clicking, add something like this to the [daemon] section of /etc/gdm/gdm.conf-custom:

TimedLoginEnable=true
TimedLoginDelay=10
TimedLogin=myusername

Once done, restart gdm (this will force an immediate logout):

$ sudo /etc/init.d/gdm restart

Using the Hama MCE remote on Linux with MPlayer

This blog post describes how to control your MPlayer media playback using the Hama MCE remote control on Linux. The instructions and software were tested on Ubuntu Jaunty, but they should work on any Linux 2.6 system which has sysfs mounted and which has USB HID support compiled to the kernel (can be checked with ls -l /sys/bus/usb/drivers/usbhid).

The Hama MCE dongle, when connected to the USB port of a Linux machine, registers itself as two USB HID devices: a keyboard and a mouse. Most of its buttons generate regular keyboard events, for example, the numeric keys correspond to numpad keys, and its Enter key corresponds to the Enter key. You can use its mouse controls to move the mouse or click (the left and right buttons). On Ubuntu Jaunty, even the volume buttons work (they adjust the master volume).

However, if you don't want to use your Hama MCE remote as a regular keyboard or mouse, but you'd like to control MPlayer (and possibly some few other applications you specify) with it, then you need special software. LIRC, the de facto standard remote control driver and server does support USB HID devices in general, but the Hama MCE sends some quite unusual events which LIRC seems to be impossible to make recognize. (For example, the hashmark button sends Shift, 3 and 5, and some other buttons send Shift or Control too, and LIRC doesn't seem to be able to track the Shift and Control state, which would be needed to distinguish some buttons from each other.)

So I've implemented my own lircd, hama_mce_lircd.py, which can read button presses and other events from the Hama MCE remote, and it can broadcast them to applications via the socket /dev/lircd, using the traditional lircd protocol. The link above contains installation, usage and configuration instructions for controlling MPlayer with Hama MCE, using hama_mce_lircd.py.

Each button works and can be bound to any MPlayer input.conf command, with the following limitations (of the dongle hardware):

• the ok and enter buttons are the same;
• the play and pause buttons are the same;
• the right click and the info buttons are the same;
• most buttons don't repeat when held down for a long time.

Design and print your own geek clock with LaTeX and TikZ

Would you like to have your custom clock for geeks (geek clock, math clock)? You can
design and print the front plate of your own analog geek clock using LaTeX and TikZ. Here is how I did it (download):

% geek_clock.tex
% by pts@fazekas.hu at Thu Oct 22 14:24:35 CEST 2009
\documentclass[a4paper]{article}
\usepackage{tikz}
\usepackage[latin2]{inputenc}
\usepackage{t1enc}
\usepackage{lmodern}

\pdfpagewidth=\paperwidth
\pdfpageheight=\paperheight

\newenvironment{fullpage}{%
  \shipout\vbox\bgroup\kern-1in\moveleft1in\vbox to\paperheight\bgroup
  \parindent0pt\hsize\paperwidth
}{%
  \vfil\egroup\egroup
}

\begin{document}

% Define a few constants for easy configuration
\def\framehalf{9.2cm}
\def\radius{8.8cm}
\def\onedegrad{8.6cm}
\def\fivedegrad{8.5cm}
\def\tendegrad{8.2cm}
\def\labelrad{8.3cm}

\begin{fullpage}
  \vfil\noindent\hfil
  \begin{tikzpicture}[scale=1]
    \draw (-\framehalf,-\framehalf) --
          (\framehalf,-\framehalf) --
          (\framehalf,\framehalf) --
          (-\framehalf,\framehalf) -- cycle;
    \draw (0,0) circle (\framehalf);
    \draw (0,0) circle (\radius);
    \draw[fill=black] (0,0) circle (2.5mm);
    \node[draw, circle, inner sep=.2mm] (a) at (0,0) {};
    % main lines
    \foreach \x in {0, 6,...,360}
        \draw[line width=1.5pt] (\x:\onedegrad) -- (\x:\radius);
    % labels and longer lines at every 6 degrees
    \foreach \x in {30, 60, ...,  360} {
        \draw[line width=4pt] (\x:\tendegrad) -- (\x:\radius);
    }
    \node[scale=3,anchor=north east] at (450-1*30:\labelrad)
      {$B'_L$\kern-1ex};
    \node[scale=2.5,anchor=east] at (450-2*30:\labelrad)
      {\lower8ex\hbox{$\displaystyle \sum_{i=0}^{\infty}1/2^i$\kern-2ex}};
    \node[scale=3,anchor=east] at (450-3*30:\labelrad)
      {$7\odot-5$};
    \node[scale=3,anchor=south east] at (450-4*30:\labelrad)
      {$\lceil\pi\rceil$};
    \node[scale=3,anchor=south east] at (450-5*30:\labelrad)
      {$(2\varphi-1)^2$\kern-4ex};
    \node[scale=3,anchor=south] at (450-6*30:\labelrad)
      {$3!$};
    \node[scale=3,anchor=south west] at (450-7*30:\labelrad)
      {$\!6.\overline{9}$};
    \node[scale=4,anchor=west] at (450-8*30:\labelrad)
      {\raise1ex\hbox{${\bullet}{\circ}{\circ}{\circ}$}};
    \node[scale=2.5,anchor=west] at (450-9*30:\labelrad)
      {$\lfloor(7\cdot.1\!-\!.7)^{\textrm{-6e-2}}\rfloor$};
    \node[scale=2.5,anchor=west] at (450-10*30:\labelrad)
      {\lower6ex\hbox{$\displaystyle{5\choose2}$}};
    \node[scale=2.5,anchor=north west] at (450-11*30:\labelrad)
      {\lower3ex\hbox{\kern-3ex$7^5\mathop{\mathrm{mod}} 13$}};
    \node[scale=3,anchor=north] at (450-12*30:\labelrad)
      {$\sqrt[3]{1728}$};
  \end{tikzpicture}
\end{fullpage}

\end{document}

I used pdflatex and TikZ in TeX Live 2008 to compile the file above to a PDF file, which I printed, cut and glued to a stock clock.

References:

• Original math clock product page, out of stock right now.
• Explanation for the formulas on the original math clock
• Forum thread giving an example how to draw an analog clock in TikZ

Screen blanking, DPMS, screen saver control and timeout settings on X11

This blog post explains how to blank and unblank the screen manually, how to activate and deactivate display power saving (DPMS), and how to specify inactivity timeouts for screen blanking on X11. The examples were tried with Ubuntu Jaunty, X.Org 7.4 and icewm. This blog post doesn't explain how to make your changes permanent after logout, or how to change your GNOME settings.

The results below may be distorted if gnome-screensaver is running. To get rid of it, do this:

$ gconftool-2 --type boolean -s /apps/gnome_settings_daemon/screensaver/start_screensaver false
$ killall gnome-screensaver

Without setting start_screensaver to false above, gnome-settings-daemon would restart gnome-screensaver whenever a GNOME application gets started. (Firefox a GNOME Terminal is GNOME applications in Ubuntu Jaunty, but icewm and xterm aren't.)

X11 supports two distinct features for power saving when the user is away from the computer: screen blanking and power saving. Confusingly enough, the man page for xset uses the term screen saver instead of screen blanking. In our terminology, screen blanking is a feature to display a blank (solid black) screen after long enough user inactivity. However, screen saver is a program which gets started upon long enough user inactivity, which draws some simple animation on the screen, and asks the user's password before letting him back to his screen and applications. Power saving (DPMS turns the display off or puts it to some power-saving mode after long enough user inactivity. In power saving mode the display doesn't show anything, but it consumes much less power, see the DPMS link above for typical watt values for the power-saving modes standby, suspend and off (and also in the mode on, when power saving is inactive). User inactivity means that the user doesn't use the keyboard or the mouse for a configurable number of seconds. The typical default inactivity timeouts are around 2 minutes. By default, the X server disables screen blanking and power saving as soon as the user presses a key or moves the mouse. From now on we assume that there is no screen saver running, or its timeout is set to be high enough so it won't ever trigger. Screen blanking and power saving are independent features, they have to be configured independently.

How to set up the timeouts

• To disable screen blanking, run xset s off
• To make the screen blank after 600 seconds of user inactivity, run xset s 600
To put the display to standby after 100 seconds, to suspend after 200 seconds, and turn it off after 300 seconds, run xset dpms 100 200 300
• To disable power saving, run xset dpms 0 0 0
• To make the screen blanking display thee X11 logo on a blue background instead of displaying a solid black screen, run xset s noblank
• To make the screen blanking display a solid black screen (default), run xset s blank

How to change the screen state immediately

• To blank the screen, run xset s activate
• To unblank the screen, run xset s reset
• To turn the screen off, run xset dmps force off
• To activate the suspend power-saving mode (good savings, good resume time), run xset dpms force suspend
• To activate the standby power-saving mode (minimal savings, very quick resume time), run xset dpms force standby
• To turn the display on, run xset dpms force on; xset s reset (the latter is necessary because the screen was blanked automatically when power saving was activated).

How to prevent screen blanking and power saving in MPlayer

Most media player applications disable blanking and power saving while they are playing a video. To enable this functionality in MPlayer, start it with mplayer -stop-xscreensaver or add the line stop-xscreensaver=1 to your ~/.mplayer/config . By doing so, MPlayer will simulate user activity (just as if the user has moved the mouse) upon startup and then each 30 seconds. So this will prevent the screen from blanking or going to power saving mode if all your timeouts are larger than 30 seconds. Please note that the name of the flag is confusing, because its effect isn't related to screen savers, it just simulates user activity. If a screen saver is running, it may or may not pick up the simulated activity generated by MPlayer. To prevent screen saver activation in this case, you may have to run mplayer -heartbeat-cmd "xscreensaver-command -deactivate" or mplayer -heartbeat-cmd "gnome-screensaver-command -p" .

JavaScript and ActionScript performance for big integer multiplication

This blog post documents the speed measurement of various JavaScript and ActionScript (and other) implementations multiplying 2048-bit big integers. I needed big integer multiplication for the Diffie-Hellman key exchange in one of my projects.

Since neither JavaScript nor ActionScript has big integer support built in, I had to implement it. I represent a big integer as a JavaScript array of digits in base 32768, in the base To multiply two 2048-bit integers use Karatsuba multiplication, which does 3 multiplications with 1024-bit operand size. To multiply two 1024-bit integers, I use Karatsuba multiplication (again), which does 3 multiplications with 512-bit operand size. To multiply two 512-bit integers, I use traditional O(n^2) ``schoolbook'' multiplication. (The Karatsuba multiplication would have been slower for this size.) All JavaScript and ActionScript code is heavily optimized, constants are inlined, and functions are instantiated for different operand sizes.

I did at least 500 multiplications of 2048-bit operand size. Here is the average number of milliseconds needed for 1 multiplication in the various implementations on an Intel Core 2 Duo 1.5 GHz machine running 32-bit Unbuntu Hardy:

• 00.0156 ms: Python (C) 2.4 gmpy (mpz of libgmp)
  
• 00.0396 ms: Java 1.6.0_16 BigInteger
  
• 00.0516 ms: Ruby (C) 1.9.1 Bignum
  
• 00.0650 ms: Python (C) 2.4 long
  
• 01.3030 ms: JavaScript Google Chrome V8 1.1.1.4
  
• 02.1750 ms: ActionScript 3 for Flash Player 10
  
• 02.7120 ms: JavaScript Google Chrome V8 1.1.1.4 using BigInt.js' mult()
  
• 04.5150 ms: JavaScript Firefox 3.5 TraceMonkey from Mercurial on 2009-10-04
  
• 05.0400 ms: JavaScript Firefox 3.5.2 TraceMonkey
  
• 05.2860 ms: JavaScript Firefox 3.0.14 TraceMonkey (?) JavaScript
  
• 06.9050 ms: ActionScript 3 for Flash Player 9 on Flash Player 9
  
• 08.5320 ms: ActionScript 3 for Flash Player 9 on Flash Player 10
  
• 10.2900 ms: JavaScript SpiderMonkey command-line interpreter 1.7.0 2007-10-03

The speed tests were run on multiple 32 and 64-bit Debian and Ubuntu installations, and the ratios came out almost the same as above.

The JavaScript and ActionScript (and some other) sources are avaialble at http://code.google.com/p/pts-mini-gpl/source/browse/#svn/trunk/javascript-multiplication.

Conclusion: If you need big integer arithmetic in the browser, detect Java and use its BigInteger from JavaScript. Otherwise, if the browser is Google Chrome, then do it in JavaScript. Otherwise, if there is Flash Player 10, do it in ActionScript 3. Otherwise, use JavaScript (and suggest installation of Java or Flash to the user).

How to create an ActionScript 3 (AS3) flash movie (SWF) without a GUI

This blog post explains how to create a Flash movie completely automatically, without using a GUI, just by writing a program in the language ActionScript 3 (AS3). The post gives instructions for Linux, but they should work with minor modifications on other operating systems as well since the compilers to be used are either written in Java or they are available for multiple systems.

ActionScript is a language similar to JavaScript, with some differences and incompatibilities. The most important difference is that you can declare types of variables (strongly recommended in ActionScript 3), and it has class-based inheritance, so classes are declared and use differently from JavaScript, more similarly to Java. To get a quick introduction to ActionScript, read http://en.wikipedia.org/wiki/ActionScript. The reference manual is hosted by Adobe, at http://livedocs.adobe.com/flash/9.0/ActionScriptLangRefV3/. The filename extension is .as.

ActionScript has two major versions in used today: ActionScript 2 (AS2) is supported by Flash Player 7, 8, 9 and 10. ActionScript 3 (AS3) is supported by Flash Player 9 and 10. ActionScript 2 is about 3.5 times (or even more) slower than the JavaScript interpreter in Firefox 3.0 (SpiderMonkey). ActionScript 3, as implemented in Flash Player 9, has just about the same speed as Firefox 3.0 JavaScript. ActionScript 3, as implemented in Flash Player 10, can be up to 4 times faster than FireFox 3.0 JavaScript if int is used instead of Number for integer computations, and Vector.<int> is used instead of Vector. Both Flash Players 9 and 10 have a JIT engine that compiles ActionScript to native code, but this doesn't seem to become faster than Firefox 3.0 JavaScript in some number-crunching applications.

It is possible to call a JavaScript function from ActionScript and vice versa, using the ExternalInterface ActionScript class (flash.external.ExternalInterface). Please note that ExternalInterface doesn't work for the file:/// protocol due to security reasons (as documented by Adobe here) – you have to upload both your .swf and .html files to the same web server to make use of ExternalInterface.

To compile ActionScript 2 to an SWF file, the free MTASC compiler is recommended, which is very fast, works at the command line, and is said to be safer than the compilers written by Adobe. An example ActionScript2 script:

// Tuto2.as, an example ActionScript 2 script
import flash.external.ExternalInterface;         
class Tuto2 {
  static var app : Tuto2;
  function Tuto2() {
    _root.createTextField("tf",0,0,0,800,200);
    _root.tf.mutliline = true;      
    _root.tf.text = "first";
    var myformat = new TextFormat();      
    myformat.color = 0x00000;      
    myformat.underline = true;      
    myformat.size = 30;
    // Set this, because the default font, "Times New Roman" may not be available on Linux.
    myformat.font = "serif";      
    _root.tf.setTextFormat(myformat);
    _root.tf.text = ExternalInterface.call + "\nHello";
    // Call this again after changing the text, otherwise the formatting is lost.
    _root.tf.setTextFormat(myformat);
  }
  static function main(mc) {  // Entry point.
    app = new Tuto2();
  }
}

To compile it, download MTASM first:

$ wget http://www.mtasc.org/zip/mtasc-1.12-linux.tgz
$ mkdir mtasc
$ cd mtasc
$ tar xzvf ../mtasc-1.12-linux.tgz
$ cp .../Tuto2.as .

Then compile with:

$ ./mtasc -version 8 -swf tuto2.swf -main -header 800:200:20 Tuto2.as

The corresponding HTML would look like this for Firefox:

<embed src="tuto2.swf"
       quality="high" bgcolor="#eeeeee" fgcolor="#000000" width="800" height="2
       name="tuto" id="tuto" align="middle" allowScriptAccess="always"
       allowFullScreen="true" type="application/x-shockwave-flash"
       pluginspage="http://www.macromedia.com/go/getflashplayer" />

To make it cross-browser, use SWFObject to embed it. Once created, load the HTML in your browser to view the Flash animation you've just created.

From now on we focus on ActionScript 3 and Flash Player 10. Adobe provides the Flex SDK, which contains a compiler (called mxmlc, or mxmlc.exe on Windows) which can compie ActionScript 3 .as source files to .swf. The Flex SDK is free to use and is partially open source, and it is implemented in Java. Download version 3.4 (which is the most recent stable version at the time of writing this blog post, and is about 120 MB compressed) from http://opensource.adobe.com/wiki/display/flexsdk/Downloads. Also make sure you have Java (at least a JRE) installed. Here are the corresponding download command-lines:

$ wget http://fpdownload.adobe.com/pub/flex/sdk/builds/flex3/flex_sdk_3.4.0.9271.zip
$ mkdir flex
$ cd flex
$ unzip ../flex_sdk_3.4.0.9271.zip
$ bin/mxmlc -help

Here is a sample ActionScript 3 program:

// Tuto3.as, a sample ActionScript3 program
package {
  import flash.display.Sprite;
  import flash.events.Event;
  import flash.text.TextField;
  import flash.text.TextFormat;
  import flash.utils.getTimer;
  import flash.external.ExternalInterface;
  public class Tuto3 extends Sprite {
    function Tuto3() : void {
      var circle : Sprite = new Sprite();
      circle.graphics.beginFill(0xFF7744);
      circle.graphics.drawCircle(0, 0, 30);
      circle.graphics.endFill();
      addChild(circle);
      //myInit();
      //addEventListener(Event.ADDED_TO_STAGE, myInit);
      ExternalInterface.addCallback("myFunction", myInit);
    }
    public function myInit(e : Event = null) : String {
      var startTime : Number = getTimer();
      var tField : TextField = new TextField();
      var tFormat : TextFormat = new TextFormat();
      tFormat.size = 20;
      // Set this, because the default font, "Times New Roman" may not be available on Linux.
      tFormat.font = "serif";
      tField.autoSize = "left";
      tField.background = true;
      tField.border = true;
      tField.multiline = true;
      tField.x = 20;
      tField.y = 40;
      tField.text = "hello1";
      tField.setTextFormat(tFormat);
      var endTime : Number = getTimer();
      trace('hello2 ' + startTime + ' ' + (endTime - startTime));
      tField.text = 'ms=' + (endTime - startTime);
      // Call this again after changing the text, otherwise the formatting is lost.
      tField.setTextFormat(tFormat);
      addChild(tField);
    }
  }  
}

Here is how to compile it (creates Tuto3.swf):

$ ./bin/mxmlc -target-player 10 -default-size 800 200 -optimize Tuto3.as

Here is the corresponding HTML:

<html><head><script type="text/javascript">
function myonload() {
  // document.tuto for Firefox, window.tuto for Internet Explorer
  var tuto = document.tuto || window.tuto || document.getElementById("tuto");
  if (tuto && tuto.myFunction)
    document.getElementById("result").innerHTML = tuto.myFunction();
}
</script></head><body onload="myonload()">
<embed src="Tuto3.swf"
       quality="high" bgcolor="#eeeeee" fgcolor="#000000" width="800" height="200"
       name="tuto" id="tuto" align="middle" allowScriptAccess="always"
       allowFullScreen="true" type="application/x-shockwave-flash"
       pluginspage="http://www.macromedia.com/go/getflashplayer" />
<div id="result">no result yet</div>
</body></html>

The HTML above also demonstrates calling an ActionScript method form JavaScript. To call a JavaScript function from ActionScript, do

ExternalInterface.call("JavaScriptFunctionName", arg1, ...)

Don't forget to copy the .html and the .swf to a http:// location, otherwise ExternalInterface won't work because the security limitations imposed by the Flash Player.

Please note that according to Adobe, AVM2, the virtual machine in Flash Player 9 and 10 for ActionScript 3 execution does a JIT for all functions and methods except for constructors. So don't put performance-critical code to the constructor. I couldn't measure any speed difference in the constructor and other methods. The last statement that the constructor is not JITted I could find is from 2007 (http://blog.pixelbreaker.com/flash/as30-jit-vs-interpreted/).

How to compile TraceMonkey for the Linux command-line

This blog posts describes how to compile TraceMonkey, Firefox's 3.5 fast JavaScript interpreter at the i386 32-bit Linux command-line. The created executable, js will not work in a browser, but it can be used for benchmarking JavaScript code (without HTML or DOM). The instructions below have been tested on an Ubuntu Hardy system.

The corresponding old, slow JavaScript interpreter (shipping Firefox 3.0 and older) is SpiderMonkey. The command-line version can be installed with

$ apt-get install spidermonkey-bin

After that, the command smjs file.js is available. In the executed JavaScript, the function print(expr) can be used to print a line to the terminal window, and load(filename) can be used to load another .js file.

To get the same functionality with TraceMonkey, you have to compile TraceMonkey for yourself. Here is how to do it:

$ sudo apt-get install gcc g++ make autoconf2.13 mercurial
$ hg clone http://hg.mozilla.org/tracemonkey/
$ cd tracemonkey/js/src
$ CC='gcc -m32' CXX='g++ -m32' AR=ar \
  ./configure --disable-debug --enable-optimize --target=i686-pc-linux-gnu
$ make
$ ls -l shell/js

This creates the shell/js executable, which can be used just like smjs.

To get a statically linked version of the TraceMonkey command-line interpreter, run this after the regular compilation with make:

$ bash -c '
  cd shell || exit 1; rm -f js
  function c++() { command c++ -static "$@"; }
  function g++() { command g++ -static "$@"; }
  eval "`make js`"; cd ..; ls -l shell/js'

More information about compiling and benchmarking TraceMonkey can be found on http://blog.mozilla.com/nnethercote/2009/07/27/how-i-work-on-tracemonkey/.

How to fix F-Spot tags for renamed files on Ubuntu Hardy

The photo manager F-Spot stores all picture file names in the SQLite database ~/.gnome2/f-spot/photos.db. This blog post explains how to update the database manually if picture files were renamed or moved. Without the manual update, F-Spot would not recognize the moved files, and it it will not see the tags and other information attached to them.

To view what files F-Spot knows about, install the command-line SQLite tool:

apt-get install sqlite3

Then get the list with

sqlite3 ~/.gnome2/f-spot/photos.db "SELECT uri FROM photos"

Please note that newer versions of Ubuntu have the database file as ~/.config/f-spot/photos.db instead, so you may have to change the command above.

Create a backup of the photos table in F-Spot database file:

sqlite ~/.gnome2/f-spot/photos.db ".dump photos" >f-spot.table_photos.sql

Copy the backup before modification:

cp f-spot.table_photos.sql f-spot.modified_photos.sql

Modify the file f-spot.modified_photos.sql in your favorite text editor to rename the files, and update the table with

sqlite ~/.gnome2/f-spot/photos.db "DROP TABLE photos"
sqlite -init f-spot.modified_photos.sql ~/.gnome2/f-spot/photos.db

As an alternative, if you don't want to use a text editor, run something like

sqlite3 ~/.gnome2/f-spot/photos.db "UPDATE photos SET uri=REPLACE(uri, '/OLD_DIR/', '/NEW_DIR/')"

If F-Spot doesn't notice the changes, restart F-Spot.

References

Another solution to the same problem: http://ubuntuforums.org/showthread.php?t=394241

How to refresh /dev/disk on Linux

To refresh /dev/disk/by-label or /dev/disk/by-uuid on Linux (tested on Ubuntu Hardy), run

sudo udevadm trigger

and wait a few seconds. It's OK that the system becomes unresponsive for 5 seconds.

How to get rid of PulseAudio on Debian and Ubuntu

This blog post describes how to get rid of pulseaudio (the PulseAudio sound server) on a Debian or Ubuntu system. The solution presented here is tested with Debian Etch, Ubuntu Hardy and Ubuntu Intrepid. As a side effect, the GNOME system sounds (the short sound effect played when clicking etc.) won't work.

PulseAudio contains a sound server used for software mixing, multiplexing (audio playback from multiple programs at the same time) and filtering. However, in some cases it is the sound server which actually prevents proper multiplexing, because it locks the sound card, so if a program (e.g. mplayer) is using pulseaudio for playback, and the second program (e.g. Skype) wants to use ALSA, the second program will not be able to start playback because the sound card is locked by pulseaudio. One simple solution is to get rid of pulseaudio, and let applications use ALSA with the default output device. This takes advantage of hardware mixing or Dmix, which does software mixing for simultaneous playbacks.

To get rid of the pulseaudio sound server, run:

$ sudo apt-get remove pulseaudio
$ sudo killall pulseaudio
$ sudo killall pulseaudio
pulseaudio: no process killed
$ (echo Package: pulseaudio; echo Pin: release a=fakerepo; echo Pin-Priority: 1999) |
  sudo tee -a /etc/apt/preferences

The last command (the one which appends to /etc/apt/preferences) is a hack to make the pulseaudio package impossible to install (i.e. apt-get install pulseaudio will fail with an unhelpful error message).

If you are logged in graphically, log out and log in again. (Logging out kills all running interactive applications.)

How to parse optional arguments without braces or brackets in TeX

This blog post shows the two winning solutions for the TeX argument parsing problem proposed by Kees van der Laan on the EuroTeX 2009 conference on 2009-08-31. My conclusion is that TeX macro programming, especially undelimited input parsing is tricky, and can become ugly since there are no powerful string inspection and matching operations built into TeX.

The basic problem

The basic problem is to write the argument parsing part of a TeX macro called \jpg, which can be used to include external (JPEG) images, optionally overriding the image width and/or height, like below:

\jpg file.jpg  % at original size
\jpg width5cm  file.jpg  % scale proportionally (keeping aspect ratio)
\jpg height6cm file.jpg  % ditto
\jpg width5cm height6cm file.jpg  % resize ignoring aspect ratio
\jpg height6cm width5cm file.jpg  % ditto

It is OK to assume that no file name starts with width, height or depth. The space after the dimension (e.g. 5cm) is mandatory.

Solution for the basic problem

This is my winning solution (download source) which solves the basic problem:

%
% jpgcmd_simple.tex: a simple braceless image inclusion TeX macro
% by pts@fazekas.hu at Mon Aug 31 13:42:49 CEST 2009
%
% This is one of the winning solutions for the problem proposed by Kees
% van der Laan on the conference EuroTeX 2009 (2009-08-31).

\def\jpgfilename#1 {%
  \egroup  % end of the \setbox0\vbox
  %\showthe\dp0
  \ifdim\dp0=0pt \else \errmessage{depth specified for jpg}\fi%
  %\pdfximage
  %  \ifdim\wd0=0pt  \else width\wd0\fi
  %  \ifdim\ht0=0pt  \else height\ht0\fi
  %  {#1}%
  %\pdfrefximage\pdflastximage
  \message{JPG file=#1 width=\the\wd0 \space height=\the\ht0;}%
  \endgroup}
\def\jpg{%
  \begingroup
  \setbox0\vtop\bgroup
  \hsize0pt \parindent0pt
  \everypar{\jpgfilename}%
  \hrule height0pt }

ABC\jpg height3cm smiley.jpg
DEF\jpg width3cm smiley.jpg
GHI\jpg width4cm height3cm smiley.jpg
JKL\jpg width3cm smiley.jpg

MNO\jpg smiley.jpg  % test for \par in the line below

PQR\jpg depth5mm smiley.jpg

\end

The basic solution above creates a vbox (using \vrule) containing an \hrule of the specified size, and then measures the size of the vbox. Thus the parsing of the optional width and height arguments gets delegated to to the TeX \hrule primitive. A \vtop is used instead of a \vbox so a depth can be detected.

A more versatile solution

This is another winning solution of mine (download source) which solves the basic problem, but it allows for more versatile optional arguments:

%
% jpgcmd_versatile.tex: a versatile braceless image inclusion TeX macro
% by pts@fazekas.hu at Thu Sep  3 11:01:07 CEST 2009
%
% This is one of the winning solutions for the problem proposed by Kees
% van der Laan on the conference EuroTeX 2009 (2009-08-31).

% If #2 starts with #1, then do #3{#z}, otherwise do #4. We get #z by removing
% #2 from the beginning of #1.

\def\ifprefix#1#2#3#4{%
  \ifprefixloop#1\hbox\vbox!#2\hbox\vbox!{#3}{#4}%

}

\def\firstoftwo#1#2{#1}
\def\secondoftwo#1#2{#2}
\def\ifxgroup#1#2{%
  \ifx#1#2\expandafter\firstoftwo\else\expandafter\secondoftwo\fi}
\def\striphbox#1\hbox\vbox!#2#3{#2{#1}}
\def\ifprefixloop#1#2\vbox!#3#4\vbox!{%
  \ifxgroup#1\hbox{\striphbox#3#4\vbox!}{%
  \ifxgroup#1#3{\ifprefixloop#2\vbox!#4\vbox!}\secondoftwo}}

% Tests.
\def\paren#1{(#1)}
\message{\ifprefix{}{barden}{1\paren}{0}!}

\message{\ifprefix{bar}{barden}{1\paren}{0}!}
\message{\ifprefix{bar}{bad}{1\paren}{0}!}

\message{\ifprefix{bar}{ba}{1\paren}{0}!}

\newdimen\jpgwidth
\newdimen\jpgheight
\newdimen\jpgscale

\def\jpg{%
  \jpgwidth0pt
  \jpgheight0pt
  \jpgscale0pt
  \jpgparse}

% #2 can start with or without =.
\def\jpgsetdimen#1#2{#1#2 \jpgparse}

\def\skipuntilhbox#1\hbox{}

% #2 can start with or without =.
% #2 can end with `pt' or not.
\def\jpgsetfloat#1#2{
  \afterassignment\skipuntilhbox
  #1#2pt\space\space\hbox\jpgparse}

\def\jpgdeptherror#1{%
  \errmessage{depth specified for jpg}\jpgparse}

\def\jpgparse#1 {%
  \ifprefix{height}{#1}{\jpgsetdimen\jpgwidth}{%
  \ifprefix{width}{#1}{\jpgsetdimen\jpgheight}{%
  \ifprefix{depth}{#1}{\jpgdeptherror}{%
  \ifprefix{scale}{#1}{\jpgsetfloat\jpgscale}{%
  \jpgshow{#1}}}}}}

\def\jpgshow#1{%
  \message{JPG file=#1 width=\the\jpgwidth\space height=\the\jpgheight\space
           scale=\the\jpgscale;}%
}
 
ABC\jpg height3cm smiley.jpg
DEF\jpg width3cm smiley.jpg 
GHI\jpg width4cm height=3cm smiley.jpg
JKL\jpg width3cm scale=4 smiley.jpg   

% test for \par in the line above

MNO\jpg width3cm scale=4pt smiley.jpg
PQR\jpg smiley.jpg
STU\jpg depth5cm smiley.jpg

\end

This solution accepts the optional argument scale with or without a unit after its optional argument (the default unit is pt), and it also accepts an equals sign (=) after the optional argument keywords. The implementation is a lot longer now since it has to parse the optional arguments manually. It uses some well-known TeX macro programming tricks to scan a string character-by-character. The \ifxgroup macro is worth mentioning: it is an \ifx, but it accepts the code for the then and else branches as brace-delimited arguments. Using braces here is a fundamental trick for implementing nested ifs and recursion, because otherwise the the macros in the then and else branches would receive \else and \fi (respectively) instead of the next token.

ASRock ION 330 nettop with Ubuntu 9.04 (Jaunty Jackalope)

This is a collection of random notes about using the ASRock ION 330 nettop with Ubuntu Jaunty. This nettop is a perfect candidate for a Linux-based HTPC for me:

• it has low power consumption (about 30 watts);
• it is quiet;
• its CPU is powerful (Intel Atom dual-core 1.6GHz);
• it has a hard drive of a decent size (320 GB);
• it can play full HD videos on Linux (with 35% CPU consumption) using the VDPAU acceleration;
• it can drive 2 monitors at the same time (one VGA, one HDMI or DVI);
• it is suspendable on Linux (not in every software configuration!);
• it is wakeable on LAN;
• almost all its hardware works out-of-the-box with Ubuntu 9.04 (Jaunty Jackalope), both i386 and amd64;
• it works with MythTV;
• it is cheap.

Installing Ubuntu Jaunty

Both the 32-bit (i386) and the 64-bit (amd64) desktop edition of Ubuntu Jaunty can be installed without problems either from CD or pen drive (created using UNetbootin, either a network or a normal install). The 64-bit edition is recommended, because the hardware-accelerated (VDPAU) video playback consumes much less CPU in 64-bit mode than in 32-bit mode (see below). The hard drive, the CD-ROM, the USB ports, the sound card and the LAN (ethernet) port work out-of-the box with Ubuntu Jaunty. The video controller, however, operates in compatibility mode (VESA) because Jaunty doesn't have the the proper display driver (see below for extra installation instructions), and the one shipping with Jaunty doesn't support the chipset.

Installing the nVidia display driver

Add the Medibuntu repository and the Avenard repository. Install packages with

sudo apt-get update && sudo apt-get install nvidia-glx-190 nvidia-190-libvdpau

If you don't have a Driver "nvidia" line in your /etc/X11/xorg.conf, then run sudo nvidia-xconfig to generate the proper lines. Make sure that your InputDevice in that file is not /dev/psaux; it should be Option "Device" "/dev/input/mice". Now do these as root:

# /etc/init.d/gdm stop  # kills your X session
# rmmod nvidia  # may fail
# modprobe nvidia
# echo nvidia >>/etc/modules
# /etc/init.d/gdm start

Now you should have full video acceleration

Suspending

Suspending works out-of-the-box in the Gnome menu for both 32-bit and 64-bit mode. Click with the mouse or press a key to resume.

To suspend remotely, issue command /etc/acpi/sleep.sh sleep in a root SSH session. (This doesn't work with the 64-bit version and the nVidia driver version 190: the system wakes up a few seconds after going to sleep. It does work with 64-bit and nVidia driver version 180. I remember it working with the 32-bit version, but I cannot remember for sure.)

To suspend so the computer can be woken up on ethernet, first run sudo ifconfig to get the MAC address of eth0, then run ethtool -s eth0 wol g; echo NMAC > /proc/acpi/wakeup as root (maybe one of these two commands is enough), then send it to sleep. To wake it up, run etherwake AA:BB:CC:DD:EE:00 (with its MAC address substituted) on another Linux machine on the local network. You may have to install the package etherwake (or something similar) for that.

Full HD video playback with mplayer

You need a recent nVidia display driver (see above) and a corresponding mplayer (from the Avenard repository). Version 190 of the nVidia display driver is recommended (from the testing Avenard repository). Version 185 may also work, but earlier versions (such as 180) don't support the ION chipset. If your package lists are set up correctly (Jaunty Universe + Jaunty Multiverse + Medibuntu + Avenard Release + Avenard Testing), apt-get install nvidia-glx-190 nvidia-190-libvdpau mplayer will install both the newest nVidia display driver and the mplayer which supports hardware-accelerated video playback. However, sometimes (e.g. on the 64-bit system for me), apt-get insists on installing and older mplayer, which would trigger the removal of nvidia-glx-190. The good mplayer package is in the Avenard repository, and it can be installed with something like this:

$ wget http://www.avenard.org/files/ubuntu-repos/testing/mplayer_1.0-svn29435-x264-vdpau-190-0ubuntu2_amd64.deb
$ sudo dpkg -i mplayer_1.0-svn29435-x264-vdpau-190-0ubuntu2_amd64.deb
$ sudo apt-get -f install

Play the video with mplayer -vo vdpau -vc ffmpeg12vdpau,ffwmv3vdpau,ffvc1vdpau,ffh264vdpau, FILENAME.VID. This will consume between 30% and 40% of CPU (as measured using top) with OSD or subititles and between 8% and 9% without OSD or subtitles. Please note that the comma at the end of the -vc argument specifies that MPlayer should try other video codecs if those specified in -vc don't work for the video being played. Please note that that -vo vdpau supports non-accelerated software codecs as well, but there is a performance cost (e.g. for a non-accelerated 320x200 video file I tried -vo xv consumed 5% CPU and -vo vdpau consumed 15% CPU), but -- unfortunately -- it is not possible to make MPlayer autodetect when to use -vo xv and when to use -vo vdpau.

If you don't have a full HD video file at hand, get one from http://www.apple.com/trailers/. Use the browser's View page source functionality to get the download URL, e.g. http://movies.apple.com/movies/fox/allaboutsteve/allaboutsteve-tlrc_1080p.mov You have to insert a h in front of 1080, so the download command would be

wget http://movies.apple.com/movies/fox/allaboutsteve/allaboutsteve-tlrc_h1080p.mov

Get more information about HD video playback on Ubuntu and an nVidia ION board at http://gagravarr.livejournal.com/137571.html. If you don't want to specify the accelerated codec flags every time on the command line, you should copy them to your ~/.mplayer/config file, for example:

vo=vdpau
vc=ffmpeg12vdpau,ffwmv3vdpau,ffvc1vdpau,ffh264vdpau,
#vo=xv
ao=alsa
#subcp=latin2
stop-xscreensaver=1
font=Arial
subfont-text-scale=3
subfont-osd-scale=3

HDMI and S/PDIF audio output

When diagnosing sound problems, make sure that pulseaudio is not running:

sudo killall pulseaudio

Please also make sure that no other program might be using the soundcard: disable system sounds, kill all web browsers (to kill the flash player), and kill all media players.

To get default audio output on the analog stereo jack, run

$ mplayer -ao alsa $FILE

To force audio output on the analog stereo jack, run

$ mplayer -ao alsa:device=front=NVidia $FILE

To force audio output on HDMI, run

$ mplayer -ao alsa:device=hdmi=NVidia $FILE

To force output on S/PDIF, run

$ mplayer -ao alsa:device=iec958=NVidia $FILE

To list all available audio output devices, run

$ aplay -L

Please note than when specifying anything other than -ao alsa to MPlayer, you may lose DMIX support, so you won't be able to run two playbacks at the same time, and MPlayer would print:

[AO_ALSA] Unable to set hw-parameters: Device or resource busy

It is possible to fix that by using plain -ao alsa and specifying the the output device in $HOME/.asourdrc. Using the environment variable ALSA_CARD does work, but ALSA_PCM_DEVICE doesn't seem to work. Set up your $HOME/.asoundrc like this:

defaults.pcm.!card NVidia
defaults.ctl.!card NVidia
defaults.pcm.!device 0
defaults.ctl.!device 0

The device number 0 is analog output, 1 is S/PDIF and 3 is HDMI. The device numbers come from the output of aplay -l.

How to disable Skype chat notification popup boxes on the Mac OS/X

This blog post explains how to disable the notification box with dark gray (black) background in the top right corner of the Mac OS/X screen whenever somebody sends you a Skype chat message.

You will not find the setting in Skype's preferences. To disable these notifications, you have to modify your Growl preferences. In the Apple menu (Apple logo in the top left corner of the screen), select System Preferences / Other / Growl / Application / Skype / Configure / Notifications. Here you can change the settings for various notification types. Disable Contact is typing and all containing message received.

How to set up keys PageUp, PageDown, Home and End properly in the Mac OS/X Terminal application

This blog post explains how to set up keys PageUp, PageDown, Home and End and their shifted variants in the Mac OS/X Terminal application, so these keys work as in a Linux xterm or gnome-terminal, i.e. PageUp, PageDown, Home, End, Shift-Home, Shift-End send the corresponding escape sequence, and PageUp and PageDown navigate up and down in the scrollback buffer.

Start the Terminal, and copy-paste these commands:

S=$(defaults read com.apple.Terminal "Window Settings" | perl -0777 -pe '$X=q#
            "$F729" = "\033[1;2H";
            "$F72B" = "\033[1;2F";
            "$F72C" = "scrollPageUp:";
            "$F72D" = "scrollPageDown:";
            F729 = "\033[H";
            F72B = "\033[F";
            F72C = "\033[5~";
            F72D = "\033[6~";
#; s@(keyMapBoundKeys *= *\{.*?)\};\n@$1$X};@gs')
test "$S" && defaults write com.apple.Terminal "Window Settings" "$S"

If you don't get any error messages, exit from the Terminal application (closing the individual terminal windows is not enough) and start it again.

Alternatively, you can change the individual key bindings in Terminal / Preferences / Startup / Basic / Keyboard.

The allow non-ASCII characters (such as accented letters) in the bash command line, run

cat >>~/.inputrc <<'END'
set meta-flag On 
set convert-meta Off
set output-meta On  
END
bind -f ~/.inputrc

To make the PageUp and PageDown keys search the shell history forward and backward for a command with a prefix before the cursor, run

cat >>~/.inputrc <<'END'
"\e[5~":  history-search-backward
"\e[6~":  history-search-forward
END
bind -f ~/.inputrc

How to set up static DHCP with OpenWRT

This blog post explains how to set up the DHCP server running within OpenWRT so it always assigns the same IP address to some of the hosts.

In the LuCI web interface, select Administration, then Network / DHCP / Leases. Add your (MAC address, IP address) entries to the Static leases list. Click on Save & Apply. Ask for a new IP address at your computer. (On Debian and Ubuntu, do an ifdown eth0; ifup eth0; on Windows, do an ipconfig /renew /all, or use the Control Panel to bring the network interface down and then up again.)

Should you not get the static IP address you've set up, double check that you have specified the MAC address correctly. If you still don't get it, SSH onto the router, and run the following commands:

root@OpenWrt:~# /etc/init.d/dnsmasq stop
root@OpenWrt:~# true >/var/dhcp.leases 
root@OpenWrt:~# /etc/init.d/dnsmasq start

Once this is done, ask for a new IP address at your computer.

BibTeX cheat sheet and entry templates

See the most up-to-date version of the cheat sheet at http://www.inf.bme.hu/~pts/bibtex-cheat-sheet.txt . A quick copy:

* Based on information form http://en.wikipedia.org/wiki/BibTeX
* See also: http://amath.colorado.edu/documentation/LaTeX/reference/faq/bibstyles.html
* Nonstandard entries: url=, annote=, crossref=,
* No url= field in standard types, should be put to note={URL \url{...}}.
* There are no comments in BibTeX .bib files, not even %
* ?NAME= describes an optional field.
* BibTeX silently ignores ignores a field whose name it doesn't know.

.bib file entry templates
"""""""""""""""""""""""""
@article{NAME,
  author={},
  title={},
  journal={},
  year=,
  ?volume={},
  ?number={},
  ?pages={},
  ?month=,
  ?note={},
  ?key={},
}

@book{NAME,
  author/editor={},
  title={},
  publisher={},
  year=,
  ?volume={},
  ?series={},
  ?address={},
  ?edition={},
  ?month=,
  ?note={},
  ?key={},
  ?pages={},
}

@booklet{NAME,
  title={},
  ?author={},
  ?howpublished={},
  ?address={},
  ?month=,
  ?year=,
  ?note={},
  ?key={},
}

@conference{NAME,
  author={},
  title={},
  booktitle={},
  year=,
  ?editor={},
  ?pages={},
  ?organization={},
  ?publisher={},
  ?address={},
  ?month=,
  ?note={},
  ?key={},
}

@inbook{NAME,
  author/editor={},
  title={},
  chapter/pages={},
  publisher={},
  year=,
  ?volume={},
  ?series={},
  ?address={},
  ?edition={},
  ?month=,
  ?note={},
  ?key={},
}

@incollection{NAME,
  author={},
  title={},
  booktitle={},
  year=,
  ?editor={},
  ?pages={},
  ?organization={},
  ?publisher={},
  ?address={},
  ?month=, 
  ?note={},
  ?key={},
}

@inproceedings{NAME,
  author={},
  title={},
  booktitle={},
  year=,
  ?editor={},
  ?pages={},
  ?organization={},
  ?publisher={},
  ?address={},
  ?month=, 
  ?note={},
  ?key={},
}

@manual{NAME,
  title={},  
  ?author={},
  ?organization={},
  ?address={},
  ?edition={},
  ?month=, 
  ?year=,  
  ?note={},
  ?key={},
}

@mastersthesis{NAME,
  author={},
  title={}, 
  school={},
  year=,
  ?address={},
  ?month=, 
  ?note={},
  ?key={},
}

@misc{NAME,  
  ?author={},
  ?title={},
  ?howpublished={},
  ?month=, 
  ?year=,  
  ?note={},
  ?key={},
}

@phdthesis{NAME,
  author={},
  title={}, 
  school={},
  year=,
  ?address={},
  ?month=, 
  ?note={},
  ?key={},
}

@proceedings{NAME,
  title={},
  year=,
  ?editor={},      
  ?publisher={},   
  ?organization={},
  ?address={},
  ?month=, 
  ?note={},
  ?key={},
}

@techreport{NAME,
  author={},
  title={}, 
  institution={},
  year=={},
  ?type={},
  ?number={},
  ?address={},
  ?month=,
  ?note={},
  ?key={}, 
}
 
@unpublished{NAME,
  author={},
  title={}, 
  note={},  
  ?month=,  
  ?year=,   
  ?key={},  
}
 
Description of entries and fields
"""""""""""""""""""""""""""""""""
* @article: An article from a journal or magazine.
* @book: A book with an explicit publisher.
* @booklet: A work that is printed and bound, but without a named publisher or
  sponsoring institution.
* @conference: The same as inproceedings, included for Scribe compatibility.
* @inbook: A part of a book, usually untitled. May be a chapter (or section or
  whatever) and/or a range of pages.
* @incollection: A part of a book having its own title.
* @inproceedings: An article in a conference proceedings.
* @manual: Technical documentation.
* @mastersthesis: A Master's thesis.
* @misc: For use when nothing else fits.
* @phdthesis: A Ph.D. thesis.
* @proceedings: The proceedings of a conference.
* @techreport: A report published by a school or other institution, usually
  numbered within a series.
* @unpublished: A document having an author and title, but not formally
  published.
* address=: Publisher's address (usually just the city, but can be the full
  address for lesser-known publishers)
* annote=: An annotation for annotated bibliography styles (not typical)
* author=: The name(s) of the author(s) (in the case of more than one author,
  separated by and)
* booktitle=: The title of the book, if only part of it is being cited
* chapter=: The chapter number
* crossref=: The key of the cross-referenced entry
* edition=: The edition of a book, long form (such as "first" or "second")
* editor=: The name(s) of the editor(s)
* eprint=: A specification of an electronic publication, often a preprint or a
  technical report
* howpublished=: How it was published, if the publishing method is nonstandard
* institution=: The institution that was involved in the publishing, but not  
  necessarily the publisher
* journal=: The journal or magazine the work was published in
* key=: A hidden field used for specifying or overriding the alphabetical order
  of entries (when the "author" and "editor" fields are missing). Note that
  this is very different from the key (mentioned just after this list) that is
  used to cite or cross-reference the entry.
* month=: The month of publication (or, if unpublished, the month of creation).
  Example 1=: month=jan. Example 2=: month="17~" # feb.
* note=: Miscellaneous extra information
* number=: The "number" of a journal, magazine, or tech-report, if applicable.
  (Most publications have a "volume", but no "number" field.)
* organization=: The conference sponsor
* pages=: Page numbers, separated either by commas or double-hyphens. For books,
  the total number of pages.
* publisher=: The publisher's name
* school=: The school where the thesis was written
* series=: The series of books the book was published in (e.g. "The Hardy Boys"
  or "Lecture Notes in Computer Science")
* title=: The title of the work
* type=: The type of tech-report, for example, "Research Note"
* url=: The WWW address
* volume=: The volume of a journal or multi-volume book
* year=: The year of publication (or, if unpublished, the year of creation)

How to create a bootable CD running GRUB4DOS on Linux (Ubuntu Hardy)

GRUB4DOS is a flexible, feature-extended version of the GRUB boot manager. This blog post explains how to create a bootable CD which boots GRUB4DOS, from which you can boot almost anything. To create a bootable floppy instead, see http://ptspts.blogspot.com/2009/07/how-to-create-bootable-floppy-running.html.

Run this shell script to create the CD image grldr.iso:

#! /bin/bash
# by pts@fazekas.hu at Sat Jul 11 16:24:31 CEST 2009
GRUB4DOS_ZIP=grub4dos-0.4.4-2009-06-20.zip
GRUB4DOS_DIR=grub4dos-0.4.4  # as extracted from $GRUB4DOS_ZIP
set -ex
TO_INSTALL=''
type -p mkisofs || TO_INSTALL="$TO_INSTALL mkisofs"
type -p wget || TO_INSTALL="$TO_INSTALL wget"
type -p unzip || TO_INSTALL="$TO_INSTALL unzip"
test "$TO_INSTALL" && sudo apt-get install $TO_INSTALL
wget -O "$GRUB4DOS_ZIP" http://download.gna.org/grub4dos/"$GRUB4DOS_ZIP"
unzip -o "$GRUB4DOS_ZIP"
test "$GRUB4DOS_DIR"/grldr
rm -rf grldr.iso.dir
mkdir grldr.iso.dir
cp "$GRUB4DOS_DIR"/{grldr,menu.lst} grldr.iso.dir
mkisofs -R -b grldr -no-emul-boot -boot-load-size 4 -o grldr.iso grldr.iso.dir
: All OK, CD image grldr.iso created.

Please note that you may want to adjust the GRUB4DOS_ZIP variable above in order to download a more recent version of GRUB4DOS, when its available. you can get the list of versions from http://download.gna.org/grub4dos/.

You may use the CD image grldr.iso in your virtualization software (such as VirtualBox), or you may burn it to a CD with:

$ sudo apt-get install growisofs
$ growisofs -dvd-compat -Z /dev/cdrom=grldr.iso

How to create a bootable floppy running GRUB4DOS on Linux (Ubuntu Hardy)

GRUB4DOS is a flexible, feature-extended version of the GRUB boot manager. This blog post explains how to create a bootable floppy which boots GRUB4DOS, from which you can boot almost anything. To create bootable CD instead, see http://ptspts.blogspot.com/2009/07/how-to-create-bootable-cd-running.html.

Run this shell script to create the 1.44MB floppy image grldr.img:

#! /bin/bash
# by pts@fazekas.hu at Sat Jul 11 11:02:14 CEST 2009
GRUB4DOS_ZIP=grub4dos-0.4.4-2009-06-20.zip
GRUB4DOS_DIR=grub4dos-0.4.4  # as extracted from $GRUB4DOS_ZIP
set -ex
TO_INSTALL=''
type -p wget || TO_INSTALL="$TO_INSTALL wget"
type -p mformat || TO_INSTALL="$TO_INSTALL mtools"
type -p unzip || TO_INSTALL="$TO_INSTALL unzip"
test "$TO_INSTALL" && sudo apt-get install $TO_INSTALL
wget -O "$GRUB4DOS_ZIP" http://download.gna.org/grub4dos/"$GRUB4DOS_ZIP"
unzip -o "$GRUB4DOS_ZIP"
dd if=/dev/zero of=grldr.img bs=1474560 count=1
mformat -i grldr.img -f1440 ::
"$GRUB4DOS_DIR"/bootlace.com --floppy --chs grldr.img
mcopy -i grldr.img "$GRUB4DOS_DIR/grldr" ::GRLDR
mcopy -i grldr.img "$GRUB4DOS_DIR/menu.lst" ::menu.lst
: All OK, 1.44MB floppy image grldr.img created.

Please note that you may want to adjust the GRUB4DOS_ZIP variable above in order to download a more recent version of GRUB4DOS, when its available. you can get the list of versions from http://download.gna.org/grub4dos/.

You may use the floppy image grldr.img in your virtualization software (such as VirtualBox), or you may write it to a floppy disk with:

$ cat grldr.img >/dev/fd0

How to configure dynamic DNS (dyndns) with OpenWRT and LuCI

This blog post tells you how to configure dynamic DNS (using dyndns.org) on your router running OpenWRT Kamikaze. The instructions havene be tested on a Linksys WRT54GL running OpenWRT Kamikaze (r14417).

Go to the menu System / Software. If you don't see package descriptions, click on the link Update package lists. Make sure you don't have the obsolete packages updatedd or ipupdate or ez-ipupdate. (You'll still find some old, obsolete forum posts recommending them on the net.) Install package luci-app-ddns.

Reboot (power-cycle) the router, or run rm -rf /var/luci-* in the SSH prompt. Without this step, the menu Services / Dynamic DNS doesn't appear.

Go to the menu Services / Dynamic DNS. Adjust your settings. The package supports dyndns.org, changeip.com, zoneedit.com, no-ip.com, and freedns.afraid.org . Please note that if you specify a force update interval less then 28 hours (the default is 72 hours), then dyndns.org will ban you. Click on the button Save & Apply.

Reboot (power-cycle) the router, or run INTERFACE=wan ACTION=ifup sh /etc/hotplug.d/iface/25-ddns in the SSH prompt. Without this step, the updater daemon script (/bin/sh /usr/lib/ddns/dynamic_dns_updater.sh myddns 0) is not started. This script sends the IP address update to the dynamic DNS provider. Wait a few minutes for the DNS update to take effect, then try to ping your new host name.

How to render SVG with sharp text and manual kerning

If you want to render SVG vector graphics with text (i.e. convert it to a PNG), you might experience some problems: 1. the antialiasing glitch: the edges of the characters (even horizontal and vertical edges) are blurry; 2. the manual kerning glitch: if you specify manual kerning (by pressing Alt-Left or Alt-Right between two characters) in Inkscape, this gets ignored upon rendering. This blog post gives a solution to these problems.

To get rid of the antialiasing glitch, make sure your text is a text object, not a path. To check this, try to edit it with the text tool of Inkscape. If you can edit the text, then it's a text object. (If you have your text as path, you'll get the antialiasing glitch.) Then render the SVG with rsvg or The GIMP. There will be no antialiasing glitch in the output PNG. (If you render by exporting from Inkscape 0.46 or earlier, you'll get the antialiasing glitch. We tested it on Linux and Windows.)

To get rid of the manual kerning glitch, export the SVG to PNG in Inkscape. However, this introduces the antialiasing glitch. To get rid of both, you should use a patched rsvg, i.e. downloading, patching, compiling and installing rsvg for yourself. Get the patch from http://code.google.com/p/pts-mini-gpl/source/browse/#svn/trunk/pts-librsvg-manual-kerning-patch. As of now, the patch is for librsvg-2.26.0, but it is reported to apply cleanly to librsvg-2.22.3 as well. If you use Ubuntu Intrepid on an x86 system, you can get .deb packages from the attachments on https://www.epointsystem.org/trac/vending_machine/wiki/HotSpotLogo.

An example rendered image which avoids both problems is visible on https://www.epointsystem.org/trac/vending_machine/wiki/HotSpotLogo .

How to make Ghostscript embed a Type 1 font to the PDF in one part, without splitting

I was facing a problem with Ghostscript 8.54 -sDEVICE=pdfwrite. When embedding a Type 1 font to the PDF, Ghostscript splitted the font to two font objects (and the corresponding FontDescriptor and CFF stream objects as well), instead of generating only one font object. This blog post explains how I solved the problem.

Please note that the problem didn't happen with a newer Ghostscript (such as GPL Ghostscript 8.61 (2007-11-21), it always generated a single PDF font object.

In a few hours I was able to reduce the source font to two glyphs, Ghostscript still splitting it to two 1-glyph fonts. Then I had the idea to change the /Encoding array, putting the two glyph names next to each other. It solved the problem, Ghostscript didn't split the font anymore. In fact, it turned out that one of the glyphs was missing from the /Encoding, and once I added it (anywhere), the problem was solved.

See the source PostScript file on http://www.math.bme.hu/~pts/twob.ps.txt . It contains the 2-glyph font, demonstrates the problem and shows the solution as well (when compiled with -dfill_encoding=true).

Using \romannumeral in TeX to do multiple macro expansions

This blog post demonstrates a TeX macro hack: using \romannumeral to expand many macros in a single expansion. The example goal is to define a macro \stars which (when called properly) expands to the specified number of stars as a single expansion. Using a single expansion only makes the macro work in an \expandafter\def...{...} context. The similar macro given in The TeXbook doesn't work in this context because it needs multiple expansions.

\documentclass{article} 
%** Usage: \romannumeral\stars{NUMSTARS} 
%** This expands to a NUMSTARS stars (* tokens) in a single expansion step. 
\def\stars#1{% 
  \expandafter\mtostar\expandafter{\expandafter}\romannumeral\number#1 000z} 
\def\firstoftwo#1#2{#1}% 
\def\secondoftwo#1#2{#2}% 
\def\mtostar#1#2{% 
  \ifx#2z\expandafter\firstoftwo\else\expandafter\secondoftwo\fi 
  {0 #1}{\mtostar{#1*}}% 
} 
\begin{document} 
\expandafter\def\expandafter\mystars\expandafter{\romannumeral\stars{4}}% 
\texttt{(\meaning\mystars)} 
\end{document} 

How to typeset text with bilingual fragments with LaTeX

This post gives a possible solution for typesetting two similar versions of the same LaTeX document differing only in some text fragments.

Here is the code:

\documentclass{article}

\makeatletter
\newif\ifsecondlanguage
\def\@bracedfirst#1#2{{#1}}
\def\@bracedsecond#1#2{{#2}}

\def\do@localized#1[#2]{%
  \let\@@localized\@undefined
  \newcommand\@@localized[#2]{#1}\@@localized}%

%** Typeset different text depending on the current value of \ifsecondlanguage.
%** Usage: \bilingual{LANG1PAT}{LANG2PAT}[ARGCOUNT]{ARG1}{ARG2}...
%** If \ifsecondlanguage is true, pattern LANG2PAT is used, else LANG1PAT is
%** used. Arguments #1, #2 etc. in the pattern are substituted with
%** ARG1, ARG2 etc.
\def\bilingual{%
  \expandafter\expandafter\expandafter\do@localized\csname
    @braced\ifsecondlanguage second\else first\fi\endcsname
}

\begin{document}

\secondlanguagetrue
\bilingual
  {This is addition: #1; and this is multiplication: #2.}
  {Translation to another language: #1 and another translation: #2.}
  [2]{$x+y$}{$x*y$}

\secondlanguagefalse
\bilingual
  {These are additions: #1, #2; and this is multiplication: #3.}
  {Translation to another language: #1, #2 and another translation: #3.}
  [3]{$x+y$}{$x\oplus y$}{$x*y$}

\end{document}

Select the language to use with \secondlanguagetrue or \secondlanguagefalse. Whenever you have a text fragment which needs to be typeset differently depending on the languagem, use the \bilingual command, as demonstrated above.

How to migrate or merge a CVS or SVN repository to a remote SVN repository

Let's suppose you have one or more local CVS and/or SVN repositories, and you want to merge them (with their change history) to new directories of an existing, possibly remote target SVN repository. This blog post explains how to do this using Unix tools.

The following tools will be used:

• CVS
• SVN (Subversion) tools including svn, svnadmin and svndumpfilter
• cvs2svn
• svn-pusher

Please note that we won't use svnsync, because it requires the target repository to be empty. We won't use svn-merge-repos.pl much either, because it requires the target repository to be local. We won't use svnadmin load much either, because it requires the target repository to be local.

Access remote repositories for the first time

For some repositories, you have to specify your username (usually in the command line) and password (usually answering to an interactive prompt) in order to be able to connect. For each machine, you have to do it once, because SVN records your username and password to files under $HOME/.subversion/auth. To avoid problems connecting later, make sure you access the the remote repositories on each machine you'll be working on, so your credentials get saved. The easiest way to do it is to run

svn ls URL://TO/SVN/REPOSITORY --username MYUSER

Install svn-pusher

svn-pusher can add any SVN repository to another SVN repository (as a subdirectory) no matter local or remote, keeping the commit history of the specified revision interval. You can skip this installation step now and come back only if you are asked to use svn-pusher in some of the steps below.

svn-pusher is implemented as a Perl script using SVN's Perl bindings (SVN::Core). The easiest way to install svn-pusher is with root access on a Unix system. For example, on Debian or Ubuntu, run this as root:

# apt-get update
# apt-get install libsvn-core-perl
# echo no | cpan -i SVN::Pusher  # this may take about a minute
# type -p svn-pusher
/usr/local/bin/svn-pusher
# svn-pusher help

For the sake of completeness we mention (but we recommend against using) svn-push as an alternative of svn-pusher. svn-push is a tool written in C, available in the SVN contrib directory, and it does something like svn-pusher, but it's dumber: it can commit only a single revision at once, and it needs the head revision number. Here is how to compile it:

$ wget http://svn.collab.net/repos/svn/trunk/contrib/client-side/svn-push/svn-push.c
$ sudo apt-get install libsvn-dev
$ gcc -W -Wall -I/usr/include/subversion-1 -I/usr/include/apr-1.0 \
      -Doff64_t='unsigned long long' -o svn-push svn-push.c -lsvn_client-1
$ ./svn-push
Usage : svn-push -r N:M SRC_URL DEST_URL

For the sake of completeness, we also mention the SVN::Push Perl module, which provides the svnpush command. SVN::Pusher seems to be more up-to-date.

Convert the CVS repository to an SVN repository dump

You can skip this step if your source repository is not a CVS repository.

A CVS repository is a directory containing *,v files (possibly in subdirectories), and containing a directory named CVSROOT (or one of its parents containing the CVSROOT).

Download and install cvs2svn from here. It's a Python script, so install Python as well (2.4 or 2.5 should be OK). You don't need root access to run cvs2svn; in fact, it can be run as extracted from the tarball. Example:

$ wget http://cvs2svn.tigris.org/files/documents/1462/44372/cvs2svn-2.2.0.tar.gz
$ tar xzvf cvs2svn-2.2.0.tar.gz
$ cvs2svn-2.2.0/cvs2svn --help

You don't need Subversion itself for this step – cvs2svn (if run with --dumpfile=) needs only Python and the standard Unix sort utility.

Make sure you have your CVS repository on the same machine as cvs2svn. (Copy with scp -r or rsync if necessary.) It is a good and safe idea to make a copy and to use it for the purpose of the conversion. Make sure you have a neighbor or parent directory named CVSROOT next to the repository. The CVSROOT directory can be empty. Make sure that all files in your CVS repository directory are named *,v (i.e. their name ends with ,v). If you don't need all files or all directories, feel free to remove them now. The effect would be as if those files and/or directories have never been added to the CVS repository.

Run cvs2svn --dumpfile=PROJECT.dump PATH/TO/CVS/REPOSITORY . This creates the file PROJECT.dump, which contains all files in the CVS repository, with their full commit history.

Convert the SVN repository to an SVN repository dump

You can skip this step if your source repository is not a SVN repository.

If your source repository is local and you have read access to it, just run

svnadmin dump PATH/TO/SVN/REPOSITORY >PROJECT.dump

Otherwise use svn-pusher like this:

$ svnadmin create PROJECT.copy
$ svn-pusher push URL://OF/SVN/REPOSITORY PROJECT.copy
$ svnadmin dump PROJECT.copy >PROJECT.dump  # this may take some time
$ rm -rf PROJECT.copy

As an alternative to svn-pusher, you can use svnsync as well (part of the standard SVN installation), see blog post Dump a SVN repository from a URL how to do it. Please note that both svn-pusher and svnsync are quite slow (as compared to svnadmin dump), and svnsync is better supported and documented since it is part of standard SVN.

Once you have your PROJECT.dump file, use svndumpfilter (part of standard SVN) to get rid of the unnecessary files and directories. You may also edit the file in a text editor to do some other modifications (such as renaming files). The file format should be self-explanatory.

If you expect a file name conflict between the repositories (between source1 and source2 or source1 and target), e.g. multiple repositories have trunk/version.h, it is safest to move/rename all the source repositories to their own directories, and once the merge is done, do a careful and safe svn mv. Here is how to rename everything (e.g. from DIR/TO/FILE to PROJECT.merge/DIR/TO/FILE) in a *.dump file:

$ perl -pi -e's@^(Node-path: )@${1}PROJECT.merge/@' PROJECT.dump

After that, please make sure you add the directory creation into revision 1 of PROJECT.dump. Use your text editor to insert the following lines just below the first PROPS-END line:

Node-path: PROJECT.merge
Node-kind: dir
Node-action: add

Merge an SVN repository dump to a local target SVN repository

You can skip this step if the target repository is not local.
The contents of the dump file PROJECT.dump can be added to an existing local target SVN repository using svnadmin load PATH/TO/TARGET/SVN/REPOSITROY <PROJECT.dump . An example for creating a new target SVN repostiory, and adding multiple projects to it:

$ svnadmin create myprojects
$ cvs2svn --dumpfile=myproject1.dump cvsrepo/dir/myproject1
$ svnadmin load myprojects <myproject1.dump
$ cvs2svn --dumpfile=myproject2.dump cvsrepo/dir/myproject2
$ # (edit myproject2.dump, see below)
$ svnadmin load myprojects <myproject2.dump
$ svn ls -R file://$PWD/myprojects

Please note that cvs2svn adds the creation of the directories trunk, tags and branches to the *.dump file. This will be a problem in svnadmin load myprojects <myproject2.dump, because this tries to add directory trunk, which already exists in repository mpyrojects, so the operation will fail. The solution is to edit the file myproject2.dump, and remove the following lines from near the beginning:

Node-path: trunk
Node-kind: dir
Node-action: add


Node-path: branches
Node-kind: dir
Node-action: add


Node-path: tags
Node-kind: dir
Node-action: add

Merge an SVN repository dump to a target SVN repository

This steps works for both local and remote target SVN repositories, but it's a lot slower than the svnadmin load method described above (which works only if the target SVN repository is local). Install svn-pusher (see the step for it above). It doesn't matter which machine you install svn-pusher to as long as it can connect to the target repository. Copy PROJECT.dump created above to the machine you've installed svn-pusher to. Then run

$ svnadmin create PROJECT.copy
$ svnadmin load PROJECT <PROJECT.dump
$ svn-pusher push file://$PWD/PROJECT.copy URL://TO/TARGET/SVN/REPOSITORY  # slow
$ rm -rf PROJECT.copy

Please note that revision numbers in messages reported by svn-pusher are usually off by one, e.g. Committed revision 1 from revision 0. This is normal, the revision numbers will match perfectly in the target repository.

You can use svn-pusher multiple times on the same target repository to merge multiple source repositories. If the target repository is not empty by the time you start svn-pusher, it is safest to dump it first (to have a backup), and then please pay attention to the following facts. svn-pusher (unlike svnadmin load) reports a warning if a directory (e.g. trunk) has already been added. This warning is usally harmless. If svn-pusher wants to add a file which already exists, it will skip merging that source revision, but it will proceed merging subsequent source revisions. This is not always what you want, because you may want to fix the conflict first by renaming files, and only then proceed with subsequent revisions. Should this happen, you may have to rebuild the target repository from scratch using the backup.

Syntax highlighted Python

Example syntax highlighted Python code: (please ignore)

def Hello(self, x, y): 
  return x * y + "foo"  # bar 

The syntax highlighting was done in the borland style on http://pygments.org/demo/.

Ruby on Rails 2.3 unit tests

The blog post Ruby on Rails Unit Tests explains why it is a bad idea to give the unit tests access to the database, and how to set up your Rails project so that you get an exception if you try to access the database with ActiveRecord model objects from a unit test. Unfortunately, that blog post doesn't work with the latest Rails. This blog post explains how to set up a Ruby on Rails 2.3 project to disallow database accees in the unit tests. The instructions were tested with Rails 2.3.2.

Step 1. Create file lib/tasks/unit_tests.rake with the following contents:

Rake::Task[:'test:units'].prerequisites.delete('db:test:prepare')

Step 2. Create file test/unit_test_helper.rb with the following contents:

# similar to autogenerated test/test_helper.rb
#
fail "some of the unit tests has loaded test_helper.rb. Please change " +
     "(require 'test_helper') to (require 'unit_test_helper') in " +
     "tests/unit/**/*.rb" if $".include?('test_helper.rb')
fail "some of the unit tests has loaded test_help.rb. Please make sure that " +
     "the first line is (require 'unit_test_helper') in tests/unit/**/*.rb" if
     $".include?('test_help.rb')
require File.expand_path(File.dirname(__FILE__) + "/../config/environment")
HIDE_ActiveRecord = self.class.send(:remove_const, :ActiveRecord)
require 'test_help'  # rails 2.3.2 standard module
ActiveRecord = self.class.send(:remove_const, :HIDE_ActiveRecord)

class FakeConnection
  class InvalidActionError < StandardError
  end
  COLUMNS = {}
  def self.columns(table_name, name=nil)
    if COLUMNS.has_key?(table_name)
      COLUMNS[table_name]
    else
      raise InvalidActionError, "please create something like this first: " +
          "FakeConnection::COLUMNS[#{table_name.inspect}] = [ " +
          "ActiveRecord::ConnectionAdapters::Column.new(name=..., " +
          "default=nil, sql_type=\"text\", null=false), ...]"
    end
  end
  DB_ERROR_MSG = 'You cannot access the database from a unit test'
  def self.quote_table_name(*args)  # called from ActiveRecord::Base.find
    raise InvalidActionError, DB_ERROR_MSG, caller
  end
  def self.quote_column_name(*args)  # called from ActiveRecord::Base.delete
    raise InvalidActionError, DB_ERROR_MSG, caller
  end
  def self.select_all(*args)  # called from ActiveRecord::Base.find_by_sql
    raise InvalidActionError, DB_ERROR_MSG, caller
  end
  def self.transaction(*args)  # called from ActiveRecord::Base.save
    raise InvalidActionError, DB_ERROR_MSG, caller
  end
end
class << ActiveRecord::Base
  def connection
    FakeConnection
  end
end

Step 3. Make sure all your unit tests (i.e. =*.rb= files in =test/unit=, recursively) start with the line require 'unit_test_helper' instead of require 'test_helper'. Don't forget any of the =*.rb= files, otherwise you'll get an error message at test file load time.

Step 4. If you need access to your model objects, add the necessary column declaration to your test file. For example, for model class Foo:

FakeConnection::COLUMNS['foos'] = [
  ActiveRecord::ConnectionAdapters::Column.new(
      name="bar1", default=nil, sql_type="varchar(255)", null=false),
  ActiveRecord::ConnectionAdapters::Column.new(
      name="bar2", default=nil, sql_type="integer", null=false),
]

After this, you can do a Foo.new in your tests – but you won't be able to save the object, and Foo.find, Foo.find_by_sql and Foo.delete won't work either. If you try any of those, a FakeConnection::InvalidAction gets raised. To test such a functionality, use an integration test instead of a unit test. To do so, create your test file as test/integration/*.rb instead of test/unit/*.rb.

Once all steps are done, run your unit tests with rake test:units. This will run all tests in all *.rb files in the test/units directory (recursively). If you get an exception, and you need the full backtrace, rerun with BACKTRACE=1 rake test:units. For an even longer backtrace, run rake --trace test:units.

Java .class file converter which makes method and fields public available

ClassPublic.Java is a Java .class file converter which makes a Java class public and non-final, its fields public and its methods public and non-final. ClassPublic.java has a compact implementation, depending only on a J2SE 1.5.0 or newer. ClassPublic.java can be used to increase interoperability and code reusability of a .class file whose .java source is not available or it is not feasible to recompile.

Download classpublic-latest.zip.

Java class disassembler with offset display available

jdisasm.py is a Java class file disassembler implemented as a Python script. jdisasm.py displays a java .class file in a human readable form, showing the class name, the field names and types, the method names, types and codes (including instruction mnemonics). For each item shown, the file offset is prepended. (Neither javap or jad can display the file offset.)

Download jdisasm-latest.tar.gz.

How to deploy a Ruby on Rails application using Phusion Passenger and Rails Enterprise Edition on Debian Etch or Ubuntu Hardy

This post gives step-by-step instructions on deploying a Ruby on Rails application on a Linux-based server using Phusion Passenger and Ruby Enterprise Edition. The instructions are for a Debian Etch system, but they should work with minor modifications on any Linux system based on Debian or Ubuntu (including Debian Etch Mini, available as Minimal Gnome Desktop from http://www.visoracle.com/download/debian/.

It doesn't matter if you have Ruby installed on the server, because you'll compile and install Ruby Enterprise Edition (REE) 1.8.6 into its own directory, and run the deployed Rails application using REE. REE is Ruby with some garbage collection improvements to allow memory page reuse across fork()ed subprocesses.

So the Ruby interpreter, Ruby packages and Ruby gems already installed to the system won't be used or needed. But some libraries (such as SQLite client and MySQL client) will be used. You have to install those libraries including the development packages using the system's package manager. Example:

# apt-get install libsqlite3-dev
# apt-get install libpq-dev  # optional, needed if Rails app connects to PostgreSQL
# apt-get install libmysqlclient15-dev  # needed if app connects to MySQL

You also have to install some development packages in order to be able to compile and install REE. Do this:

# apt-get install wget gcc g++ make libc6-dev libreadline5-dev zlib1g-dev libssl-dev

Should other packages be missing, the installer script run below will tell you the apt-get install command to run.

You can get the REE installer tarball from http://www.rubyenterpriseedition.com/. Feel free to substitute any newer ruby-enterprise-*.tar.gz. filename to the command below. To download and install REE (including a bundled Ruby, rubygems, Rake, Rails and Ruby client modules for SQLite 3, MySQL and PostgreSQL), run this:

# cd /usr/src
# wget http://rubyforge.org/frs/download.php/55511/\
  ruby-enterprise-1.8.6-20090421.tar.gz
# tar xzvf ruby-enterprise-1.8.6-20090421.tar.gz
# ruby-enterprise-1.8.6-20090421/installer \
  -c--enable-pthread -a/usr/local/ruby-enterprise-1.8.6
# strip /usr/local/ruby-enterprise-1.8.6/bin/ruby

The installer is an interactive script, and it would have asked questions and waited for you to press Enter occasionally if you hadn't called it with the -a flag. Nevertheless, it show you some nice, colorful messages indicating progress. In a few minutes, it finishes compilation and installation to /usr/local/ruby-enterprise-1.8.6. Please note that the installer runs /usr/local/ruby-enterprise-1.8.6/gem install to download, compile and install some gems (Ruby modules). Please note that -c--enable-pthread is necessary to work around a bug in ruby-enterprise-1.8.6-20090421.tar.gz, which causes fork()ed subprocesses to exit early with SIGVTALRM.

Scroll up and check that the MySQL and PostgreSQL Ruby client modules were installed properly. If not (and you need those), then apt-get install the necessary libraries, and run the installer or gem install again. Example command lines (optional, needed only if you need MySQL or PostgreSQL and the installer failed the compile the relavant Ruby modules):

# apt-get install libmysqlclient15-dev
# /usr/local/ruby-enterprise-1.8.6/bin/gem install --no-ri --no-rdoc mysql

# apt-get install libpq-dev
# /usr/local/ruby-enterprise-1.8.6/bin/gem install --no-ri --no-rdoc postgres

If you have any other gems needed by your Rails application, install them now (similarly to the postgres gem above). Please note that all gems should be installed as root in the setup described in this tutorial, and gems are shared among Rails applications.

Create a user (non-root) and download your Rails application to the user's home (e.g. /home/myrails/myapp). The best way to download the application is the checkout function of a version control system (such as Subversion), because using that you can easily re-deploy the application later. (Please note that Capistrano can be used to fully automate Rails application deployment, but this is beyond the scope of this tutorial.) If you have multiple Rails applications, you can create as many users as you want, and map applications to users freely. The application would run in production as the UID of that user (myrails). As myrails, add these lines to /home/myrails/.bash_profile:

export GEM_HOME=/usr/local/ruby-enterprise-1.8.6/lib/ruby/gems/1.8
export PATH="/usr/local/ruby-enterprise-1.8.6/bin:$PATH"

Log in again, and check that type -p ruby gem rake rails prints a filename inside /usr/local/ruby-enterprise-1.8.6. Make sure that all scripts in /home/myrails/myapp/scrit/* start with /usr/bin/env ruby, and they don't have a specific Ruby interpreter (such as /usr/bin/ruby) hardcoded. Create the initial database of your Rails application (by running rake db:migrate etc.). Quickly try your application by starting

/home/myrails/myapp/script/server --environment=production

and visiting http://localhost:3000/ . Try some functionality which accesses the database. As soon as it works fine, stop the server script.

Now it is time to install Apache2 (please note that Apache 1.3 won't suffice, because Phusion Passenger, the Rails--Apache connector this tutorial uses needs Apache 2). Run this:

# apt-get install apache2 apache2-prefork-dev libapr1-dev
# /usr/local/ruby-enterprise-1.8.6/bin/passenger-install-apache2-module -a

The passenger-install script gives you nice, colorful instructions how to configure your Apache. Memorize those instructions, and feel free to use them in place of the instructions given in this tutorial. Configure your Apache2 as usual. (Setting up and populating a DocumentRoot, setting up SSL (https://) and setting up Apache VirtualHost entries is not covered in this tutorial.) Create file /etc/apache2/mods-available/passenger.conf containing the right paths, for example (type it without the line break):

PassengerRoot /usr/local/ruby-enterprise-1.8.6/lib/ruby/gems/1.8/gems
    /passenger-2.2.2
PassengerRuby /usr/local/ruby-enterprise-1.8.6/bin/ruby

Crete file /etc/apache2/mods-available/passenger.load containing the right paths, for example (type it without the line break):

LoadModule passenger_module /usr/local/ruby-enterprise-1.8.6/lib/ruby/gems/
    1.8/gems/passenger-2.2.2/ext/apache2/mod_passenger.so

Run this:

# ln -s ../mods-available/passenger.load /etc/apache2/mods-enabled/
# ln -s ../mods-available/passenger.conf /etc/apache2/mods-enabled/

Restart apache with

# /etc/init.d/apache2 restart

Wait about 60 seconds, and make sure you don't get any Passenger-related error messages at the end of /var/log/apache2/error.log.

If you want to deploy your Rails application to the root URI of an Apache VirtualHost, all you have to do is specifying

<VirtualHost server.name:80>
  ServerName server.name
  DocumentRoot /home/myrails/myapp/public
</VirtualHost>

in your /etc/apache2/sites-available/default. Do not specify RailsBaseUrl /, that wouldn't work. Restart Apache, and visit http://server.name/ . The first page download may take a few seconds, because Phusion Passenger starts the Rails application at that time. The latency of further downloads should be negligible. If you get a colorful, but unhelpful error message like The page you were looking for doesn't exist.; You may have mistyped the address or the page may have moved., then visit http://localhost/ instead, on which Phusion Passenger doesn't hide the exception raised by Rails. Examining /var/log/apache2/error.log can also help you diagnose the problem.

If you want to deploy your Rails application to a non-root URI (e.g. http://server.name/myapp), then first make sure that the necessary workarounds are applied. Follow the instructions on http://ptspts.blogspot.com/2009/05/how-to-fix-railsbaseuri-sub-uri-with.html of adding files /home/myrails/myapp/config/initializers/!fix_relative_url_root.rb and /home/myrails/myapp/config/initializers/!relative_url_for.rb. Make sure those files are added to your source code repository as well, if applicable. Then create a symlink to your application's public directory. Assuming that your Apache2 DocumentRoot is /var/www, create the symlink by running

# ln -s /home/myrails/myapp/public /var/www/myapp

Make sure you have these lines in your Apache2 configuration (most probably /etc/apache2/sites-available/default):

DocumentRoot /var/www
<Directory /var/www>
  Options +FollowSymlinks
</Directory>
RailsBaseUri /myrails

It's OK to have multiple RailsBaseUri directives, both for multiple or a single application. Restart Apache2 if needed. Visit http://server.name/myrails . If you see an unhelpful error message instead of your application's main page, then visit http://localhost/myrails to get the details. Examining /var/log/apache2/error.log can also help you diagnose the problem.

Please note that if you have mutiple Rails applications (i.e. multiple public directories (after following symlinks)), Phusion Passenger will not
reuse the same Ruby worker process for running multiple applications. Please also note that Ruby worker processes spawned by Phusion Passenger are single-threaded. Phusion Passenger treats two Rails applications the same if they have the same public directory (after following symlinks), so the http:// and https:// versions and aliases of your Rails application may run on the same Ruby worker process.

Rails applications running under Phusion Passenger's control in the production environment don't pick up file changes automatically. So if you change your application's code or config, you have to restart the application: run (as myrails)

# touch /home/myrails/myapp/tmp/restart.txt

and visit the application's URL. The first download should take a few seconds, because Phusion Passenger is restarting your Rails application.

Please refer to Phusion Passenger's documentation for more information about running your Rails application with Phusion Passenger with an Apache2 frontend.

How to fix RailsBaseUri (sub URI) with Phusion Passenger without setting relative_url_root

There is a bug which prevents correct routing with Phusion Passenger 2.2.2 and Rails 2.3.2 (>= 2.2.2) and RailsBaseUri. For example, if you have RailsBaseUri myapp, and you visit http://localhost/myapp/mycontroller/myview, then the request gets routed to controller myapp instead of mycontroller, and you most probably get the error message Routing Error; No route matches "/myapp/mycontroller/myview" with {:method=>:get}. If you visit http://myserver/myapp/mycontroller/myview, you'll get the error message The page you were looking for doesn't exist. You may have mistyped the address or the page may have moved.. This post gives instructions how to work around the bug.

The bug is mentioned in various places, such as
http://blog.ashchan.com/archive/2008/12/10/passengers-railsbaseuri-not-working/ and https://rails.lighthouseapp.com/projects/8994-ruby-on-rails/tickets/1946-setting-a-relative-root-url-via-a- and http://github.com/rails/rails/commit/a87462afcb6c642e59bfcd2e11e3351e2b718c38 .

The manual workaround (according to what is suggested on the pages above) is adding the line below to config/environments/production.rb:

config.action_controller.relative_url_root = '/myapp'

We propose a completely automatic workaround here, which doesn't need hardcoding URIs to the Rails application configuration. To fix this, create a file config/initializers/!fix_relative_url_root.rb with the following contents:

# automatic relative_url_root fix
# for Phusion Passenger 2.2.2 and Rails 2.3.2 (>= 2.2.2)
# by pts@fazekas.hu at Mon May  4 20:48:38 CEST 2009
# from http://ptspts.blogspot.com/2009/05/how-to-fix-railsbaseuri-sub-uri-with.html
fail unless ActionController::Request       # check loaded
module ActionController
  class Request
    def initialize(env)
      @env = env  # Rack::Request#initialize does only this
      path = request_uri.to_s[/\A[^\?]*/]
      sn = @env['SCRIPT_NAME']
      if (RAILS_ENV == 'production' and
          (sn.empty? or sn.starts_with?('/')) and
          path == sn + @env['PATH_INFO'])
        Base.relative_url_root = sn
      end
    end
  end
end

In addition to the fix above, here is another fix for url_for and redirect_to so they automatically prepend ActionController::Base.relative_url_root when they get a string starting with a slash. To apply the fix, create file config/initializers/!relative_url_for.rb with the following contents:

# fix url_for and redirect_to to use ActionController::Base.relative_url_for
# fix for Rails 2.3.2
# by pts@fazekas.hu at Mon May  4 22:38:44 CEST 2009
# from http://ptspts.blogspot.com/2009/05/how-to-fix-railsbaseuri-sub-uri-with.html
fail unless ActionController::Base          # check loaded
fail unless ActionView::Helpers::UrlHelper  # check loaded
module ActionController
  class Base
    alias url_for__ptsroot__ url_for
    def url_for(options = {})
      options = Base.relative_url_root.to_s + options if
          options.kind_of?(String) and options.starts_with?('/')
      url_for__ptsroot__(options)
    end
    alias redirect_to__ptsroot__ redirect_to
    def redirect_to(options = {})
      options = Base.relative_url_root.to_s + options if
          options.kind_of?(String) and options.starts_with?('/')
      redirect_to__ptsroot__(options)
    end
  end
end
module ActionView
  module Helpers
    module UrlHelper
      alias url_for__ptsroot__ url_for
      def url_for(options = {})
        return escape_once(
            ::ActionController::Base.relative_url_root.to_s + options) if
            options.kind_of?(String) and options.starts_with?('/')
        url_for__ptsroot__(options)
      end
    end
  end
end

How to fix a broken registry key if Windows XP is not booting

Recently I modified the Windows XP registry key HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control\Nls\CodePage\OEMCP from 437 to 65001, which resulted an immediate Windows XP system crash, and Windows didn't boot anymore, not even in safe mode. This post describes how I changed the key back without a working Windows on the machine.

• I downloaded SystemRescueCD 1.1.7 from http://www.sysresccd.org/Download, burnt it to CD, booted the system from it.
  
• I pressed Enter at the CD boot menu to boot with the default options.
  
• I waited a few minutes for the SystemRescueCD Gentoo system to boot.
  
• At the root@sysresccd /root % prompt (I pressed Alt-F2 at the help screen), I listed the partitions with the command fdisk -l. It turned out that my Windows XP partition was /dev/sda1.
  
• I created a mount point directory with mkdir /mnt/, and mounted the Windows XP partition with mount -t ntfs-3g /dev/sda1 /mnt/p .
  
• I edited the system registry with chntpw -e /mnt/p/WINDOWS/system32/config/system (see the transcript below).
  
• I ensured that changes are written to disk sync.
  
• I rebooted without the CD, and now my Windows XP worked.

Here is the transcript of the chntpw session:

chntpw version 0.99.5 070923 (decade), (c) Petter N Hagen
Hive  name (from header): 
ROOT KEY at offset: 0x001020 * Subkey indexing type is: 686c <lh>
Page at 0x54c000 is not 'hbin', assuming file contains garbage at end
File size 5767168 [580000] bytes, containing 1301 pages (+ 1 headerpage)
Used for data: 103727/5482832 blocks/bytes, unused: 2263/25616 blocks/bytes.

Simple registry editor. ? for help.

> ls
Node has 7 subkeys and 0 values
  key name
  <ControlSet001>
  <ControlSet002>
  <LastKnownGoodRecovery>
  <MountedDevices>
  <Select>
  <Setup>
  <WPA>

> cd \ControlSet001\Control\Nls\CodePage

\ControlSet001\Control\Nls\CodePage> cat OEMCP

Value <OEMCP> of type REG_SZ, data length 12 [0xc]
65001


\ControlSet001\Control\Nls\CodePage> ed OEMCP
EDIT: <OEMCP> of type REG_SZ with length 12 [0xc]
[ 0]: 65001

Now enter new strings, one by one.
Enter nothing to keep old.
[ 0]: 65001
-> 437
newkv->len: 8

\ControlSet001\Control\Nls\CodePage> q

Hives that have changed:
 #  Name
 0  <system>
Write hive files? (y/n) [n] : y
 0  <system> - OK

Please note that a similar procedure (with the exact same fdisk, mount, and chntpw comamnds) using the Knoppix 5.3.1 live CD instead of SystemRescueCD.

Please note that it is possible to edit any registry file with the Windows XP regedit. Here are some relevant links how to do this: http://smallvoid.com/article/winnt-offline-registry-edit.html; http://www.hardforum.com/showthread.php?t=1162302; http://www.911cd.net/forums//index.php?showtopic=7066.

Please note that it is possible to edit the registry with the regedit application shipping with the ERD Commander 2005 Windows XP live boot CD. Get the CD from http://www.fullandfree.info/software/erd-commander-2005/ . There is a Linux tool nrg2iso which can convert the .nrg file in the download for burning to CD. On Windows, it is possible to burn the .nrg file using ImgBurn.

Here are some methods which I tried, but they didn't work to edit the registry system registry file:

• Using kregedit – it crashed opening the system registry file.
  
• Running wine's /usr/bin/regedit – I wasn't able to specify which file to edit.
  
• Running wine regedit.exe with the Windows XP regedit.exe – it failed to start up because it hasn't found some DLLs. I haven't bothered forcing it.

How to fix Ubuntu Hardy and Intrepid boot hang on a Philips laptop

If you have a Philips laptop and you want to boot Ubuntu Hardy and Intrepid on it, the system might hang indefinitely while booting (not reacting to any keypress, not even the Num Lock key). The reason for the hang may be the non-working driver for the notebook's built-in memory card reader (MMC). This post describes the solution: how to disable the driver for the memory card reader and thus get Ubuntu boot. (But you won't be able to use the memory card in Ubuntu.)

If you have another Linux boot CD at hand (such as Knoppix) which boots, and you are familiar with it, then boot it, open file /etc/modprobe.d/local on your Linux root partition, and add the line blacklist mmc_core . Reboot the machine to Ubuntu. Now it should boot normally, without hanging. By doing this you ensure that that the memory card reader driver kernel module won't ever be loaded in the future at boot time. This is a long-term solution, it survives kernel and distribution upgrades.

If you don't have an alternative boot CD at hand, then do the fixing like this. Reboot the system (power-cycle it if necessary). In the GRUB boot menu (a list of long lines starting with Ubuntu 8, press key E, in the new menu, select the line starting with kernel, press key E, press key Space, type init=/bin/bash rw , press key Enter, press key B. This will start the system boot process, giving you a shell soon. In a minute, you'll get a prompt ending with #, and the cursor blinking after that. Type bash -c 'echo blacklist mmc_core >>/etc/modprobe.d/local', press Enter. Type sync , press Enter. Wait a few seconds. Reboot the machine by pressing Ctrl-Alt-Del (or by any other means). Now Ubuntu should boot normally, without hanging.

If the instructions above don't work for you, here is a brute force, temporary fix, which won't survive some system upgrades. Do exactly as above, up to the # prompt. Then type rm -rf /lib/mmc , press Enter. Type mv -fT /lib/modules/`uname -r`/kernel/drivers/mmc /lib/mmc , press Enter. Continue with sync etc.

How to set up an external monitor for watching movies with the nvidia proprietary x.org driver

This post describes how to set up your /etc/X11/xorg.conf on Linux if you have a laptop with an NVIDIA video card, and you want to attach an external monitor for watching movies with MPlayer. Please note that the external monitor is not for regular work, so normally you don't want to put windows there, or use the mouse or keyboard to interact with windows on the external monitor.

How to configure the X server

If you enable TwinView in /etc/X11/xorg.conf (possibly using the command nvidia-xconfig, you'll get one big X11 screen (DISPLAY=:0), with windows possibly spanning two two displays. This is not what you want now. What you want is to have two, independent X11 screens, the laptop's built-in LCD display (DISPLAY=:0.0) running your regular X11 session, and the external monitor (DISPLAY=:0.1) showing the movies played. (Sending audio to the external monitor's speakers is not described here.)

To configure the X.Org X11 server for this, you have to modify /etc/X11/xorg.conf the following way. You have to add a new "Monitor" section, a new "Device" section and a new "Screen" section. First make a backup of your config file: sudo cp /etc/X11/xorg.conf{,.single.good}. Then modify the config file like this:

# modifications to /etc/X11/xorg.conf

Section "Device"
  # This is the original Section "Device"
  Identifier      "..."
  Driver          "nvidia"
  Busid           ...
  Option          ...
  ...
  # ADD Screen 0.
  Screen          0
EndSection

Section "Device"
  Identifier      "nvidia1"  
  Driver          "nvidia"   
  # COPY Busid from original
  Busid           ...
  # COPY Option(s) from original
  Option          ...
  # ADD Screen 1.
  Screen          1
EndSection

# ADD this Monitor section.
# The specified HorizSync and VertRefresh ranges are good for most
# external LCD monitors. You may want to widen them for your monitor.
Section "Monitor"
  Identifier      "monitor1"
  Option          "DPMS"
  HorizSync       28-64 
  VertRefresh     43-60 
EndSection

# ADD this Screen section.
Section "Screen"
  Identifier      "screen1" 
  Device          "nvidia1" 
  Monitor         "monitor1"
  # If it doesn't work with 24, try changing to 32 (and below as well).
  DefaultDepth    24
  SubSection "Display"
    Depth           24
    # SET this to the preferred (maximum) resolution of your external monitor.
    Modes "1920x1080"
  EndSubSection
EndSection

# ADD or modify this ServerLayout section
Section "ServerLayout"
  Identifier  ...
  # Multiple InputDevice entries are OK
  InputDevice ...
  ...
  # SET ... to the name of your original Section "Screen".
  Screen 0 "..." 0 0
  # MAKE sure that the number you specify is larger than
  # the maximum width of your screens. Otherwise the mouse
  # pointer may accidentally wrap one the edge of one screen
  # to another.
  Screen 1 "screen1" 1300 0
EndSection

Make sure that the position you specify for Screen 1 in ServerLayout is large enough, i.e. it is larger than the maximum width of your screens. http://users.tkk.fi/spniskan/switchscreen/ gives the same instructions: Define in the ServerLayout section the second screen's position to be farther away than the first screen's width.. Doing this makes sure that the mouse pointer won't accidentaly wrap from one screen to another when you move it out at the edge of any of the screens.

After modifying /etc/X11/xorg.conf, make a backup copy of the new version (sudo cp /etc/X11/xorg.conf{,.twoscreens.try}), close all applications, connect the external monitor, then restart your X11 session by pressing Ctrl-Alt-<BackSpace>. If it doesn't start up again, or you get some display configuration dialog, then there was something wrong with your modifications to /etc/X11/xorg.conf, and you have to fix it. To do so, press Ctrl-Alt-<F1> to switch to text mode, log in, then stop GDM by running sudo /etc/init.d/gdm stop, then have a look at the startup log file /var/log/Xorg.0.log , and edit /etc/X11/xorg.conf accordingly (you may have to copy it back from /etc/X11/xorg.conf.twoscreens.try first). To try your changes, run X (or sudo X), which displays a black-and-white dotted background on both screens, and an X-shaped mouse cursor on screen 0 (the laptop LCD). Exit by pressing Ctrl-Alt-<BackSpace>. If that one works, you may want to try startx to get a more interesting X session. Exit again by pressing Ctrl-Alt-<BackSpace>. Make a backup copy of the config file (sudo cp /etc/X11/xorg.conf{,.twoscreens.good}, and restart GDM: sudo /etc/X11/gdm restart.

How to prevent the GNOME panel and the icons from appearing on the second screen

Sorry, I don't know the right answer. The panels are drawn by gnome-panel. You can remove all but one, and GNOME will remember it upon relogin. If you remove all panels from a screen, GNOME will recreate the panels with the default configuration :-(. The desktop icons are drawn by nautilus. I have no idea how to disable it on the second screen. The window manager is metacity or compiz (use ps x to find out which is running). You can disable compiz on the second screen by appending the line COMPIZ_OPTIONS="$COMPIZ_OPTIONS --only-current-screen" to ~/.config/compiz/compiz-manager .

How to change the background image

Install ImageMagick with sudo apt-get install imagemagick. Then run display -window root -display :0.1 background.jpg. Please note that this will tile (repeat) copies of the image. You may want to resize the image to fit the screen size: display -window root -display :0.1 -resize 1920x1080 background.jpg, but this may ruin the aspect ratio. Use your favorite image editor (such as GIMP) to create an image of the right size. If you use GNOME, this trick might be useful: http://gnome-hacks.org/hacks.html?id=6 (but it sets the background for multiple screens).

How to move the mouse and switch the keyboard focus to the other X11 screen

There is a handy tool for that named switchscreen. Apparently there are two programs named switchscreen, one by Sampo Niskanen (http://users.tkk.fi/spniskan/switchscreen/) and one by David Antliff (http://en.gentoo-wiki.com/wiki/X.Org/Dual_Monitors#Moving_focus_between_screens and http://unlogical.net/files/scripts/switchscreen-0.4.tar.gz; alternate download for switchscreen.c: http://www.math.bme.hu/~pts/switchscreen.c). We are going the use the latter. Compile it with gcc -s -O2 -W -Wall -L/usr/X11R6/lib -lX11 -lXtst -lXext -o switchscreen switchscreen.c. (On Debian Etch and Ubuntu Hardy, you'll have to install some packages first: sudo apt-get install gcc libc6-dev x11-proto-core-dev x11proto-xext-dev x11proto-xext-dev .) Install with sudo cp switchscreen /usr/local/bin/ .

If you use GNOME, run switchscreen by pressing Ctrl-<F2>, and typing switchscreen plus Enter. You can use your desktop environment's configuration or xbindkeys to bind a key combination to switchscreen.

How to play a movie with mplayer on the external monitor

Run mplayer -display :0.1 -osdlevel 3 -fs movie.avi . If you don't see the the number of seconds elapsed, you may have to configure the MPlayer OSD font. If the OSD text is two large, append lines subfont-text-scale = 3 and subfont-osd-scale = 3 to the file ~/.mplayer/config . If the aspect ratio is wrong on the external monitor, but it is good on the LCD display, specify the flag -monitorpixelaspect X in the mplayer command line above, experimenting with values between 0.5 and 2.0 for X.

How to prevent stderr log messages from interleaving on Linux

Let's suppose you have a program which forks subprocesses, and each subprocess writes log messages to the stderr they share. Since processes run concurrently, it may be possible that the bytes they write interleave (overlap). For example, process A calls write(2, "ABC\n", 4) == 4, and, at the same time process D calls write(2, "DEF\n", 4) == 4. You expect that stderr will contain "ABC\nDEF\n" or "DEF\nABC\n", depending on which write succeeds first. But this is not always the case (!): messages written can interleave, e.g. stderr might contain "ABDCEF\n\n" or something similar. This post gives some recepies how to prevent that, and thus always get either "ABC\nDEF\n" or "DEF\nABC\n".

The instructions given below were tested on Linux 2.6.22 and 2.6.24, but they may probably apply other Linux kernels and other Unices as well.

• Use the write(2) system call to write your log messages. Avoid anything which can buffer or split the message.


• In C, don't use printf(3), fwrite(3), fputs(3) etc.; use write(2)


• In Java 6, don't use System.err.println; use System.err.print, and add a "\n" by hand.


• In Python, don't use sys.stderr.write or print >>sys.stderr; use os.write(2, ...)
.

• In Ruby, don't use $stderr.write or $stderr <<; use $stderr.syswrite. Don't forget to check the return value.


• In Perl, don't use print(STDERR ...); use syswrite(STDERR, ...).


• If stderr is a pipe, make sure you don't write more than 4096 bytes at a time. If you have a longer log message, split it to multiple log messages of at most 4096 bytes (including the newline). This size limit doesn't apply if stderr is a regular file. This size limit has been verified for the kernels above.


• If stderr is a regular file, make sure that it is in append mode when you are writing it.


• Without append mode, even data loss is possible, e.g. instead of "ABC\nDEF\n" you may get "ADBEF\n" (losing "C\n").


• You can ensure append mode by using the append-redirection 2>>file.log instead of 2>file.log when invoking the program.


• In C you can use fcntl(2) to put a file descriptor to append mode: fcntl(2, F_SETFL, fcntl(2, F_GETFL, 0) | O_APPEND). (Don't forget about checking the return values to detect errors.)


• In Python you can use fcntl.fcntl to put a file descriptor to append mode: import fcntl; import os; fcntl.fcntl(2, fcntl.F_SETFL, fcntl.fcntl(2, fcntl.F_GETFL) | os.O_APPEND).


• In Ruby you can use IO#fcntl to put a file descriptor to append mode: require 'fcntl'; $stderr.fcntl(Fcntl::F_SETFL, $stderr.fcntl(Fcntl::F_GETFL) | Fcntl::O_APPEND).


• In Perl you can use Fcntl to put a file descriptor to append mode: use Fcntl; fcntl(STDERR, Fcntl::F_SETFL, fcntl(STDERR, Fcntl::F_GETFL, 0) | Fcntl::O_APPEND).

As a more generic solution (not specific to Unix), you may also consider using mutexes or semaphores to prevent concurrent writes to the same file.

How to convert some LaTeX text to a high resolution PNG image

Create the TeX source file dump.tex like this:

\documentclass{article}
\pdfpagewidth2cm 
\pdfpageheight1cm
\hoffset-2.3cm
\voffset-2.3cm
\begin{document}
\shipout\hbox{$\infty \sum \int$}
\end{document}

Convert it to PDF:

% pdflatex dump.tex

Make sure that pk files don't appear on the console output of pdflatex. If they happen to appear, choose a different font, e.g. \usepackage{lmodern}.

Convert the PDF to PNG with Ghostscript:

% gs -dBATCH -dNOPAUSE -sDEVICE=pngmono -r1000 -sOutputFile=dump.png dump.pdf

If you need a larger resolution image, increase the number after the -r flag above.

This solution generates a PNG with sharp edges (no smoothing). If you take a screenshot of the PDF in a PDF viewer, or you load the PDF directly into GIMP, you'll most probably get a smoothed (antialiased) image.

How to set up ruby gems on Debian Etch as non-root

This tutorial describes how to install Ruby gems to your home directory. The gems you install there will not be visible to other user, and they won't affect the system default configuration. You'll need root access in the first few steps to install Ruby and the rubygems packaging framework itself. It is possible to install those without root access as well, but that's beyond the scope of this tutorial.

You need a working ruby and rubygems. You have to install it as root:

$ su -
# apt-get update
# apt-get install ruby1.8 rubygems
(if you want to install gems which need C compilation, e.g. ruby-sqlite:)
# apt-get install ruby1.8-dev gcc libc6-dev
# ruby -v
ruby 1.8.5 (2006-08-25) [i486-linux]

This is optional, to have the latest rubygems (you can do this any time later):

$ su -
# gem install rubygems-update
...
# /var/lib/gems/1.8/bin/update_rubygems
# rm -f /usr/bin/gem
# ln -s gem1.8 /usr/bin/gem
$ gem -v
1.3.1

All the rest works as non-root:

$ export GEM_HOME=$HOME/gems
$ rm -rf $GEM_HOME
$ mkdir $GEM_HOME{,/cache,/doc,/gems,/specifications}
$ cp -a /var/lib/gems/1.8/cache/sources-*.gem $GEM_HOME/cache/
$ cp -a /var/lib/gems/1.8/gems/sources-* $GEM_HOME/gems/
$ cp -a /var/lib/gems/1.8/specifications/sources-*.gemspec $GEM_HOME/specifications/
$ gem update
Updating installed gems...
Bulk updating Gem source index for: http://gems.rubyforge.org
(this takes a few minutes)
Gems: [] updated

Try querying or installing something:

$ gem search rake --remote
$ gem install rake
Successfully installed rake-0.8.4
Installing ri documentation for rake-0.8.4...
Installing RDoc documentation for rake-0.8.4...

Try installing rails:

$ gem install rails --include-dependencies
(... takes some time)
$ ~/gems/bin/rails -v
2.3.2

Try installing something which needs C code. Please note that you have to install the C prerequisite (libsqlite3) as root:

$ su -c 'apt-get install libsqlite3-dev'
$ gem search sqlite3 --remote
$ gem install --platform ruby sqlite3-ruby
Building native extensions.  This could take a while...
Successfully installed sqlite3-ruby-1.2.4
1 gem installed
Installing ri documentation for sqlite3-ruby-1.2.4...
Installing RDoc documentation for sqlite3-ruby-1.2.4...

Please note that you need the GEM_HOME environment variable set for installing gems and for running applications requiring gems. To have this variable set for you, do this:

$ echo 'export GEM_HOME=$HOME/gems' >>~/.bashrc
$ echo 'export GEM_HOME=$HOME/gems' >>~/.bash_profile

If you need the rails command without having to specify ~/gems/bin/rails, make sure you have an export PATH=$HOME/gems/bin:$PATH in your .bashrc and/or .bash_profile.

How to make Flash full screen work in Ubuntu Hardy and Firefox

We were using the Flash applet on Livescribe's web site in Firefox running on Ubuntu Hardy. We were using Flash 9.0 r124, and Firefox 3.0.7. It had a full screen button, but nothing happened. It was strange, because the full screen button worked a week ago. I've found a solution on http://ubuntulinuxtipstricks.blogspot.com/2007/12/fullscreen-youtube-now-available.html :

If you're using Compiz Fusion, turn off "Unredirect Fullscreen Windows" in the General section and turn off "Legacy Fullscreen Support" in the Workarounds plugin. If you don't, the controls for the player get cut off.

We've found the settings above in our Gnome desktop > System > Preferences > Advanced desktop effects > General options. Turning both settings off solved the problem for us.

ASCII isdigit, isalpha and isxdigit macros in ANSI C with arithmetic operations

The naïve way of defining a C macro which tests whether a character is a digit (assuming an ASCII-based character set) is #define ISDIGIT(c) ((c) >= '0' && (c) <= '9'). There is a fundamental problem with this naïve definition: it evaluates its argument c more than once, so for example ISDIGIT(x++) will increment x by 2 in some cases. The question naturally arises if there is a macro definition #define ISDIGIT(c) ..., which uses c exactly once. Indeed, there is:
#define ISDIGIT(c) ((c) - '0' + 0U <= 9U). The capital Us in the expression enforce unsigned calculation, so if c is less than '0', then (c) - '0' + 0U becomes a large positive number instead of a negative number with small absolute value, so the comparison will (correctly) return false.

The following code contains solutions for ISDIGIT, ISALPHA and ISXDIGIT, the latter not being practical because of the excessive use of arithmetic operations.

/* by pts@fazekas.hu at Thu Mar 26 01:10:56 CET 2009
*
* 0123456789 ISDIGIT
* ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz ISALPHA
* 0123456789ABCDEF ISCAPITALHEX
* 0123456789ABCDEFabcdef ISXDIGIT
*/
#include <stdio.h>

#define ISDIGIT(c) ((c) - '0' + 0U <= 9U)
#define ISALPHA(c) (((c) | 32) - 'a' + 0U <= 'z' - 'a' + 0U)
#define ISCAPITALHEX(c) ((((((c) - 48U) & 255) * 23 / 22 + 4) / 7 ^ 1) <= 2U)
#define ISXDIGIT(c) (((((((((c) - 48U) & 255) * 18 / 17 * 52 / 51 * 58 / 114 \
     * 13 / 11 * 14 / 13 * 35 + 35) / 36 * 35 / 33 * 34 / 33 * 35 / 170 ^ 4) \
     - 3) & 255) ^ 1) <= 2U)

int main(int argc, char **argv) {
  int i;
  (void)argc; (void)argv;
  for (i = 0; i < 256; ++i) if (ISDIGIT(i)) putchar(i);
  printf(" ISDIGIT\n");
  for (i = 0; i < 256; ++i) if (ISALPHA(i)) putchar(i);
  printf(" ISALPHA\n");
  for (i = 0; i < 256; ++i) if (ISCAPITALHEX(i)) putchar(i);
  printf(" ISCAPITALHEX\n");
  for (i = 0; i < 256; ++i) if (ISXDIGIT(i)) putchar(i);
  printf(" ISXDIGIT\n");
  return 0;
}

How to select an ALSA sound card and have concurrent, simultaneus playback using dmix

This blog post is tutorial which describes how to select a default sound card and run multiple playbacks simultaneously, using ALSA on Linux, without a sound server.

The quick trick is having

defaults.pcm.!card Headset
defaults.ctl.!card Headset
defaults.pcm.!device 0
defaults.ctl.!device 0

in your ~/.asoundrc. (Replace Headset with the name of the card on which you want to hear sound. Get the list of available sound cards with aplay -l | awk '/^card/{print$3}'|sort|uniq. To apply this setting for all users, add the lines above to /etc/asound.conf instead.) If this doesn't work for you, please continue reading.

Let's suppose you have a Linux system and many sound cards with an ALSA (>= 0.9) driver, and you don't use any sound server (e.g. pulse, ESD == esound or artsd). The dmix feature of ALSA does software sound mixing, so it makes it possible to run multiple playbacks simultaneously on the same card. ALSA, by default, enables dmix for all cards which don't support concurrent playback of more than one sound stream.

So you just run mplayer -ao alsa file1.mp3, and simultaneously (possibly in another terminal window) mplayer -ao alsa file2.mp3. You should hear both playbacks at the same time. (Please note that you can omit =-ao alsa= from the mplayer command line if you add ao=alsa to your ~/.mplayer/config file.) If the second mplayer doesn't start playback, but exists with an error message containing Device or resource busy, this means there is something wrong with your settings – this tutorial will help to fix that.

If even the first mplayer produces the Device or resource busy error, then close or kill all applications that might play sound. This includes the web browser (with flash), pulse, esd, artsd, mplayer, Skype, MPD and system sounds (disable everything in Gnome / System / Preferences / Sound / Sound). After that, mplayer -ao alsa file1.mp3 should start the first playback, and simultaneously, mplayer -ao alsa file2.mp3 should start the second playback.

If you have multiple sound cards (possibly the sound card in your computer, and an external USB headset), you can set the environment variable ALSA_CARD to direct playback to a specific card. Example 1: ALSA_CARD=Headset mplayer -ao alsa file1.mp3. Example 2: start Firefox as ALSA_CARD=Headset firefox to have the sound Flash movies played on the card named Headset.

You can get a list of sound cards (with some extra information) with aplay -l. For example, on my system, I get

$ aplay -l
card 0: Intel [HDA Intel], device 0: CONEXANT Analog [CONEXANT Analog]
  Subdevices: 1/1
  Subdevice #0: subdevice #0
card 0: Intel [HDA Intel], device 1: Conexant Digital [Conexant Digital]
  Subdevices: 1/1
  Subdevice #0: subdevice #0
card 1: TuxDroid [TuxDroid], device 0: USB Audio [USB Audio]
  Subdevices: 1/1
  Subdevice #0: subdevice #0
card 1: TuxDroid [TuxDroid], device 1: USB Audio [USB Audio #1]
  Subdevices: 1/1
  Subdevice #0: subdevice #0
card 2: Headset [Plantronics Headset], device 0: USB Audio [USB Audio]
  Subdevices: 0/1
  Subdevice #0: subdevice #0

Here is how I can select each card and device:

card 0: Intel [HDA Intel], device 0: CONEXANT Analog [CONEXANT Analog]
  ALSA_CARD=0
  ALSA_CARD=Intel
card 0: Intel [HDA Intel], device 1: Conexant Digital [Conexant Digital]
  ALSA_CARD=0 ALSA_PCM_CARD=1
  ALSA_CARD=Intel ALSA_PCM_CARD=1
card 1: TuxDroid [TuxDroid], device 0: USB Audio [USB Audio]
  ALSA_CARD=1
  ALSA_CARD=TuxDroid
card 1: TuxDroid [TuxDroid], device 1: USB Audio [USB Audio #1]
  ALSA_CARD=1 ALSA_PCM_CARD=1
  ALSA_CARD=TuxDroid ALSA_PCM_CARD=1
card 2: Headset [Plantronics Headset], device 0: USB Audio [USB Audio]
  ALSA_CARD=2
  ALSA_CARD=Headset

Once you have the proper ALSA_CARD and ALSA_PCM_CARD setting, you can add them to your ~/.bashrc and ~/.gnomerc and ~/.xprofile (and possibly to /etc/environment and /etc/X11/Xsession.d/* and /etc/gdm/Xsession). If you log out and log in to your graphic session, you'll have these environment variables by default.

It is possible to select the default sound card without changing setting the ALSA_CARD or ALSA_PCM_CARD environment variables. To do so, add these lines to your ~/.asoundrc:

defaults.pcm.!card Headset
defaults.ctl.!card Headset
defaults.pcm.!device 0
defaults.ctl.!device 0

Each !card line corresponds to the ALSA_CARD value, and each !device line corresponds to the ALSA_PCM_CARD value.

If you set the default in ~/.asoundrc and set the environment variables as well, then the environment variables take effect.

Please note that if you specify any specific ALSA device to any program (other than default), and the program starts playback, then automatic dmix would not work until that program finishes playback and closes the device. This implies that no other program will be able to start playback (but will yield Device or resource busy) until that happens. For example, Skype keeps the sound card open while it is running. So if you want to play sound not coming from Skype while Skype is running, you have to select Default device (default) in Options / Sound Devices for both Sound Out and Ringing. A similar restriction applies to music players and other software: if you specify the playback device for them in their command line or preferences, then the software will lock the sound card, and you lose dmix and concurrent playback. The only dmix-safe ways to select a sound card are the ALSA_CARD etc. environment variables and ~/asoundrc.

To get information about the ALSA cards, run aplay -l and aplay -v -v -L.

ALSA provides OSS emulation (i.e. /dev/dsp, /dev/dsp1), but dmix doesn't work with OSS emulation. To get it work, please run the software which needs OSS using the aoss wrapper, e.g. aoss mplayer -ao oss file1.oss, and concurrently, aoss mplayer -ao oss file2.oss or mplayer -ao alsa file2.oss . If you get the error message /dev/dsp: Device or resource busy from a program, then you'll either have to change it to use ALSA, or run it within aoss.

Here are some sine wave sound generators if you don't have an MP3 ready to test playback with:

perl -e 'print pack("v",32000*sin($_/34))."\0\0"; ++$_ while 1' | aplay -f dat  # Left ear
perl -e 'print "\0\0".pack("v",32000*sin($_/34)); ++$_ while 1' | aplay -f dat  # Right ear

If you have the asoundconf utility, you can use it to set up the default sound card in your ~/.asoundrc. For example, after removing ~/.asoundrc and running
asoundconf set-default-card Headset, you'll get a line <:/home/USERNAME/.asoundrc.asoundconf> in file ~/.asoundrc, and the file ~/.asoundrc.asoundconf would contain more than 50 config lines, the essential ones being

!defaults.pcm.card Headset
defaults.ctl.card Headset 
defaults.pcm.device 0
defaults.pcm.subdevice -1
defaults.pcm.nonblock 1  
defaults.pcm.ipc_key 5678293
defaults.pcm.ipc_gid audio  
defaults.pcm.ipc_perm 0660
defaults.pcm.dmix.max_periods 0
defaults.pcm.dmix.rate 48000   
defaults.pcm.dmix.format S16_LE
defaults.pcm.dmix.card defaults.pcm.card
defaults.pcm.dmix.device defaults.pcm.device
defaults.pcm.dsnoop.card defaults.pcm.card  
defaults.pcm.dsnoop.device defaults.pcm.device
defaults.namehint.extended off

This seems to be too much compared to the 4 lines the beginning of this tutorial suggests.

Random notes

With ALSA 1.0.15, only plughw: works for tuxdroid (so it cannot be used with ALSA_CARD, which implies hw:). Here is how to play: mplayer -ao alsa:device=plughw=TuxDroid file1.mp3 or aplay -D plughw:TuxDroid </dev/urandom. Please note that this restriction applies to both mplayer and aplay. (Maybe that's because dmix was busy when the tuxdroid was connected -- and if we reload ALSA, maybe it will work if I connect the tuxdroid first, and then load the ALSA modules?) The reason why it doesn't work seems to be that the U8 sample format needed by the tuxdroid was introduced only in ALSA 1.0.16. I've verified with libasound 1.0.16 installed (with the same old ALSA kernel) in a chroot, and dmix works with the tuxdroid.

The setting ALSA_CARD=Foo ALSA_PCM_CARD=2 corresponds to aplay -D hw:Foo,2 and mplayer -ao alsa:device=hw=Foo.2. Please note that there is no corresponding environment variable setting for plughw instead of hw. Please also note that mplayer won't use dmix (thus it won't be able to run multiple playbacks concurrently) if you specify any other ALSA setting than mplayer -ao alsa or mplayer -ao alsa:device=default . A similar restriction applies to aplay -D: if you specify any device other than default there, it won't use dmix.

An example full mplayer error message when dmix didn't work:

[AO_ALSA] alsa-lib: pcm_hw.c:1099:(snd_pcm_hw_open) open /dev/snd/pcmC2D0p failed: Device or resource busy
[AO_ALSA] Playback open error: Device or resource busy
Could not open/initialize audio device -> no sound.
Audio: no sound

Using pytz to display a timestamp in a time zone

Download pytz from http://pytz.sourceforge.net/ . You can try out the code snippet below without installing pytz. Just create your Python script in the directory where pytz's setup.py lives.

1. import time  
2. import pytz  
3. import datetime  
4. dt = datetime.datetime.fromtimestamp(time.time(), pytz.utc)  
5. #tz = pytz.timezone('EST')  
6. tz = pytz.timezone('CET')  
7. print tz.normalize(dt.astimezone(tz)).strftime('%Y-%m-%d %H:%M:%S %Z(%z)')