pts.blog

Guess the programming language with array and dictionary addition

2012-01-24T14:11:00.000+01:00

Guess the programming language.

In which programming language are all of these true?
- [] + [] is an empty string.
- [] + {} is an empty object.
- {} + [] is 0.
- {} + {} is not-a-number.
In which programming language are all of these true?
- [] + [] is an empty list.
- [] + {} raises TypeError.
- {} + [] raises TypeError
- {} + {} raises TypeError.
In which programming language are all of these true?
- [] + [] is an empty list.
- [] + {} raises TypeError.
- {} + [] raises NoMethodError
- {} + {} raises NoMethodError.
In which programming language are all of these true?
- [] + [] is a positive number, e.g. 19806432.
- [] + {} is a positive number, e.g. 20142304.
- {} + [] is a positive number, e.g. 58300640.
- {} + {} is a positive number, e.g. 27597024.

How to put the window close button to the right on Ubuntu Lucid

2012-01-16T22:20:00.003+01:00

This blog post explains how to put the the window close button to the top right corner of the window on Ubuntu Lucid, instead of the top left corner, where new Ubuntu versions tend to put it by default.

To put the window close button to the top right corner, and to arrange the other buttuns in the classic (2008 or earlier) layout, run the following command in a terminal window (without the leading $ sign):

$ gconftool-2 --type=string --set /apps/metacity/general/button_layout \
  menu:minimize,maximize,close

C++ operator definitions inside and outside the class are not equivalent

2012-01-08T15:37:00.000+01:00

This blog post presents some example code which demonstrates that having an operator outside a class is not equivalent to having it inside the class. As shown below, some implicit casts are applied only if the operator definition is outside the class.

It compiles when the operator definition is outside:

class C {
 public:
  C(unsigned a) {}
  C operator^(const C& masik) {
    return *this;
  }
 private:
  C();
};

int main() {
  C a = 5;
  unsigned b = 6;
  a ^ b;
  (C)b ^ a;
  b ^ a;  // Does not compile.
  return 0;
}

It compiles if the operator definition is outside:

class C {
 public:
  C(unsigned a) {}
 private:
  C();
};

C operator^(const C& a, const C& b) {
  return a;
}

int main() {
  C a = 5;
  unsigned b = 6;
  a ^ b;
  b ^ a;
  return 0;
}

How to log out from an AppEngine app only

2011-12-18T20:55:00.001+01:00

This blog post explains how to add a logout page to an AppEngine Python application, which will log out of the application only, without logging out of all of Google (e.g. Gmail, Calendar, Picasa, YouTube).

The default AppEngine users.create_logout_url(...) crates a logout URL which would log out from all of Google. To log out from the AppEngine app only, remove the SACSID and ACSID session cookies, which AppEngine has set right after logging in. Here is how to do it in Python:

import Cookie
import os
from google.appengine.api import users
from google.appengine.ext import webapp

class LogoutPage(webapp.RequestHandler):
  def get(self):
    target_url = self.request.referer or '/'
    if os.environ.get('SERVER_SOFTWARE', '').startswith('Development/'):
      self.redirect(users.create_logout_url(target_url))
      return

    # On the production instance, we just remove the session cookie, because
    # redirecting users.create_logout_url(...) would log out of all Google
    # (e.g. Gmail, Google Calendar).
    #
    # It seems that AppEngine is setting the ACSID cookie for http:// ,
    # and the SACSID cookie for https:// . We just unset both below.
    cookie = Cookie.SimpleCookie()
    cookie['ACSID'] = ''
    cookie['ACSID']['expires'] = -86400  # In the past, a day ago.
    self.response.headers.add_header(*cookie.output().split(': ', 1))
    cookie = Cookie.SimpleCookie()
    cookie['SACSID'] = ''
    cookie['SACSID']['expires'] = -86400
    self.response.headers.add_header(*cookie.output().split(': ', 1))
    self.redirect(target_url) 

...

application = webapp.WSGIApplication([..., ('/logout', LogoutPage), ...])
def main():
  run_wsgi_app(application)
if __name__ == '__main__':
  main()

After adding the /logout page as indicated above, offer the logout link like this:

self.response.out.write('<a href="/logout">Logout</a>')

Please note that adding a cookie which expires in the past makes the browser forget about the cookie immediately.

How to turn off voicemail for Swisscom (.ch) mobiles

2011-12-11T11:35:00.000+01:00

This blog post explains how to turn off the voicemail service (named COMBOX) provided (and enabled by default) by the Swiss mobile carrier Swisscom.

According to this PDF, lined from http://www.swisscom.ch/res/mobile/combox/index.htm, dial the following numbers:

##61#
##62#
##67#

Measure your reaction time using Python

2011-11-18T22:34:00.005+01:00

Run this in a terminal window (without the leading $):

$ python -c 'import random, time; time.sleep(2 + 8 * random.random()); \
  print 1; t = time.time(); raw_input(); print time.time() - t'

Press Enter as soon as the number 1 appears. Your reaction time (in seconds) will be printed. For best results, don't do it over an SSH connection. If you can go below .2 second, then you are most probably not human.

Announcing Portable MariaDB: Small, portable binary MariaDB distribution for Linux

2011-11-13T17:58:00.000+01:00

Portable MariaDB is a small, portable binary distribution of the SQL server MariaDB (Monty's fork of MySQL) for Linux i386 (32-bit). Only the mysqld binary and a versatile init script are included. Portable MariaDB can be run by any user in any directory, it doesn't try to access any mysqld data or config files outside its directory. Portable MariaDB can coexist with regular mysqld (MySQL or MariaDB) and other instances of Portable MariaDB on a single machine, as long as they are not configured to listen on the same TCP port. The only dependency of Portable MariaDB is glibc 2.4 (available in Ubuntu Hardy or later).

The most up-to-date documentation of Portable MariaDB is here.

The sources are here.

Why use Portable MariaDB?

It's small (not bloated). Fast to dowload, fast to extract, fast to install. Quick size comparison: mariadb-5.2.9-Linux-i686.tar.gz is 144 MB, the corresponding Portable MariaDB .tbz2 is less than 6 MB.
It's portable: does not interfere with other MySQL server installations on the same machine.
It's self-contained and consistent: copy the database and the configuration in a single directory from one machine to another.

Installation

To run Portable MariaDB, you need a Linux system with glibc 2.4 (e.g. Ubuntu Hardy) or later. 32-bit and 64-bit systems are fine. For 64-bit systems you need the 32-bit compatibility libraries installed. You also need Perl.

  $ cd /tmp  # Or any other with write access.
  $ BASE=http://pts-mini-gpl.googlecode.com/svn/trunk/portable-mariadb.release/ 
  $ wget -O portable-mariadb.tbz2 $BASE/portable-mariadb.tbz2
  $ tar xjvf portable-mariadb.tbz2
  $ chmod 700 /tmp/portable-mariadb  # For security.
  $ /tmp/portable-mariadb/mariadb_init.pl stop-set-root-password

Usage

For security, don't do anything as root.

  $ cd /tmp/portable-mariadb
  $ ./mariadb_init.pl restart
  Connect with: mysql --socket=/tmp/portable-mariadb/mysqld.sock --user=root --database=test --password
  Connect with: mysql --host=127.0.0.1 --user=root --database=test --password

Feel free to take a look at /tmp/portable-mariadb/my.cnf, make modifications, and restart mysqld so that the modifications take effect.

Security

By default, connections are accepted from localhost (Unix domain socket and TCP) only, all MySQL users are refused (except if a password has been set for root above), and root has unrestricted access. Unix permissions (such as the chmod 700 above) are protecting against data theft and manipulation on the file level.

It is strongly recommended to change the password of root to a non-empty, strong password before populating the database.

Java support

Java clients with JDBC (MySQL Connector/J) are fully supported. Please note that Java doesn't support Unix doman socket, so make sure in my.cnf that mysqld listens on a TCP port. Please make sure you have ?characterEncoding=UTF8 specified in your JDBC connection URL, otherwise some non-ASCII, non-Latin-1 characters would be converted to ?.

Unicode support

Just as with MariaDB. All encodings and collations are supported. The latin1 encoding is the default, which can be changed in my.cnf.

Language support

All natural languages (of MariaDB) are supported for error messages. Set the `language' flag in my.cnf accordingly. English is the default.

How to simply compress a C++ string with LZMA

2011-11-10T14:57:00.003+01:00

This blog post explains how to simply compress C++ with LZMA compression, using liblzma.

Use the following functions:

#include <stdlib.h>
#include "lzma.h"
#include <string>

// Level is between 0 (no compression), 9 (slow compression, small output).
std::string CompressWithLzma(const std::string& in, int level) {
  std::string result;
  result.resize(in.size() + (in.size() >> 2) + 128);
  size_t out_pos = 0;
  if (LZMA_OK != lzma_easy_buffer_encode(
      level, LZMA_CHECK_CRC32, NULL,
      reinterpret_cast<uint8_t*>(const_cast<char*>(in.data())), in.size(),
      reinterpret_cast<uint8_t*>(&result[0]), &out_pos, result.size()))
    abort();
  result.resize(out_pos);
  return result;
}

std::string DecompressWithLzma(const std::string& in) {
  static const size_t kMemLimit = 1 << 30;  // 1 GB.
  lzma_stream strm = LZMA_STREAM_INIT;
  std::string result;
  result.resize(8192);
  size_t result_used = 0;
  lzma_ret ret;
  ret = lzma_stream_decoder(&strm, kMemLimit, LZMA_CONCATENATED);
  if (ret != LZMA_OK)
    abort();
  size_t avail0 = result.size();
  strm.next_in = reinterpret_cast<const uint8_t*>(in.data());
  strm.avail_in = in.size();
  strm.next_out = reinterpret_cast<uint8_t*>(&result[0]);
  strm.avail_out = avail0;
  while (true) {
    ret = lzma_code(&strm, strm.avail_in == 0 ? LZMA_FINISH : LZMA_RUN);
    if (ret == LZMA_STREAM_END) {
      result_used += avail0 - strm.avail_out;
      if (0 != strm.avail_in)  // Guaranteed by lzma_stream_decoder().
        abort();
      result.resize(result_used);
      lzma_end(&strm);
      return result;
    }
    if (ret != LZMA_OK)
      abort();
    if (strm.avail_out == 0) {
      result_used += avail0 - strm.avail_out;
      result.resize(result.size() << 1);
      strm.next_out = reinterpret_cast<uint8_t*>(&result[0] + result_used);
      strm.avail_out = avail0 = result.size() - result_used;
    }
  }
}

Please note that in some use cases there may exist a solution which uses less memory.

See the liblzma/container. for documentation of lzma_easy_buffer_encode().

The decompression code was based on xzdec.c in xz-utils.

Long standby time for Cyanogenmod 7.1 on ZTE Blade phones

2011-10-27T08:58:00.001+02:00

This is a testimonial of the Cyanogenmod 7.1 Android spinoff operating system on the ZTE Blade phone.

I've recently installed Cyanogenmod 7.1 to my ZTE Blade. Prevously I had Cyanogenmod 7.0.3 on it, and the battery life was terrible. It didn't last more than 48 hours in standby (3G, mobile data, wifi, GPS, background synchronization switched off, only receiving a few text messages). But after installing 7.1, the battery lasted for 12 days plus 23 hours in standby mode. Awesome! Finally I have a small and cheap Android phone with long battery life.

The phone user interface also feels much snappier now, and 3D games (e.g. Falldown 3D) which lagged and were unplayable with 7.0.3 are fast and playable now.

Unfortunately I wasn't able to upgrade 7.0.3 to 7.1, but I had to wipe the phone (keeping only the SD card contents) before installing 7.1.

Getting started with IntelliJ to write Android applications on Linux

2011-10-18T13:55:00.003+02:00

This blog post gives instructions to get started with Android application development with IntelliJ on Linux.

Install the Java runtime environment (JRE) and the Java compiler (in the JDK). Command to to it on Ubuntu Lucid: sudo apt-get install openjdk-6-jdk . Please note that the JRE without the JDK is not enough, IntelliJ needs the JDK.
You will need about 1.5 GB of free disk space.
Download IntelliJ (either the Ultimate edition or the Community edition; please note that you have to pay for the Ultimate edition after the evaluation period expires) from from. http://www.jetbrains.com/idea/download/ . I've downloaded the file from http://download.jetbrains.com/idea/ideaIU-10.5.2.tar.gz , it was about 160 MB.
Download the Android SDK tools from http://developer.android.com/sdk/ . I've downloaded the file from http://dl.google.com/android/android-sdk_r13-linux_x86.tgz . It was about 160 MB.
If you ever want to connect a real phone via USB, then follow the instructions http://ptspts.blogspot.com/2011/10/how-to-fix-adb-no-permissions-error-on.html do create and install the android.rules udev rule. Please also restart the udev service.
Extract the downloaded archive android-sdk_r13-linux_x86.tgz .
Run the android-sdk-linux_x86/tools/android tool.
On the Available packages tab, find and install the following packages:
- Android SDK Tools
- Android SDK Platform-tools
- Documentation for Android SDK (the latest one)
- SDK Platform Android 2.2, API 8 (or whichever Android version you are developing for).
- Samples for SDK API 8 (or whichever Android version you are developing for).
- Google APIs by Google Inc., Android API 8 (or whichever Android version you are developing for).
- Android Compatibility package
In the Virtual devices tab, create a virtual device. Use these settings (make sure to enabling snapshots and disabling audio playback and recording):
Start the virtual device, play with it (it's an emulated Android phone), watch how it eats your CPU capacity, and stop it by closing the emulator window.
Please note that the emulator is very slow. It will happily eat 100% even if the virtual device is idle. When you disable sound playback and sound recording, it still eats about 800 MHz (tested on Intel(R) Core(TM)2 Duo CPU P9500 @ 2.53GHz).
A possible bug in the Android emulator: The Launch from snapshot functionality didn't work for me in the emulator, even though the snapshot was present.
In the file ~/Downloads/idea-IU-107.587/bin/idea.sh (the actual filename may depend on where you have downloaded and extracted IntelliJ to), change the line containing OPEN_JDK=$? to OPEN_JDK=1 . This will disable the startup warning.
Start IntelliJ. Enter license data (or choose evaluation), accept the license agreement, just click OK in the Select VCS Integration dialog (or select a superset of the version control systems you are planning to use), unselect all Web/JavaEE plugins (they are not needed for Android development), unselect all the HTML/JavaScript plugins, in the Other plugins dialog select at least these: Android, GenerateToString, Inspection Gadgets, Intention Power Pack, JUnit, Remote Hosts Access, SpellChecker, Structural Search, Task Management, Type Migration; make sure Android is checked; finish the installation.
File / New project. Create project from scratch. Next. Name: afirst. Select type: Android Module. Next. Create source... src. Next. Project JDK / Configure. /usr/lib/jvm/java-6-openjdk . OK. Next. Android SDK: ... . In the top left corner of the window, click +, and select Android SDK. Specify /home/USERNAME/Downloads/android-sdk-linux_x86 (where you have downloaded and extracted the Android SDK to). Select internal JDK: 1.6. OK. Select build target: Android 2.2 (or the Android version of your choice). OK. OK. Now you are back in the Create project wizard, with the Android SDK: selection containing Android 2.2 Platform. Finish.
Wait a few minutes until the project tree afirst appears. Open afirst. Open src. Open com.example. Double click on MyActivity. The MyActivity.java source file appears.
Make sure the virtual device barvirt is running in the Android emulator. Play it safe and restart the emulator.
Stop adb: sudo ~/Downloads/android-sdk-linux_x86/platform-tools/adb kill-server
Restart adb: sudo ~/Downloads/android-sdk-linux_x86/platform-tools/adb devices
In IntelliJ Run / Edit configurations, set up devices and virtual devices like this:
In IntelliJ: Run / Run. Wait a minute. The app should start on the emulator.
Connect your phone with USB debugging enabled (in Settings / Applications / Development). In IntelliJ: Run / Run. Wait a minute. The app should start on the phone.

Typical IntelliJ message for starting the app on the phone:

Waiting for device.
Target device: 1234567890ABCDEF
Uploading file
 local path: /home/pts/IdeaProjects/afirst/out/production/afirst/afirst.apk
 remote path: /data/local/tmp/com.example
Installing com.example
DEVICE SHELL COMMAND: pm install -r "/data/local/tmp/com.example"
pkg: /data/local/tmp/com.example
Success


Launching application: com.example/com.example.MyActivity.
DEVICE SHELL COMMAND: am start -n "com.example/com.example.MyActivity"
Starting: Intent { cmp=com.example/.MyActivity }

Typical IntelliJ message for starting the app on the emulator:

Waiting for device.
Target device: emulator-5554 (barvirt)
Uploading file
 local path: /home/pts/IdeaProjects/afirst/out/production/afirst/afirst.apk
 remote path: /data/local/tmp/com.example
Installing com.example
DEVICE SHELL COMMAND: pm install -r "/data/local/tmp/com.example"
pkg: /data/local/tmp/com.example
Success


Launching application: com.example/com.example.MyActivity.
DEVICE SHELL COMMAND: am start -n "com.example/com.example.MyActivity"
Starting: Intent { cmp=com.example/.MyActivity }

Named return value optimization in gcc

2011-10-06T12:08:00.000+02:00

This blog post demonstrates that GCC 4.1 does Named return value optimization, i.e. it omits creating a temporary object in a function returning an object if all return statements in the function return the same local variable.Example code:

#include <stdio.h>

class C {
 public:
  C() { printf("+\n"); }
  ~C() { printf("-\n"); }
  C(const C&) { printf(":\n"); }
  C& operator=(const C&) { printf("=\n"); }
};

C F(int i) {
  C x;
  if (i > 1) {
    return x;
  } else {
    return x;
  }
}

C G(int i) {
  if (i > 1) {
    C x;
    return x;
  } else {
    C y;
    return y;
  }
}

int main(int argc, char**) {
  F(argc);
  printf("~~~\n");
  G(argc);
  return 0;
}

The output, as expected, even without -O... compiler optimization flags:

$ g++ test_return_object.cc && ./a.out
+
-
~~~
+
:
-
-

How to initiate and receive voice and video calls from within Gmail?

2011-10-04T11:34:00.000+02:00

This blog post explains how to set up Gmail so it will be able to initiate and receive voice and video calls.

Download and install the Google Talk Plugin from here.
Restart your browser, preferably Google Chrome.
Log into gmail in your browser.
In the top right corner click on the cogwheel icon, and then select Mail settings.
In the Chat tab, set up the settings of Voice and video chat.
Click on Verify your settings and make sure it works properly.
Save the settings (Save Changes).
Select a friend in the chat list, open a chat window. In the chat window there will be a phone or camera icon, with which you can start the voice or video chat.

How to fix the adb no permissions error on Ubuntu Lucid

2011-10-02T14:59:00.002+02:00

This blog post explains how to fix the no permissions error the Android Debugger displays when running the adb devices command Linux. The solutions presented here have been tested and found working on Ubuntu Lucid, but they should also work on other Ubuntu and Linux versions as well.

This official help center page has many working and non-working answers to the problem, and many comments. The root cause of the problem is that the current user doesn't have sufficient permissions to access the USB device file created when the phone was connected in USB debug mode.

Killing all running adb server instances with sudo killall adb, disconnecting the phone, reconnecting the phone, and then running sudo adb devices instead (so adb gets run as root) fixes the problem.

Alternatively, it is possible to implement a long-term fix which doesn't require sudo, thus it works with e.g. IntelliJ IDE integration:

Create a file named /tmp/android.rules with the following contents (hex vendor numbers were taken from the vendor list page):

SUBSYSTEM=="usb", ATTRS{idVendor}=="0bb4", MODE="0666"
SUBSYSTEM=="usb", ATTRS{idVendor}=="0502", MODE="0666"
SUBSYSTEM=="usb", ATTRS{idVendor}=="0b05", MODE="0666"
SUBSYSTEM=="usb", ATTRS{idVendor}=="413c", MODE="0666"
SUBSYSTEM=="usb", ATTRS{idVendor}=="0489", MODE="0666"
SUBSYSTEM=="usb", ATTRS{idVendor}=="091e", MODE="0666"
SUBSYSTEM=="usb", ATTRS{idVendor}=="18d1", MODE="0666"
SUBSYSTEM=="usb", ATTRS{idVendor}=="0bb4", MODE="0666"
SUBSYSTEM=="usb", ATTRS{idVendor}=="12d1", MODE="0666"
SUBSYSTEM=="usb", ATTRS{idVendor}=="24e3", MODE="0666"
SUBSYSTEM=="usb", ATTRS{idVendor}=="2116", MODE="0666"
SUBSYSTEM=="usb", ATTRS{idVendor}=="0482", MODE="0666"
SUBSYSTEM=="usb", ATTRS{idVendor}=="17ef", MODE="0666"
SUBSYSTEM=="usb", ATTRS{idVendor}=="1004", MODE="0666"
SUBSYSTEM=="usb", ATTRS{idVendor}=="22b8", MODE="0666"
SUBSYSTEM=="usb", ATTRS{idVendor}=="0409", MODE="0666"
SUBSYSTEM=="usb", ATTRS{idVendor}=="2080", MODE="0666"
SUBSYSTEM=="usb", ATTRS{idVendor}=="0955", MODE="0666"
SUBSYSTEM=="usb", ATTRS{idVendor}=="2257", MODE="0666"
SUBSYSTEM=="usb", ATTRS{idVendor}=="10a9", MODE="0666"
SUBSYSTEM=="usb", ATTRS{idVendor}=="1d4d", MODE="0666"
SUBSYSTEM=="usb", ATTRS{idVendor}=="0471", MODE="0666"
SUBSYSTEM=="usb", ATTRS{idVendor}=="04da", MODE="0666"
SUBSYSTEM=="usb", ATTRS{idVendor}=="05c6", MODE="0666"
SUBSYSTEM=="usb", ATTRS{idVendor}=="1f53", MODE="0666"
SUBSYSTEM=="usb", ATTRS{idVendor}=="04e8", MODE="0666"
SUBSYSTEM=="usb", ATTRS{idVendor}=="04dd", MODE="0666"
SUBSYSTEM=="usb", ATTRS{idVendor}=="0fce", MODE="0666"
SUBSYSTEM=="usb", ATTRS{idVendor}=="0930", MODE="0666"
SUBSYSTEM=="usb", ATTRS{idVendor}=="19d2", MODE="0666"

Run the following commands (without the leading $):

$ sudo cp /tmp/android.rules /etc/udev/rules.d/51-android.rules
$ sudo chmod 644   /etc/udev/rules.d/51-android.rules
$ sudo chown root. /etc/udev/rules.d/51-android.rules
$ sudo service udev restart
$ sudo killall adb

Disconnect the USB cable between the phone and the computer.
Reconnect the phone.
Run adb devices to confirm that now it has permission to access the phone.

Please note that it's possible to use , USER="$LOGINNAME" instead of , MODE="0666" in the .rules file, substituting $LOGINNAME for your login name, i.e. what id -nu prints.

How to install kradview on Ubuntu Lucid

2011-09-09T19:01:00.000+02:00

This blog post explains how to install and run kradview on Ubuntu Lucid.

Compiling and installing:

$ sudo apt-get install libqt3-headers kdelibs4-dev
$ wget -O /tmp/kradview-1.1.0.tgz http://www.orcero.org/irbis/kradview/kradview-1.1.0.tgz
$ (cd /tmp && tar xzvf kradview-1.1.0.tgz)
$ cd /tmp/kradview-1.1.0
$ ./configure --without-arts
$ make
$ sudo make install
$ sudo ln -sf ../kde/bin/kradview /usr/local/bin/kradview

Running:

$ kradview

Running with a file preopened:

$ kradview FILENAME

Please note that it might not be necessary to install kradview, because the display utility of ImageMagick may be able to display the image. Installation: $ sudo apt-get install imagemagick . Usage: $ display FILENAME .

How to set up Skype notifications on Linux to be non-distruptive

2011-08-06T15:58:00.001+02:00

This blog post presents my Skype notification setup on Linux. This setup contains no audio notifications and only a few visual notifications which don't disturb my regular work.

In the Skype Options dialog box, select the Chat tab, and set it up like this:

When somebody starts a chat with me...: Create a minimised chat window
Show emoticons: yes
Show animated emoticons: no

In the Skype Options dialog box, select the Notifications tab, and set it up like this:

Skype Login: no Enable Event
Skype Logout: no Enable Event
Skype Login Failed: Enable Event, no sound file
Contact Came Online: no Enable Event
Contact Went Offline: no Enable Event
First Chat Message Received: Enable Event, no sound file, Display pop-up notification
Chat Message Received: Enable Event, no sound file, Display pop-up notification
Chat Message Sent: no Enable Event

Apply the settings, and close the Options dialog box.

How to set up a compact desktop theme on Ubuntu Lucid

2011-08-06T15:44:00.002+02:00

This blog post explains how to set up a compact GNOME desktop theme on Ubuntu Lucid, i.e. such a theme which consumes little screen real estate, leaving space for the contents you work on.

Install and activate the necessary small fonts (FixedSC and Helvetica), as described in this blog post.

Select the FixedSC font in GNOME Terminal.

Run this command (without the $) to install GNOME themes:

$ sudo apt-get install gnome-themes{,-extras,-more,-selected,-ubuntu}

In System / Preferences / Appearance / Theme / Customize, select the following themes:

Controls: Simple
Window Border: Mist
Icons: Mist

Some of the changes take effect immediately, but you may have to restart some applications.

How to set up nicely hinted default fonts on Linux

2011-08-06T15:28:00.001+02:00

This blog post explains how to set up a per-user default font configuration under Linux (tested on Ubuntu Lucid), which contains easy-to-read fonts in small sizes and properly hinted rendering.

Install the Microsoft core fonts (e.g. Times New Roman, Arial, Courier New), using the following command (without the $ sign):

$ sudo apt-get install msttcorefonts

Create (or overwrite) the file ~/.fonts.conf with the following contents:

<?xml version="1.0"?>
<!DOCTYPE fontconfig SYSTEM "fonts.dtd">
<fontconfig>

<!-- Accept bitmap fonts -->
  <selectfont>
    <acceptfont>
    <pattern>
      <patelt name="scalable"><bool>false</bool></patelt>
    </pattern>
    </acceptfont>
  </selectfont>

<!--  Enable proper hinting (medium) and disable subpixel rendering (rgba=none)
      for the Microsoft Core fonts (msttcorefonts) with high quality hints.
  --> 
  <match target="font">
    <test name="family"><string>Andale mono</string></test>
    <test name="pixelsize" compare="more"><double>7.5</double></test>
    <edit name="hinting"><bool>true</bool></edit>
    <edit name="hintstyle"><const>hintmedium</const></edit>
    <edit name="rgba"><const>none</const></edit>
  </match>
  <match target="font">
    <test name="family"><string>Arial</string></test>
    <test name="pixelsize" compare="more"><double>7.5</double></test>
    <edit name="hinting"><bool>true</bool></edit>
    <edit name="hintstyle"><const>hintmedium</const></edit>
    <edit name="rgba"><const>none</const></edit>
  </match>
  <match target="font">
    <test name="family"><string>Arial Black</string></test>
    <test name="pixelsize" compare="more"><double>7.5</double></test>
    <edit name="hinting"><bool>true</bool></edit>
    <edit name="hintstyle"><const>hintmedium</const></edit>
    <edit name="rgba"><const>none</const></edit>
  </match>
  <match target="font">
    <test name="family"><string>Comic Sans MS</string></test>
    <test name="pixelsize" compare="more"><double>7.5</double></test>
    <edit name="hinting"><bool>true</bool></edit>
    <edit name="hintstyle"><const>hintmedium</const></edit>
    <edit name="rgba"><const>none</const></edit>
  </match>
  <match target="font">
    <test name="family"><string>Courier New</string></test>
    <test name="pixelsize" compare="more"><double>7.5</double></test>
    <edit name="hinting"><bool>true</bool></edit>
    <edit name="hintstyle"><const>hintmedium</const></edit>
    <edit name="rgba"><const>none</const></edit>
  </match>
  <match target="font">
    <test name="family"><string>Georgia</string></test>
    <test name="pixelsize" compare="more"><double>7.5</double></test>
    <edit name="hinting"><bool>true</bool></edit>
    <edit name="hintstyle"><const>hintmedium</const></edit>
    <edit name="rgba"><const>none</const></edit>
  </match>
  <match target="font">
    <test name="family"><string>Impact</string></test>
    <test name="pixelsize" compare="more"><double>7.5</double></test>
    <edit name="hinting"><bool>true</bool></edit>
    <edit name="hintstyle"><const>hintmedium</const></edit>
    <edit name="rgba"><const>none</const></edit>
  </match>
  <match target="font">
    <test name="family"><string>Times New Roman</string></test>
    <test name="pixelsize" compare="more"><double>7.5</double></test>
    <edit name="hinting"><bool>true</bool></edit>
    <edit name="hintstyle"><const>hintmedium</const></edit>
    <edit name="rgba"><const>none</const></edit>
  </match>
  <match target="font">
    <test name="family"><string>Trebouchet MS</string></test>
    <test name="pixelsize" compare="more"><double>7.5</double></test>
    <edit name="hinting"><bool>true</bool></edit>
    <edit name="hintstyle"><const>hintmedium</const></edit>
    <edit name="rgba"><const>none</const></edit>
  </match>
  <match target="font">
    <test name="family"><string>Verdana</string></test>
    <test name="pixelsize" compare="more"><double>7.5</double></test>
    <edit name="hinting"><bool>true</bool></edit>
    <edit name="hintstyle"><const>hintmedium</const></edit>
    <edit name="rgba"><const>none</const></edit>
  </match>
  <match target="font">
    <test name="family"><string>Webdings</string></test>
    <test name="pixelsize" compare="more"><double>7.5</double></test>
    <edit name="hinting"><bool>true</bool></edit>
    <edit name="hintstyle"><const>hintmedium</const></edit>
    <edit name="rgba"><const>none</const></edit>
  </match>

<!--  Just a safe bet: enable proper hinting for the DejaVu font family.
      Bitstream Vera is not shipped on Ubuntu Lucid anymore (but there was a
      package on Ubuntu Hardy).
  --> 
  <match target="font">
    <test name="family"><string>DejaVu Sans</string></test>
    <test name="pixelsize" compare="more"><double>7.5</double></test>
    <edit name="hinting"><bool>true</bool></edit>
    <edit name="hintstyle"><const>hintmedium</const></edit>
    <edit name="rgba"><const>none</const></edit>
  </match>
  <match target="font">
    <test name="family"><string>DejaVu Sans Mono</string></test>
    <test name="pixelsize" compare="more"><double>7.5</double></test>
    <edit name="hinting"><bool>true</bool></edit>
    <edit name="hintstyle"><const>hintmedium</const></edit>
    <edit name="rgba"><const>none</const></edit>
  </match>
  <match target="font">
    <test name="family"><string>DejaVu Sans,DejaVu Sans Condensed</string></test>
    <test name="pixelsize" compare="more"><double>7.5</double></test>
    <edit name="hinting"><bool>true</bool></edit>
    <edit name="hintstyle"><const>hintmedium</const></edit>
    <edit name="rgba"><const>none</const></edit>
  </match>
  <match target="font">
    <test name="family"><string>DejaVu Sans Condensed</string></test>
    <test name="pixelsize" compare="more"><double>7.5</double></test>
    <edit name="hinting"><bool>true</bool></edit>
    <edit name="hintstyle"><const>hintmedium</const></edit>
    <edit name="rgba"><const>none</const></edit>
  </match>
  <match target="font">
    <test name="family"><string>DejaVu Sans,DejaVu Sans Light</string></test>
    <test name="pixelsize" compare="more"><double>7.5</double></test>
    <edit name="hinting"><bool>true</bool></edit>
    <edit name="hintstyle"><const>hintmedium</const></edit>
    <edit name="rgba"><const>none</const></edit>
  </match>
  <match target="font">
    <test name="family"><string>DejaVu Sans Light</string></test>
    <test name="pixelsize" compare="more"><double>7.5</double></test>
    <edit name="hinting"><bool>true</bool></edit>
    <edit name="hintstyle"><const>hintmedium</const></edit>
    <edit name="rgba"><const>none</const></edit>
  </match>
  <match target="font">
    <test name="family"><string>DejaVu Serif</string></test>
    <test name="pixelsize" compare="more"><double>7.5</double></test>
    <edit name="hinting"><bool>true</bool></edit>
    <edit name="hintstyle"><const>hintmedium</const></edit>
    <edit name="rgba"><const>none</const></edit>
  </match>
  <match target="font">
    <test name="family"><string>DejaVu Serif,DejaVu Serif Condensed</string></test>
    <test name="pixelsize" compare="more"><double>7.5</double></test>
    <edit name="hinting"><bool>true</bool></edit>
    <edit name="hintstyle"><const>hintmedium</const></edit>
    <edit name="rgba"><const>none</const></edit>
  </match>
  <match target="font">
    <test name="family"><string>DejaVu Serif Condensed</string></test>
    <test name="pixelsize" compare="more"><double>7.5</double></test>
    <edit name="hinting"><bool>true</bool></edit>
    <edit name="hintstyle"><const>hintmedium</const></edit>
    <edit name="rgba"><const>none</const></edit>
  </match>

<!-- Set font family preferences for Firefox. Chrome seems to ignore these.
     Please note that /etc/fonts/conf.d/60-latin.conf has more defaults, which
     we don't have to repeat here.
  -->
  <alias><family>serif</family><prefer>
    <family>Times New Roman</family>
  </prefer></alias>
  <alias><family>sans-serif</family><prefer>
    <family>Arial</family>
  </prefer></alias>
  <alias><family>monospace</family><prefer>
    <family>DejaVu Sans Mono</family>
  </prefer></alias>
</fontconfig>

The hintmedium setting above is especially important, because it turns on the proper hinting for Times New Roman etc., which makes these fonts easy to read at small sizes. The default is slight hinting (instead of medium), which is wrong, and it remains wrong (because of a bug) even if you select it at System / Preferences / Appearance / Fonts / Details / Hinting / Medium. The proper way to get the medium hinting is to write a ~/.fonts.conf file, as above.

After changing ~/.fonts.conf log out and back in, or restart all applications for the changes to take effect.

Google Chrome doesn't pay attention to the default fonts specified in ~/.fonts.conf. In Chrome, visit chrome://settings/fonts, and change the defaults to:

Standard Font: Times New Roman
Serif Font: Times New Roman
Sans Serif Font: Arial
Fixed-Width Font: DejaVu Sans Mono

You may want to change the window title and dialog box fonts as well. You can do that in System / Preferences / Appearance / Fonts. See this blog post for some crisp (i.e. non-antialiased), small font size defaults. Don't forget to install the FixedSC and Helxetica fonts first, as documented there.

How to set up English search in Google Chrome

2011-08-06T13:58:00.001+02:00

This blog post explains how to set up the default search engine to Google in English language in Google Chrome.

By default, if Google is the default search engine in Google Chrome, it returns the search result in the language of the country which is autodetected as the location of the computer. This is bad behavior for most English-speaking travelers, who always want the search results in English, no matter which country they are visiting.

To fix it for yourself, go to chrome://settings/searchEngines in Chrome, and add a new search engine named Google in English, the shortcut (second column) being gg , and the URL being http://www.google.com/search?hl=en&q=%s . Save it, make it the default, and remove all other search engines with Google in their name.

From now on, you can type a search query to the address bar (optionally prefixed by gg and a space), and you will get Google results in English.

Unfortunately the search engine setting is not synchronized between your Google Chrome installations, so you have to configure it manually on each machine and login account.

How to disable cursor blinking in GNOME applications

2011-08-04T20:23:00.002+02:00

This blog post explains how to make the cursor not blink in text input and text area widgets in GNOME applications (including GNOME Terminal). The settings have been tested and found working on Ubuntu Lucid.

Run the following commands (without the leading $) in a terminal window:

$ gconftool-2 --set /desktop/gnome/interface/cursor_blink --type boolean False
$ gconftool-2 --set /apps/gnome-terminal/profiles/Default/cursor_blink --type boolean False
$ gconftool-2 --set /apps/gnome-terminal/profiles/Default/cursor_blink_mode --type string off

Changes take effect immediately, there is no need to restart running applications.

These changes also affect Google Chrome (including the address bar and input fields and textarea fields on the page). However, changes don't take effect within a pages in existing tabs, so you should close them, and create new tabs.

If some cursors still Please see this noblink page to disable blinking in Emacs, XEmacs, GVIM, GTK, Gimp, Qt, LessTif, Mozilla Firefox, Mozilla Thunderbird, Google Chrome, TCL/Tk, Linux console and Microsoft Windows.

How to write hello-world in both Java and C++

2011-08-04T11:28:00.000+02:00

This blog post explains hwo to write a hello-world program which compiles in both Java and C++.

The following solution makes use of the language difference that //...\ continues the one-line comment in C++, but not in Java.

//\
class t{public static void main(String[]a){System.out.print/*
#include<stdio.h>
int main(){if(1){return!printf//*/
("Hello, World!\n");}}

Python 3.2 binaries released in StaticPython for Linux

2011-06-24T23:52:00.000+02:00

This blog post is to announce that StaticPython, a binary distribution of Python, has just released Python 3.2 binaries for Linux. (Previously StaticPython contained only Python 2.7 binaries.) Download links for the binaries:

python3.2: almost all built-in Python modules, C extensions (including sqlite3) and greenlet
stackless3.2: ditto, but with Stackless Python
stacklessxl3.2: ditto, and also OpenSSL (with the _ssl and _hashlib modules)

The Python 3.2 binaries of StaticPython can be useful on Linux and FreeBSD systems where Python 3.2 distribution packages are not available, and recompiling from source would be too inconvenient. They can also be used to demo the features of Python 3 for users with a little time and patience, and without a commitment to install it.

PTetriS/HTML: A minimalistic, but correct Tetris-clone in HTML + JavaScript

2011-06-05T18:56:00.000+02:00

This blog post is to announce PTetriS/HTML, a minimalistic, but correct Tetris-clone in HTML + JavaScript. Play PTetriS/HTML here, with a JavaScript-enabled browser.

PTetris/HTML was born to celebrate the coming 10-year anniversary of PTetris, the same Tetris-clone implemented as a Java applet and application. Play PTetriS here, with a Java-enabled browser.

Please note that the PTetriS games are not feature rich (e.g. they don't count the score and they don't show the next brick, and they don't have multiple levels). This is intentional, so they can work as a reference implementation of a correct Tetris game. By correct I mean that the board size is correct, bricks rotate in the correct direction, and the rotation pixel pattern of the bricks is correct (e.g. they don't jump up or down when changing the rotation phase). These properties are copied from the old game FUXOFT's Tetris2 (architecture: ZX Spectrum). There are many Tetris-clones available nowadays which are incorrect.

How to download a https:// page in Ruby 1.8

2011-06-04T02:11:00.000+02:00

This blog post shows an example low-level implementation of downloading a https:// page in Ruby 1.8. The reason why is this blog post was born is that the documentation of the openssl and socket Ruby modules didn't contain a working end-to-end example.

#! /usr/bin/ruby1.8
require 'socket'
require 'openssl'
# Returns a HTTP response header + body.
def download_https(host, port, suburl)
  s = TCPSocket.new('www.gmail.com', 443)
  begin
    ss = OpenSSL::SSL::SSLSocket.new(s)
    begin
      ss.connect
      ss << "GET #{suburl} HTTP/1.0\r\nHost: #{host}:#{port}\r\n\r\n"
      ss.flush
      ss.read
    ensure
      ss.close
    end
  ensure
    s.close
  end
end
p download_https('www.gmail.com', 443, '/')

Please note that this doesn't check the authenticity of the server, i.e. it will happily and silently accept self-signed certificates.

jtty-builder: easy-to-install JSP web server for Unix

2011-05-19T15:00:00.002+02:00

This blog post is an announcement for jtty-builder, an easy-to-install, lightweight webserver with a JSP container for Unix. More specifically, jtty-builder is just a set of shell scripts for Unix which automate downloading and building a lightweight Java web and application server for serving JSP (and Java servlets). The container used is Jtty, which uses Jetty, which uses parts of Apache Tomcat (mostly Jasper). Most steps are automated. There is no need to write configuration files.

See the most recent documentation of jtty-builder here.

To start serving a hello-world JSP page on Unix, do this:

As a shortcut, you can use a prebuilt jttyp.jar instead of building one using jtty-builder. To do that, run the following command, and skip steps 1 to 5 (inclusive):
```
$ wget -O jttyp.jar \
    http://pts-mini-gpl.googlecode.com/svn/trunk/jtty-prebuilt/jttyp.jar
```

Download jtty-builder:

$ svn co http://pts-mini-gpl.googlecode.com/svn/trunk/jtty-builder

Install java and javac. Example command on Ubuntu Lucid:
```
$ sudo apt-get install openjdk-6-jdk
```
Use jtty-builder to download the sources of Jtty, Jetty and Tomcat:
```
(cd jtty-builder && ./download_jttyp.jar)
```

Use jtty-builder to build jttyp.jar:

$ (cd jtty-builder && ./build_jttyp.jar)

Copy jttyp.jar to the application directory:
```
$ cp jtty-builder/jttyp.jar .
```

Create your .war file or application directory. A simple example:

$ mkdir Hello-World
$ (echo '<html><head><title>JSP Test</title>'
   echo '<%! String message = "Hello, World."; %>'
   echo '</head><body><h2><%= message %></h2>'
   echo '<%= new java.util.Date() %></body></html>') >Hello-World/hi.jsp
$ mkdir Hello-World/WEB-INF
$ (echo '<web-app>'
   echo '<display-name>Hello World</display-name>'
   echo '</web-app>') >Hello-World/WEB-INF/web.xml

Start the Java application server running your Hello-World application:
```
java -jar jttyp.jar 8765 Hello-World
```
(Keep the java application running.)
Try the web page in your web browser by visiting http://127.0.0.1:8765/ Reload it to get the time updated.
If you modify hi.jsp and reload the page, it gets autmatically recompiled and the new version will run.

See the Jtty project page for more information about using jtty.jar and jttyp.jar for staring Java web applications from the command line.

How to write a simple echo and chat server in Ruby 1.8

2011-04-27T15:29:00.001+02:00

This blog post is an introduction to TCP socket server programming and thread programming in Ruby 1.8. It also illustrates how compact and expressive Ruby code can be. The intended audience is Ruby beginners.

require 'socket'  # TCPServer
ss = TCPServer.new(1233)
loop {
  Thread.start(ss.accept) { |s|
    begin
      while line = s.gets;  # Returns nil on EOF.
        (s << "You wrote: #{line.inspect}\r\n").flush
      end
    rescue
      bt = $!.backtrace * "\n  "
      ($stderr << "error: #{$!.inspect}\n  #{bt}\n").flush
    ensure
      s.close
    end
  }
}

Use telnet 127.0.0.1 1233 to connect to the echo server as a new participant, and then type your chat message(s) terminated by Enter (newline). The echo server sends back each message it receives with the You wrote boilerplate around it. Multiple independent clients can connect to the chat server at a time.

The chat server:

require 'socket'  # TCPServer
require 'thread'  # Queue
ssock = TCPServer.new(1234)
msgs = Queue.new
participants = []
Thread.start {  # Send chat messages to participants.
  while msg = msgs.pop;  # Always true.
    participants.each { |s|
      (s << msg).flush rescue IOError
    }
  end
}
loop {
  Thread.start(ssock.accept) { |sock|
    participants << sock
    begin
      while line = sock.gets;  # Returns nil on EOF.
        msgs << ": #{line.chomp!}\r\n"
      end
    rescue
      bt = $!.backtrace * "\n  "
      ($stderr << "error: #{$!.inspect}\n  #{bt}\n").flush
    ensure
      participants.delete sock
      sock.close
    end
  }
}

Use telnet 127.0.0.1 1234 to connect to the chat server as a new participant, and then type your chat message(s) terminated by Enter (newline). The chat server sends each message (with a colon prepended) to all connected participants.

The Ruby source files above demonstrate the following:

how to create a thread
how to use thread-local variables (e.g. sock)
how to report exceptions in a thread
how to ignore IOError
how to read a line from a socket
how to use the Queue class for synchronization between threads
how to clean up using an ensure block
how to pass values to a newly created thread

python-xattr-compat: portable Python module to query and set POSIX extended attributes of files

2011-04-09T17:53:00.000+02:00

This blog post is an announcement of python-xattr-compat, a portable Python 2.x module to query and set POSIX extended attributes of files.

python-xattr-compat is a wrapper around the xattr Python package, but ifxattr is not available, python-xattr-compat provides compatible replacement implementations (using the dl or ctypes modules)for some systems (currently Linux), so we have a pure Python implementationfor extended attribute manipulations.

How it works: Many scripting languages have a syscall function to call arbitrary system calls (like lsetxattr(2)). Python doesn't have syscall built in, but it has the dl and ctypes modules, which let the programmer call functions in any C library, lincluding libc. Since libc on Linux has wrappers for syscall and lsetxattr etc., we can call these to get system-specific extended attribute manipulation without installing non-built-in Python extensions like xattr.

How to install CyanogenMod onto the ZTE Blade

2011-04-03T22:53:00.009+02:00

This blog post explains how I managed to install CyanogenMod to a ZTE Blade phone (both Gen 1 and Gen 2). Please note that there can be multiple solutions, some of them working only on some phone submodels. The solution I give here did work on the model I had.

Pick one of the methods below.

Method one: without rooting the phone (recommended)

This method is recommended, because it is the more straightforward and safer method. Although its description is longer than of method two, doing method one is faster.

You will need a USB cable to connect the phone to a computer and an SD card in the phone.

Instructions:

Make a backup of your data on the phone, this page has some suggestions how (don't use Titanium Backup though, because that needs a rooted phone). You may safely skip this step if you don't have any SMS, phone call log entry, application data, application configuration etc. to save (e.g. because the phone is new and you haven't customized it yet).
Make sure you know your wifi connection credentials (e.g. network name, security type, passphrase).
Make sure you know your Google account name (e-mail address) and password.
On the phone: Settings » Applications » Development » enable USB Debugging.
On the phone: Settings » Applications » enable Unknown sources – Allow installation of non-Market applications
Install the Ask Mr Pigfish application from the Market, run Ask Mr Pigfish, and see it displaying the generation number of your phone (gen1 or gen2). Take a not on paper containing the generation number.
Install the adb and the fastboot tools to your computer. For Linux, see How to install instructions below. For Windows or Mac, see this page. Another download for fastboot is on this page.
The command lines in following instructions are for Linux. Do it similarly on Mac and Windows (e.g. instead of wget, use your browser or download manager to download the file).
Download the ClockworkMod recovery with wget -O /tmp/recovery-clockwork-3.0.1.4-blade.img http://android.d3xt3r01.tk/cyanogen/blade/recovery/recovery-clockwork-3.0.1.4-blade.img
Download the latest version of CyanogenMod from this page. When these instructions were written, the equivalent download command was: wget -O /tmp/update-cm-7.0.0-RC4-Blade-signed.zip http://mirror.cyanogenmod.com/get/update-cm-7.0.0-RC4-Blade-signed.zip
Download the latest version of Google Apps from this page. When these instructions were written, the equivalent download command was: wget -O /tmp/gapps-gb-20110307-signed.zip http://android.d3xt3r01.tk/cyanogen/gapps/gapps-gb-20110307-signed.zip
If already connected, disconnect the phone from the computer.
Remove the SIM card from the phone if you don't have a data plan. This is to prevent the unconfigured CyanogenMod from making expensive data transfers before the wifi is configured. This needs powering off your phone first.
Power the phone back on and wait for it to start up.
Connect the phone to the computer using the USB cable. The phone should display a notification that an USB cable is connected (and possibly another notification that USB debugging is enabled).
Run adb devices on the computer, it should display something like
```
List of devices attached 
Blade-CM7       device
```
Copy the CyanogenMod ZIP file to the SD card by running adb push /tmp/update-cm-7.0.0-RC4-Blade-signed.zip /sdcard/cm.zip
Copy the Google Apps ZIP file to the SD card by running adb push /tmp/gapps-gb-20110307-signed.zip /sdcard/gapps.zip
To prevent data loss below, make sure the phone is fully charged. You can charge it quickly if you connect it to a wall socket instead of a computer.
If you have a gen1 phone and you want to install CyanogenMod 7.0.1 or later, then upgrade your phone to gen2 by following the How to upgrade a gen1 phone to gen2 and start ClockworkMod recovery below. Otherwise, follow these instructions:
1. Reboot the phone to fastboot mode by running adb reboot bootloader
2. Wait for about 20 seconds until the phone reboots into bootloader mode, showing only the green android logo on the screen. (If your phone boots normally a few seconds later, then this method has failed for you, please choose another method to install CyanogenMod to your phone. You may try removing the battery, reinserting the battery, holding the Volume Up key on the phone, and powering on the phone while Volume Up is still held.)
3. Run fastboot flash recovery /tmp/recovery-clockwork-3.0.1.4-blade.img
4. Run fastboot reboot
5. Wait for the phone to start up and detect that the USB cable is connected.
6. Run adb reboot recovery (this should take about 5 seconds).
7. Wait for about 5 seconds until the phone reboots into recovery mode, running the ClockworkMod recovery.
Your phone should be running ClockworkMod recovery now. It should display ClockworkMod Recovery v3.0.1.4 (or a larger version number) and a menu below, starting with reboot system now. Use the Volume Up and Volume Down keys to navigate the menu up or down. Navigate to the advanced item, use the Home button to select it. Use the Back button to go back to the main menu.
Select the main menu option Wipe data/factory reset
Select the main menu option Wipe cache partition
Select the main menu option Install zip from sdcard
Select the menu option Choose zip from sdcard
Select cm.zip . This will install CyanogenMod.
Go back to the main menu. Select the main menu option Install zip from sdcard
Select the menu option Choose zip from sdcard
Select gapps.zip . This will install Google Apps.
Go back to the main menu.
Select the main menu option reboot system now
CyanogenMod should be booting now. Configure everything for your needs.
Use the Menu button to configure the wifi first (i.e. below you enter your Google account password).
CyanogenMod might ask you which Google applications to install. If unsure, select all of them. After you make your selection, CyanogenMod will take you the Market page of all these applications. Accept and install them one by one.
If you don't have a data plan, make sure you configure the phone so it won't accidentally make data transfers.
Reinsert the SIM card. This needs powering off your phone first.

How to install adb and fastboot on Linux


# You might need to use anoter package manager for installation:
$ sudo apt-get install wget unzip
$ wget -O /tmp/linux_adb_fastboot.zip \
  http://android.d3xt3r01.tk/cyanogen/tools/linux_adb_fastboot.zip
$ (cd /usr/local/bin && sudo unzip /tmp/linux_adb_fastboot.zip)
$ sudo killall -9 adb
# This starts the daemon. `sudo' is not necessary on some systems.
$ sudo /usr/local/bin/adb devices

How to upgrade a gen1 phone to gen2 and start ClockworkMod recovery

Don't follow these steps unless you are asked to by another step in this howto.

Please note that these instructions apply only to gen1 (1st generation) ZTE Blade models. Doing it on any other phone (i.e. on a gen2 ZTE Blade or any phone other than a ZTE Blade) will most probably make the device useless beyond repair. You can use the Ask Mr Pigfish application (available from the Android Market) to check if you have a gen1 ZTE Blade.

Please note that by following these instructions, not only the configuration and user data, but the operating system will also be removed from the phone. (But the contents of the SD card will remain intact.) To make your (gen2) device useful after these instructions, you should already have an operating system installer ZIP file (e.g. cm.zip above) already on the SD card.

Instructions:

Download the ZIP file Gen1-to-Gen2-TPT-v2.zip from here. The download link can also be found in the section Update to Gen2 Radio of this page.
Unzip the contents of the ZIP file to the SD card, so there should be a folder named image in the root folder of the SD card, and that folder should contain 17 files, from amss.mbn to userdata.img . Make sure that the image folder is in the root folder of the SD card. The easiest way to do this is connecting your phone to the computer using the USB cable, mounting the SD card (using the notification area of the phone), and running a ZIP file extractor on the computer. Disconnect the drive from the computer, and umount the SD card when done.
Make sure the phone is fully charged.
Disconnect the phone from the computer and the charger.
Turn off the phone (by long pressing the power button).
Remove the battery.
Reinsert the battery.
Hold the volume up and menu buttons. Don't release them yet.
(Before doing this, read the next instruction step.) Push the power button to turn the phone on.
Text in green would start appearing on a black background. Release the volume up and menu buttons.
More text in green would start appearing with some indication of progress percentage. Wait for 30 seconds until its done.
The phone boots to ClockworkMod Recovery .

Method two: rooting the phone first

You will need an SD card in the phone.

Outline (see the detailed instructions in the links below):

Make a backup of your data on the phone. (See in method two below.)
Make sure you know your wifi connection credentials (e.g. network name, security type, passphrase).
Make sure you know your Google account name (e-mail address) and password.
Root the phone.
Install the ROM Manager application (from the Android Market).
Install ClockworkMod recovery using the ROM Manager application. (This step needs a rooted phone.)
Download the CyanogenMod update ZIP file (and possibly the Google Apps ZIP file) to the /sdcard.
To prevent data loss below, make sure the phone is fully charged. You can charge it quickly if you connect it to a wall socket instead of a computer.
Remove the SIM card from the phone if you don't have a data plan. This is to prevent the unconfigured CyanogenMod from making expensive data transfers before the wifi is configured. This needs powering off your phone first.
Use the ROM Manager application to start the ClockworkMod recovery.
Within ClockworkMod recovery, wipe the phone.
Within ClockworkMod recovery, install the CyanogenMod update ZIP file (and possibly the Google Apps ZIP file)
Within ClockworkMod recovery, reboot the phone.
CyanogenMod should be booting now. Configure everything for your needs.
If you don't have a data plan, make sure you configure the phone so it won't accidentally make data transfers.
Reinsert the SIM card. This needs powering off your phone first.

All these steps worked for me as explained in detail on the official install CyagnogenMod to ZTE Blade page, except for rooting the phone. That page and also most other web sites instruct the user to reboot the fastboot mode by holding the Volume Up key while powering up the phone. This didn't work for me, because my ZTE Blade had fastboot disabled (so it always booted normally). So I had to find another way to root the phone. The instructions on this page (those containing tinyurl.com/urooted) worked for me. These instructions use a version of the Universal Androot application specialized to the ZTE Blade. Please note that Universal Androot claims that the phone is already rooted — just ignore this message and root it anyway. Please also note that during the installation process you have to re-root the phone using Universal Androot each time you reboot the phone. So if ROM Manager complains that your phone is not rooted, then just run Universal Androot again, and then retry in ROM Manager.

Performance

If the phone feels slow and games are not enjoyable, then tune the performance settings:

Make sure that the latest version of CyanogenMod (at least 7.0.0 without RC in the version number) is installed. Get the newest version of the ZIP file from here. Copy the ZIP file to the (root of the) SD card. Reboot the phone to recovery mode (using adb reboot recovery or the ROM Manager application). In recovery mode, clear the cache and then select the update file (the ZIP file), install it, and reboot the phone.
Make sure that Application / Settings / CyanogenMOD Settings / Performance / Enable surface dithering is off.
In Application / Spare Parts make sure that all animations are disabled.
There seems to be no need for enabling GPU acceleration (debug.sf.hw=1 in /system/build.prop), CyanogenMod seems to be fast without it.

How to automatically synchronize the shell history between terminal windows

2011-03-25T18:32:00.001+01:00

This blog post explains how to automatically synchronize the Unix shell command-line history between multiple shell instances running on the same computer (but possibly different terminal windows), without requiring any manual action (such as running history -w or exiting from the shell) from the user, and also how to make commands be saved to the history file right at the point when the command is executed. The solutions described here work with bash 4.1 (tested with 4.1.5) or newer and zsh (tested with 4.3.4 and 4.3.10), run on Ubuntu Lucid. No attempt was made to synchronize the zsh history and the bash history.

Instructions for zsh

Put (append) this command to your ~/.zshrc and reopen your terminal windows so that the changes take effect:

setopt hist_ignore_dups share_history inc_append_history extended_history

The change takes effect in terminal windows and incoming SSH connections you open from now on. Close your old windows if necessary to avoid confusion. You can also run the source ~/.zshrc to make the change take effect.

Optionally, set your HISTSIZE and SAVEHIST to large enough values, e.g. put HISTSIZE=500000 SAVEHIST=5000000 in your ~/.zshrc.

Once the change took effect, you can start running a command in one terminal window, then just press Enter in another terminal window, then press Up in the other terminal window to get the command running in the other terminal window. Your shell history lines are now synchronized. Synchronization happens whenever you press Enter in a terminal window.

Instructions for bash

Since bash doesn't have the history sharing feature by default, you need to use a helper script for bash. Download it using the following command (without the first $):

$ wget -O ~/merge_history.bash http://www.math.bme.hu/~pts/cvsget.cgi/u=zsh/p=/M=zsh/c=f1/n=/zsh/merge_history.bash

Run

$ touch ~/.merged_bash_history

to signify to the helper script that you want to use the new history feature.

Put (append) this to your ~/.bashrc so the helper script gets loaded at shell startup:

source "$HOME"/merge_history.bash

Optionally, set HISTSIZE and HISTFILESIZE to large enough values, e.g. append the following to your ~/.bashrc:

HISTSIZE=500000 HISTFILESIZE=5000000

The change takes effect in terminal windows and incoming SSH connections you open from now on. Close your old windows if necessary to avoid confusion. You can also run the source command above to make the change take effect.

If you want your old shell history to be reused, please copy it any time, and then press Enter:

cp ~/.bash_history ~/.merged_bash_history

Explanation of the bash solution

Bash doesn't have support for manual or automatic synchronization of history files. By default, bash reads the history file ~/.bash_history (can be changed by setting the HISTFILE bash special variable) at startup, adds newly executed commands to its in-memory history buffer, and simply writes (or appends) the contents of the buffer to the history file upon clean exit. With this strategy, commands can be lost for good if the shell is killed with kill -9, or multiple shells are exiting (the last one wins, lines from all previous ones get irrecoverably lost). It's possible to affect this behavior with shopt -s histappend to force appending instead of overwriting, but that's still much less synchronization with command sorting by timestamp. The commands history -w, history -a history -c and history -r can be useful for managing the bash history files, but they may still cause data loss (see above).

To solve these problems, the helper script merge_history.bash above uses the following techniques:

It uses HISTTIMEFORMAT bash special variable to make sure timestamps are saved into the history file.
It contains a Perl script to merge history files (possibly created by concurrently running bash instances), sorting the results by timestamp and removing duplicates.
Before displaying the prompt (using the PROMPT_COMMAND bash special variable) loads the merged history file from disk.
When the user presses Enter to run a command, before the command is executed, it saves the in-memory history buffer to a file, and merges that file using the Perl script to the merged history file. It uses trap ... DEBUG (and various black magic trickery) to install its hook before the command is executed. Other solutions for bash history synchronization don't use this technique, so they suffer from the shortcoming of saving the most recently executed command too late (only when it finishes).

merge_history.bash was inspired by the bash preexec emulation script.

Get the most recent version of merge_history.bash from http://www.math.bme.hu/~pts/cvsget.cgi/u=zsh/p=/M=zsh/c=f1/n=/zsh/merge_history.bash . For your studying convenience, here is a (possibly outdated) copy:

MRG_DONEI=":$SHELLOPTS:"
if test "${MRG_DONEI#*:history:}" != "$PTS_DONEI" &&
   (test "${BASH_VERSION#[5-9].}" != "$BASH_VERSION" ||
    test "${BASH_VERSION#4.[1-9].}" != "$BASH_VERSION") &&
   test "$HOME" &&
   test -f "$HOME/.merged_bash_history"; then

# Merge the timestamped .bash_history files specified in $@ , remove
# duplicates, print the results to stdout.
function _mrg_merge_ts_history() {
  PERL_BADLANG=x perl -wne '
    use integer;
    use strict;
    use vars qw($prefix @lines);
    if (/^#(\d+)\n/) {
      $prefix = sprintf("%030d ", $1);
    } else {
      chomp; $prefix = sprintf("%030d ", time) if !defined $prefix;
      push @lines, "$prefix$_\n"; undef $prefix;
    }
    END {
      my $prev = "";
      for (sort @lines) {
        s@^(\d+) @@; my $ts = $1 + 0; my $cur = "#$ts\n$_";
        print $cur if $cur ne $prev;
        $prev = $cur;
      }
    }
  ' -- "$@"
}

# Read history from $HISTFILE_MRG, 
function _rdh() {
  test "$HISTFILE_MRG" || return
  local HISTFILE="$HISTFILE_MRG"
  # Make `history -w' prefix "$TIMESTAMP\n" to $HISTFILE
  local HISTTIMEFORMAT=' '
  history -c  # Clear the in-memory history.
  history -r  # Append the contents of $HISTFILE to the in-memory history.
}

export -n HISTTIMEFORMAT HISTFILE HISTFILE_MRG
unset HISTFILE  # No history file by default, equivalent to HISTFILE="".
unset HISTTIMEFORMAT
HISTFILE_MRG="$HOME/.merged_bash_history"
history -c  # Discard the current history, whatever it was.

function hook_at_debug() {
  test "$COMP_LINE" && return  # Within the completer.
  trap '' DEBUG  # Uninstall debug trap.
  test "$HISTFILE_MRG" || return
  if : >>"$HISTFILE_MRG"; then
    # Make `history -w' prefix "$TIMESTAMP\n" to $HISTFILE
    local HISTTIMEFORMAT=' '
    # TODO(pts): Don't save if nothing changed (i.e. `history -1` prints
    # the same sequence number as before).
    local TMPDIR="${TMPDIR:-/tmp}"
    local HISTFILE="$TMPDIR/whistory.$UID.$$"
    local MHISTFILE="$TMPDIR/mhistory.$UID.$$"
    history -w  # Write to /tmp/whistory.$$ .
    _mrg_merge_ts_history "$HISTFILE_MRG" "$HISTFILE" >"$MHISTFILE"
    command mv -f -- "$MHISTFILE" "$HISTFILE_MRG"
  fi
}

# Set these both so hook_at_debug gets called in a subshell.
set -o functrace > /dev/null 2>&1
shopt -s extdebug > /dev/null 2>&1

# As a side effect, we install our own debug hook. We wouldn't have to do
# that if bash had support for `preexec' (executed just after a command has
# been read and is about to be executed). in zsh.
PROMPT_COMMAND="trap '' DEBUG; _rdh; trap hook_at_debug DEBUG; $PROMPT_COMMAND"

fi  # End of the file's if guard.
unset MRG_DONEI

How to dump the full /etc/passwd and /etc/group files when NIS (YP) or LDAP is used

2011-02-16T17:57:00.004+01:00

This blog post explains how to dump the flattened version of /etc/passwd and /etc/group files on a Unix system when some user and group entries are stored in a NIS (YP), LDAP or other server.

To dump these fules, run the following commands (not as root, without the leading $).

$ getent group  >/tmp/group
$ getent passwd >/tmp/passwd

The equivalent commands in Perl (use them if getent(1) doesn't work for you):

$ perl -le'$,=":";while(@_=getgrent){$_[3]=~y@ @,@;print@_}'  >/tmp/group
$ perl -le'$,=":";while(@_=getpwent){print@_[0,1,2,3,6,7,8]}' >/tmp/passwd

For NIS (YP) users and groups only, one can use ypcat passwd and ypcat group, respectively.

How to run Syncless on Linux without installing it?

2011-02-01T18:22:00.002+01:00

This blog post explains how to run asynchronous, coroutine-based client and server I/O libraries like Syncless, gevent and Concurrence on Linux without installing them.

On Linux, it's possible to try Syncless without installation, using theStaticPython binary Python distribution, like this:

$ wget -O stacklessco2.7-static \
  http://pts-mini-gpl.googlecode.com/svn/trunk/staticpython/release/stacklessco2.7-static
$ chmod +x stacklessco2.7-static
$ ./stacklessco2.7-static
Python 2.7.1 Stackless 3.1b3 060516 (release27-maint, Feb  1 2011, 16:57:16)
[GCC 4.1.2] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> from syncless import coio
>>> coio.sleep(1.5)
(sleeping for 1.5 second)
<object object at 0xf7709490>
>>>

Limitations:

This solution requires Linux on x86 (i386) or x86_64 (amd64) architecture. If you have a different Unix system, you have to install Syncless the regular way (e.g. with sudo easy_install syncless).
This solution doesn't give you the newest version of Syncless.
You can't use Python extensions written in C, except for those compiled into stacklessco2.7-static. See the full list on the StaticPython home page.

Please note that StaticPython can run many other I/O frameworks and libraries, which don't require compiling C extensions. Such libraries include Eventlet, Tornado, Twisted, asyncore and circuits.

How to implement the C printf function in C++ in a typesafe way?

2011-01-21T20:32:00.003+01:00

This blog post explains how to implement the C printf() function in C++ in a typesafe and convenient way.

The most important advantage of the ANSI standard C printf() function are its compactness and expressive power. For example: printf("The %s answer is %05d.\n", "best", 42); means: take the next argument, treat it as a null-terminated string (const char*), and insert it. The format specifier %05d means: take the next argument as a signed integer, pad it with leading zeros to at least 5 digits, and insert it. Please note that printf() is not typesafe: the format specifier has to match the type of the argument. For example, %d wouldn't work for a long long (but %lld would be needed), and %d or %s wouldn't work for a double (but %g would be needed, and %Lg would be needed for a long double). On a mismatch, some compilers would indicate a compile error, others would generate misbehaving (i.e. with undefined behavior) code.

There is no equally expressive message formatting functionality in C++ with a similarly compact syntax. Of course, printf() as it is can be used in C++, but it isn't integrated conveniently with the C++ standard library: e.g. it's not possible to insert an std::string, it's not possible to write to an std::ostream, and it's not possible to append to an existing std::string. C++ streams (e.g. cout << "The " << "best" << " answer is " << 42 << ".\n";) bring type safety (e.g. no matter which integer, floating point or string type 42 and other arguments have, they will be inserted properly), but they don't provide compact and powerful formatting capabilities (e.g. how to specify to pad 42 with zeros to at lest 5 digits).

Would it be possible to combine the compactness and expressive power of printf() with the typesafe behavior of C++ streams? It isn't too hard to create a class named Print and implement an overloaded operator<< so that Print("The %s answer is %05d.\n") << "best" << 42 would work in a typesafe way (raising a run-time error reliably if the type of the argument doesn't match the format specifier, and converting the the size and type of the argument if needed, e.g. %d would work for both int and long long). But is it possible to do it while keeping the original syntax, like this: Printf("The %s answer is %05d.\n", "best", 42);?

Yes, that's possible, using two tricks: overloading operator, (the comma operator), and using a GCC feature of macro expansion with any number of arguments. The macro Printf would expand so that Printf("The %s answer is %05d.\n", "best", 42); expands to (C("The %s answer is %05d.\n"), "best", 42);, which is equivalent to operator,(operator,(C("The %s answer is %05d.\n", "best"), 42);. Here C is a helper class, who has a constructor C(const char* fmt) for the format string, and the relevant const C& operator,(const C&, const char*) and const C& operator,(const C&, int) comma operators are overloaded to do the actual printing, consuming a few more characters in the meantime from the format string, and using that format specifier to format the argument. Type safety is provided by the built-in overloading features of the C++ language (i.e. a different operator, is called based on the type of the argument).

One more trick is needed to get an compile-time error when printing of values of unsupported type is attempted, e.g. class Bad {}; Printf("%s", Bad());. By default, since there is no operator, overridden for the class Bad, the default comma operator is used silently, which is quite useless (compute the first argument, discard it, compute the second argument, return it), and effectively prevents the rest of the arguments from being printed. The solution here is to define template<typename T>void operator,(const C&, T t) { ... }, which would match all types (which are not matched by the non-template version of operator,), and if we write ... so that it produces a type error, then the problem is solved.

The source code of such a typesafe printf() implementation for C++ is available at http://code.google.com/p/pts-mini-gpl/source/browse/#svn/trunk/pts-printf. For the convenience of the programmer, it implements all these:

int Printf(const char* fmt, ...);  // To stdout.
int Printf(FILE*, const char* fmt, ...);
int Printf(std::ostream&, const char* fmt, ...);
int Printf(std::ostringstream&, const char* fmt, ...);
int Printf(std::ofstream&, const char* fmt, ...);
int Printf(std::string*, const char* fmt, ...);  // Append to string.
std::string SPrintf(const char* fmt, ...);

Please note that the implementation is not optimized for speed. Mostprobably it's not possible to implement a typesafe version of printf()which is also as fast.

How to drop caches on Linux?

2011-01-06T14:58:00.000+01:00

This blog post explains how to drop in-memory cache of data on disk (pagecache, dentries, inodes) on Linux. Dropping the cache is useful if one wants to run filesystem benchmarks in a reproducible way.

To drop the caches, run

$ echo 3 | sudo tee /proc/sys/vm/drop_caches

The solution above is from http://www.kernel.org/doc/Documentation/sysctl/vm.txt. Some other useful commands mentioned there:

$ echo 1 | sudo tee /proc/sys/vm/drop_caches  # drop pagecache
$ echo 2 | sudo tee /proc/sys/vm/drop_caches  # drop dentries and inodes
$ echo 3 | sudo tee /proc/sys/vm/drop_caches  # drop pagecache, dentries and inodes

Note: As this is a non-destructive operation and dirty objects are not freeable, the user should run sync first.

Announcement from the Unix kitchen staff

2011-01-05T18:09:00.005+01:00

A restaurant called fork() has recently been opened in our office building. The name was suggested and chosen by engineers. I imagine what kind of e-mail announcements they would be sending if the kitchen was run by Unix engineers.

fork() will be close()d this week, but it will not beshutdown(), we'll open() it again, but you have to wait() a couple ofdays. In the meantime, our hard-working and prof()essional kitchenstaff is not idle(): they are learning new Asian recipes. Pleaseread() the signs before coming to check if fork() is open(). We hope you'llhave a great time() eating Peking duck, a broad select()ion ofsandwiches, and ricEAGAIN with nice(), un-break()able chopsticks, 5times() a week. We have installed a new smoke extinguisher, which issplice()d to the oven, so it won't signal() the fire alarm() anymoreunnecessarily. If you have feedback, please write() to us. Or fill ourpoll() from (see the new link() on the home page) to help us keep insync() with the employees' taste. Your creat()ive ideas are alsowelcome to improve the furniture decoration – it can also be aclone() of a restaurant in popular culture, with some geek flavoradded. Please note you don't have access() to the building from theexit() on the 3rd floor, so enter at the reception, but lock() yourbike first outside, to one of the pipe()s. There is no need to pause()in break() 3 for coffee, there is a grinder in fork() now.

**environ-mentalist disclaimer: No silkworms were kill()ed for theoutfit of our cooks.

How to write a C++ program without libstdc++

2010-12-19T20:07:00.005+01:00

This blog post explains how to write a C++ program and compile with GCC (g++) so that the resulting binary doesn't depend on libstdc++. The reason for avoiding libstdc++ can be purity (don't depend on libraries you don't really need) and statically linking small tools (e.g. for Unix or Win32 console applications with MinGW).

The limitations are:

The standard C++ STL (e.g. #include <string> and #include <vector>) cannot be used. This functionality has to be reimplemented in the program.
The standard C++ streams (e.g. #include <iostream>) cannot be used. The standard C I/O library (e.g. #include <stdio.h>) is a smaller and faster replacement. A disadvantage: there is no polymorphic operator<<(ostream&, ...) method for convenient, type-agnostic output.
Exceptions cannot be used (i.e. try, catch and throw are disallowed).
RTTI (run-time type information) cannot be used.
dynamic_cast<...>(...) cannot be used (because it requires RTTI).

Here is how to do it:

Add the following C++ code to your program (can be in a separate source file):

#include <stdlib.h>
#include <unistd.h>  /* for write(), also available on Windows */
extern "C" void* emulate_cc_new(unsigned len) { \
  void *p = malloc(len);
  if (p == 0) {
    /* Don't use stdio (e.g. fputs), because that may want to allocate more
     * memory.
     */
    (void)!write(2, "out of memory\n", 14);
    abort();
  }
  return p;
}
extern "C" void emulate_cc_delete(void* p) {
  if (p != 0)
    free(p);
}
void* operator new  (unsigned len) __attribute__((alias("emulate_cc_new")));
void* operator new[](unsigned len) __attribute__((alias("emulate_cc_new")));   
void  operator delete  (void* p)   __attribute__((alias("emulate_cc_delete")));
void  operator delete[](void* p)   __attribute__((alias("emulate_cc_delete")));
void* __cxa_pure_virtual = 0;

Compile your program with g++ -c to create individual .o files. Add flags -fno-rtti -fno-exceptions to get compile errors for disabled features (exceptions and RTTI).
Link your executable with gcc -o prog code1.o code2.o ... It's important that you use gcc here instead of g++ because g++ would link against libstdc++.

This method has been tested and found working with GCC 3.2, GCC 4.2.1 and GCC 4.4.1 (both native Linux compilation and MinGW compilation), and it probably works with other GCC versions as well.

If you need dynamic_cast, add void* __gxx_personality_v0 = 0;, don't use -fno-rtti, and add dyncast.a to the gcc -o ... command-line. Here is how to create dyncast.a (about 55kB) for GCC 4.4.1:

$ ar x /usr/lib/gcc/i486-linux-gnu/4.4/libstdc++.a \
  dyncast.o class_type_info.o si_class_type_info.o pointer_type_info.o \
  pbase_type_info.o tinfo.o fundamental_type_info.o
$ ar crs dyncast.a \
  dyncast.o class_type_info.o si_class_type_info.o pointer_type_info.o \
  pbase_type_info.o tinfo.o fundamental_type_info.o

How to boot GRUB from SysLinux

2010-12-19T13:35:00.001+01:00

This blog post explains how to boot GRUB from SysLinux. Only GRUB1 is covered, the solution explained doesn't support GRUB2.

Older versions of SysLinux (such as 3.83) don't support booting GRUB, i.e. they cannot load and boot the stage2 file of GRUB. The newest version of SysLinux contains chain.c32, which can boot many operating systems and bootloaders, including GRUB. The solution explained here doesn't use this feature, so it works with old versions of SysLinux as well.

The fundamental idea is to convert the GRUB stage2 file to a format which SysLinux can boot directly. This format is bzimage, the big variant of the Linux kernel image. grub.exe, part of GRUB4DOS is already in this format, so adding the following lines to syslinux.cfg works:

label grub4dos
menu label GRUB4DOS
kernel grub.exe

However, one might wish to use the original GRUB instead, because GRUB4DOS has some features missing, e.g. it doesn't support GPT (GUID Partition Table) and UUID on reisrefs partitions. Converting the original GRUB stage2 file to bzimage format is easy: just append it to the first 20480 bytes of grub.exe. SysLinux can boot this hybrid, but then GRUB wouldn't be able to find its configfile (menu.lst). To fix that, the full pathname of the configuration file has to be embedded to the boot image. The Perl script grub2bzimage.pl automates this.

grub2bzimage is a Perl script which converts the GRUB bootloader code(stage2) to bzimage format, which can be booted directly by SysLinux andother bootloaders. grub2bzimage.pl doesn't support GRUB2.

Example use to create grubzimg:

$ perl ./grub2bzimage.pl stage2 grubzimg '(hd0,0)/boot/grub/menu.lst'

Write this into syslinux.cfg:

label grub
menu label GRUB
kernel grubzimg

Please note that it's not possible to specify the name of the configfile(menu.lst) in syslinux.cfg (using append). The configfile has to be specified when grubzimg is created.

grub2bzimage.pl has been tested with SysLinux 3.83. Please note that newerversions of SysLinux contain chain.c32 which supports loading GRUB stage2 files directly.

How to get multiple clickable desktop notifications on Ubuntu Lucid

2010-12-14T16:02:00.000+01:00

This blog post explains how to get multiple clickable desktop notifications (i.e. those that were present on Ubuntu Hardy) on the default GNOME desktop of Ubuntu Lucid.

Desktop notifications are short messages displayed by programs for a short time in one of the corners of the screen. In Ubuntu Lucid, by default, they are displayed in the top right corner, their background is black, they are not clickable (i.e. it's not possible to click them away), they disappear while the mouse is over them, and at least 1 of them is visible at the same time. In Ubuntu Hardy, they are displayed in the bottom right corner of the screen, their background is similar to light grey, they are clickable (i.e. they disappear for good if the user clicks on them), they don't disappear while the mouse is over them, and if more than one of them can be on screen without overlap.

Ubuntu Lucid uses the notify-osd backend for displaying notifications. It's not possible to configure the notify-osd backend to make it work like Ubuntu Hardy. However, it's possible to install notification-daemon, which was the default backend in Ubuntu Hardy to for displaying notifications. Here is how to make it work in Ubuntu Lucid:

$ sudo apt-get install notification-daemon
$ sudo perl -pi -e 's@^Exec=.*@Exec=/usr/lib/notification-daemon/notification-daemon@' /usr/share/dbus-1/services/org.freedesktop.Notifications.service
$ sudo killall notify-osd

Try it with:

$ notify-send foo; notify-send bar

Optional change to disable notify-osd completely: (It may screw up volume notifications etc., so don't use it unless it doesn't work without it.)

$ sudo rm -f /usr/share/dbus-1/services/org.freedesktop.Notifications.service.*

See this discussion with some links for other notify-osd improvements and alternatives.

A dramatic colored picture of a tiger's head

2010-12-11T18:14:00.002+01:00

This blog post documents my unsuccessful software archeology attempt to find the origin and the author of the famous PostScript tiger colorful vector graphics. Get the (mostly unchanged) EPS file from the Ghostscript SVN repository, here.

The earliest Ghostscript version containing the tiger I could dig up is version 2.6.1 (5/28/93) [download] in Slackware Linux 1.1.2. FYI The earliest Ghostscript version in Debian is 2.6.1 as well [download].

The image itself contains the comment %%CreationDate: 4/12/90 3:20 AM, so the earliest possible Ghostscript version that can contain it is 2.0 (released on 9/12/90) — but I wasn't able to find a download link for that Ghostscript. The author and the copyright is not indicated. The only description is found in the Ghostscript history.doc file, saying tiger.ps - A dramatic colored picture of a tiger's head.

Even Wikipedia doesn't specify the origin of the tiger graphics. All I could find is this question asking where it comes from.

Here is the relevant part of the EPS header in the tiger.ps and tiger.eps file:

%%Creator: Adobe Illustrator(TM) 1.2d4
%%For: OpenWindows Version 2
%%Title: tiger.eps
%%CreationDate: 4/12/90 3:20 AM
%%DocumentProcSets: Adobe_Illustrator_1.2d1 0 0
%%DocumentSuppliedProcSets: Adobe_Illustrator_1.2d1 0 0

What I've learned: Linux distributions (especially Debian and Slackware) are very useful sources of the source code of ancient versions of some free software.

The origin of the tiger thus remains unsolved.

It is a misconception that C++ is a superset of C

2010-12-07T01:59:00.001+01:00

This blog post shows (with a counterexample) that C++ is not a superset of C, i.e. there is a valid C program which is not a valid C++ program.

It's easy to find a counterexample: just use a C++ keyword as a variable name in C. The program is int class;. The proof:

$ gcc -v
gcc version 4.4.1 (Ubuntu 4.4.1-4ubuntu9)
$ g++ -v
gcc version 4.4.1 (Ubuntu 4.4.1-4ubuntu9)
$ echo 'int class;' | tee t.c t.cc >/dev/null
$ gcc -c -pedantic -ansi -W -Wall t.c
$ g++ -c t.cc
t.cc:1: error: expected identifier before ‘;’ token
t.cc:1: error: multiple types in one declaration
t.cc:1: error: declaration does not declare anything

There is a counterexample which doesn't contain any C++ keywords. The program is int i=&i;. The code:

$ echo 'int i=&i;' | tee t.c t.cc >/dev/null
$ gcc -c -pedantic -ansi -W -Wall t.c
t.c:1: warning: initialization makes integer from pointer without a cast
$ g++ -c t.cc
t.cc:1: error: invalid conversion from ‘int*’ to ‘int’

There is a counterexample which doesn't contain any C++ keywords, and it compiles without a warning as C code. The program is char *p=(void*)0;

$ echo 'char *p=(void*)0;' | tee t.c t.cc >/dev/null
$ gcc -c -pedantic -ansi -W -Wall t.c
$ gcc -c -pedantic -std=c89 -W -Wall t.c
$ gcc -c -pedantic -std=c99 -W -Wall t.c
$ gcc -c -pedantic -std=c9x -W -Wall t.c
$ gcc -c -pedantic -std=gnu89 -W -Wall t.c
$ gcc -c -pedantic -std=gnu99 -W -Wall t.c
$ gcc -c -pedantic -std=gnu9x -W -Wall t.c
$ tcc -W -Wall -c t.c
$ 8c -c -w t.c
$ g++ -c t.cc
t.cc:1: error: invalid conversion from ‘void*’ to ‘char*’

As a bonus: there is a program source which compiles in both C and C++, but it does something different. The following example is based on the fact that sizeof('x') is 1 in C++, but it's at least 2 in C.

#include <stdio.h>
int main(){return!printf("Hello, C%s!\n", "++"+1%sizeof('x')*2);}

Another idea to make the same program source do something else in C than C++ is to take advantage of that union t (etc.) create a type named t in C++, but not in C.

#include <stdio.h>
char t; int main(){union t{int u;};
return!printf("Hello, C%s!\n", "++"+(sizeof(t)<2)*2);}

So the true statement is: C and C++ are similar languages, with a large common subset, and C++ being much larger to the common subset than C is.

On browser compatibility issues

2010-12-05T22:36:00.000+01:00

This blog post lists the browser compatibility issues (with solutions) I've encountered when creating a simple HTML(4) + JavaScript + CSS web page containing presentation slides with a little bit of user interaction.

My goal was to make the web page compatible with Google Chrome 8.0 or later, Firefox 3.6 or later, Safari in Leopard or later, Opera 10.63 or later, Konqueror 4.4.2 or later, Internet Explorer 8.0 or later. (I'll call them main browsers from now on.) Please note that anything stated in this blog post may not be true for earlier web browser versions. There was no plan to make the web page work in any earlier web browser version, but it turned out that it was possible and easy to make the page work with Internet Explorer 7.0 with some small rendering quality degradation, so I happened to add support for that as well.

My workflow:

Edit the web page directly in a text editor, as a single HTML file (containing HTML, JavaScript and CSS). Split it to different files only after the prototype is found working and all browser compatibility issues have been registered (and most of them resolved).
Make sure the browser renders the page in standards compliant mode, not in quirks mode (see below how and why).
View the local HTML file in Google Chrome, reload the page after each file save. Try it after major modifications in Firefox. As soon as the web page works in these two browsers, fix it for other browsers as well.
When porting to other browsers, especially Internet Explorer, search in the jQuery source code for the feature (class or method name) that works in one browser. Most probably they jQuery source will contain its alternative implementations for other browsers.
Use the Developer Tools feature of Google Chrome (activate it with Ctrl-Shift-J) to debug the DOM, the CSS and the JavaScript on the current page. With Firefox, use the Firebug extension and/or the shell bookmarklet for debugging. Should you need it, use the Tools / Developer Tools (activate it with F12; it is available as an extension called Developer Toolbar for IE7 and earlier) in Internet Explorer 8 for debugging.
With Internet Explorer Developer Tools you can verify that the web page is indeed rendered in standards compliant mode, and you can also force a rendering which Internet Explorer 7 would do.
After each major change, validate the web page that it is valid HTML 4.01 transitional. You can use the HTML Validator Google Chrome extension.

My general knowledge and advice on cross-browser compatibility:

All main browsers (Google Chrome, Firefox, Safari, Opera and Konqueror) are very similar to each other except for Internet Explorer.
Google Chrome, Safari and Konqueror are even more similar to each other, because they share the same original code base (KHTML in Konqueror, which WebKit in Safari is based on, which WebKit in GoogleChrome is based on).
Rendering (application of CSS) in most main browsers in much more similar to each other when they render the web page in standards compliant mode rather than in quirks mode. So to get rid of many incompatibilities, just make sure that the browser renders the page in standards compliant mode.
To enable standards compliant mode, just start your HTML with a doctype (right at the beginning of the file). To get HTML 4.01 transitional, prepend this to your HTML file:
```
<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN"
    "http://www.w3.org/TR/html4/loose.dtd">
```
Make sure your editor saves the HTML file in UTF-8 encoding (character set), without the byte order mark (BOM).
Make sure the character set is indicatied the <head>. This is needed and used when the web page is loaded using a file:/// URL or imported to a word processor such as OpenOffice Writer or LibreOffice Writer. Example:
```
<meta http-equiv="content-type" content="text/html; charset=utf-8">
```
Make sure your webserver returns the proper character set for the web page in the HTTP response headers (e.g. Content-Type: text/html; charset=utf-8). This setting is used for http:// and https:// URLs. The HTTP response header takes precedence over <meta http-equiv=. If you use the Apache webserver, you may be able to force the UTF-8 charset by creating a file named .htaccess (filename starting with a dot) next to the HTML file, containing
```
AddDefaultCharset UTF-8
```
. If it doesn't seem to take effect, then put an error (temporarily) to the .htaccess file (e.g. nosuchdirective 42), do a Shift-reload on your web page. If that causes an Internal Server Error, then .htaccess is indeed honored by Apache. If you don't get such an error, talk to your webmaster or system administrator (i.e. copy-paste this paragraph to an e-mail in addition to the URL).
The corresponding SVN command (for Google Code) for the character set is:
```
svn propset 'svn:mime-type' 'text/html; charset=utf-8' webpage.html
```
Use jQuery, and make it take care of most browser compatibility issues. This document assumes, however, that no JavaScript framework is used, so all issues have to be resolved manually.

Some specific cross-browser compatibility issues I have encountered:

See a cross-browser comparison of all JavaScript browser events here.
Internet Explorer has a different API for installing and handling events.
- Event propagation and bubbling semantics are different. You can avoid these issues by installing all event handlers for document.body and window. (Sorry, I don't know more about this.)
- For Internet Explorer, change obj.addEventListener(eventName, handler, false) to obj.attachEvent('on' + eventName, handler).
- For Internet Explorer, change obj.removeEventListener(eventName, handler, false) to obj.detachEvent('on' + eventName, handler).
- Internet Explorer supports the DOMContentLoaded event under a different name (onreadystatechange) and different semantics, see below.
- Internet Explorer needs event.preventDefault() to be specified differently. Here is the cross-browser solution:
```
event.preventDefault ? event.preventDefault() : (event.returnValue = false)
```
- Internet Explorer needs event.stopPropagation() to be specified differently. Here is the cross-browser solution:
```
event.stopPropagation ? event.stopPropagation() : (event.cancelBubble = true)
```
- Internet Explorer doesn't pass the event object to the event handler function. To make it cross browser, write it like this:
```
function handler(event) { if (!event) event = window.event; ... }
```

Internet Explorer doesn't have window.getComputedStyle, so the current (computed) CSS properties have to be queried differently. An example, specific cross-browser solution:

function getBackgroundColor(element) {
  return element.currentStyle ? element.currentStyle.backgroundColor :
         window.getComputedStyle(element, null).
         getPropertyValue('background-color')
}

Internet Explorer 8 doesn't have elementOrDocument.getElementsByClassName, so it has to be emulated using a loop and e.g. elementOrDocument.getElementsByTagName and some manual checking. The emulation is a bit slower.
The string.substr method in Internet Explorer doesn't accept negative values in its first argument, so e.g. myString.substr(-2) has to be replaced by myString.substr(myString.length - 2).
The DOMContentLoaded event is fired as soon as the main HTML and the JavaScript and CSS files it references have finished loading — but before images and other objects on the page have finished. By that time, the rendering is done and measurements are made, so e.g. document.body.offsetWidth is valid. In contrast, the onload even fires as soon as images etc. are also finished loading Here is how to handle DOMContentLoaded in a cross-browser way:
```
function onDomReady() {
  ...
}
if (navigator.userAgent.indexOf(' KHTML/') >= 0) {
  document.addEventListener('load', onDomReady, false) 
} else if (document.addEventListener) {
  document.addEventListener('DOMContentLoaded', onDomReady, false)
} else if (document.attachEvent) {
  document.attachEvent('onreadystatechange', function() {
    if (document.readyState == 'complete') {
      document.detachEvent('onreadystatechange', onReadyStateChange)
      onDomReady()
    }
  })
}
```
Here, the load event is used instead of DOMContentLoaded, because Konqueror 4.4.2 doesn't compute element dimensions (e.g. document.body.offsetWidth and CSS properties) at DOMContentLoaded time; but it returns bogus default values. For Internet Explorer, onreadystatechange has to be used instead.
There are two keyboard events when a key is pressed or auto-repeated: keydown and keypress.
- Use keypress for Google Chrome and Safari (i.e. navigator.userAgent.indexOf(' AppleWebKit/') >= 0), because they don't send the arrow key (and any other non-letter) events for keypress.
- Use keydown for Firefox (for 3.6), because it doesn't send auto-repeats for keydown (see more here).
- Use onkeydown on Internet Explorer.
- Some browsers have a meaningless event.keyCode (such as 0 for Firefox 3.6 or the same as the event.charCode for Konqueror 4.4.2) for some keys. In this case, use the event.charCode instead.
- The event.charCode depends on whether Shift is down (e.g. 65 for A and 97 for Shift-A), but it doesn't depend on whether Ctrl or other modifiers are down. event.charCode is usually a Unicode code point (character code).
- See event.keyCode constants here.
- See more here about keyboard event compatibility.
Many browsers (such as Firefox 3.6) send multiple resize events (as part of an animation) when the window is maximized.
To get the vertical scroll position, use document.body.scrollTop || document.documentElement.scrollTop, because Konqueror always returns 0 for just document.body.scrollTop. Do this respectively for the horizontal scroll position (scrollLeft).
To scroll the page in a cross-browser way (which works in Konqueror as well), use window.scroll(newScrollLeft, newScrollTop) .
Internet Explorer doesn't support window.innerWidth, use window.innerWidth || document.documentElement.clientWidth for cross-browser compatibility. Do the same with window.innerHeight and document.documentElement.clientHeight as well.
Internet Explorer 7 doesn't support an extra comma right before ] and } in array and object constructors.

Announcing uevalrun: self-contained computation sandbox for Linux

2010-11-28T23:47:00.000+01:00

This blog post is to announce uevalrun.

uevalrun is a self-contained computation sandbox for Linux, usingUser-mode Linux for both compilation and execution of the program to besandboxed. The program can be written in C, C++, Python, Ruby, Perl or PHP.uevanrun enforces memory limits, timeouts and output size limits in thesandbox. The primary use case for uevalrun is evaluation of solutionprograms submitted by contestants of programming contests: uevalrun compilesthe solution, runs it with the test input, compares its output againstthe expected output, and writes a status report.

For your convenience, here is a (non-updated) copy of the documenation of uevalrun. See project home page for the most up-to-date version.

Installation

A 32-bit (x86, i386) or 64-bit (x86_64, amd64) Linux system is requiredwith enough RAM for both compilation and execution, plus 6 MB memoryoverhead. The Linux distribution doesn't matter, because uevalrun usesstatically linked binaries, and it's self-contained: it contains all thetools it needs, including the C and C++ compilers (from the uClibc gcc-4.1.2toolchain).

uevalrun doesn't need root privileges: it runs as a simple user.

uevalrun needs about 160MB of disk space, most of which is consumed bythe C compiler (31MB extracted + 17MB compressed), the scripting languageinterpreters (9MB compressed), and the virtual disk images (77MB,uncompressed). After removing all temporary files, 80MB will be enough.

Download and compile:

$ svn co http://pts-mini-gpl.googlecode.com/svn/trunk/uevalrun uevalrun
$ cd uevalrun
$ ./make  # This downloads some more files during compilation.

Try it (the italic parts are displayed by the program):

$ (echo '#! perl'; echo 'print "Hello"x2, "\n"') >foo.pl
$ echo HelloHello >expected.txt
$ ./uevalrun -M 10 -T 9 -E 9 -s foo.pl -t /dev/null -e expected.txt
...
@ result: pass
$ echo '/**/ int main() { return!printf("HelloHello\n"); }' >foo.c
$ ./uevalrun -M 10 -T 9 -E 9 -U 19 -N 32 -C 9 \
  -s foo.c -t /dev/null -e expected.txt
...
@ result: pass

How to use

This section has not been written yet. If you have questions, don't hesitate to ask!

Requirements

Security is the most important requirement of uevalrun, followed by highperformance and usability.

The sandboxing must be secure, more specifically:

Programs inside the sandbox must not be able to communicate with the outside world, except for their stdin and stdout. So they don't have access to the filesystem (except for some read-only access to some whitelisted system libraries and binaries), and they can't do network I/O. If uevalrun is used for programming contest submission evaluation, this restriction prevents the program from finding and reading the file containing the expected output.
Sandboxed programs must not be able to use more system resources (memory, disk, CPU) than what was allocated for them.
Sandboxed programs must not be running for a longer period of time than what was allocated.
Even the compilation of programs to be run (executed) inside the sandbox must be sandboxed (possibly with different system resource allocation), to prevent the attacker from submitting a program source code which exploits a bug in the compiler.
The sandbox must be reverted to its initial state before each compilation and execution, so sandboxed programs won't be able to gather information about previous compilations and executions. For programming contests, this restriction prevents the program from reading other contestants' submissions or their output.
Timeouts (both real time and user time) must be enforced outside the sandbox, so the program will be reliably killed if it runs for too long time, even if it tampers with time measurement inside the sandbox.
Sanboxed programs must not be able to lie about their success, their performance or the correctness of their output by writing special marker characters to their stdout or stderr.

The sandbox must be fast and waste few resources, more specifically:

Reverting the sandbox to its initial, empty state must be very fast (preferably faster than 1 second).
Starting a program inside the sandbox must be very fast (preferably faster than 1 second).
Running a program inside the sandbox must not be much slower than running the same program outside the sandbox. If the program is CPU-intensive, it may run 1.1 times slower (i.e. 10% slower) inside than outside. If the program is I/O-intensive, it may run 10 times slower inside than outside. This requirement is intentionally quite loose on I/O virtualization performance.
Running a program inside the sandbox must not need more than 10MB more memory than running the same program outside.
Running a program inside the sandbox must not need any disk space, except for the disk space needed by the program binary, the test input and the expected output, all stored as files outside the sandbox.
Compilation inside the sandbox must require only a reasonable amount of temporary disk space (for the file to be compiled, the temporary files, and the output binary).

The sandbox must be easy to use and versatile, more specifically:

Sandboxed programs must be able to accept any 8-bit binary input (stdin).
Sandboxed programs must be able to write any 8-bit binary output to their stdout.
Multiple sandboxed programs must be able to run at the same time on the same host system, without affecting each other.
If a sandboxed program fails (e.g. because it writes to its stdout different from what was expected, or it does a memory access violation, it reaches a timeout, it exceeds its allocated memory etc.), the proper failure reason has to be reported (e.g. ``wrong answer'' must not be reported if the program times out or vice versa).
The sandboxing software must have as little few system dependencies as possible. It must be able to run in a restricted (e.g. chroot) environment, it must not depend on system libraries or configuration. It must work on 32-bit (x86, i386) and 64-bit (x86_64, amd64) Linux systems, on any Linux distirubution.
Sandboxed programs can be written in C, C++, Python, Ruby, Perl and PHP, or in any language whose compiler can produce a 32-bit (i386) Linux statically linked executable binary.

Design

To fullfill the requirements, the following fundamental building blocksare used.

User-mode Linux(UML) is used for virutalization: both compilation and execution isperformed in a UML guest, reading data from the host system using virtualblock devices (ubd), and writing its output to its/dev/tty1 (con0), which can be read by a processon the host. A UML guest kernel (currently 2.6.31.1) tailered to thesandboxing requirements (security and performance) is configured andcompiled. Some kernel patches are applied to increase security, reliabilityand performance. (Please note that these patches apply to the UML guestkernel only, so the host remains unpatched, and rebooting is not neededeither.) Networking and the virtual file system are disabled in the UMLguest kernel for increased security. All unnecessary drivers and featuresare removed from the UML guest kernel to get fast boot times and to reducethe overhead.

The underlying virtualization technology used by UML isptrace(), which doesn't needroot privileges or a kernel module or kernel modifications in the host. Asan alternative to UML, Seccomp could be used forsendboxing, but that's quite cumbersome, because the sandboxed processcannot allocate memory for itself (see the Google Chrome Seccompsandbox for a generic solution), but it's still prohibitively cumbersometo sandbox GCC this way (with its requirements for forking and creatingtemporary files). Most sandboxing approaches on Linux require rootprivileges for a short amount of time (for chroot, PID namespaces (seeclone(2)) and othernamespaces). Most virtualization approaches (like KVM, Xen, VirtualBox, except possibly forQEMU) need loading of kernel modules orpatching the kernel, and support only slow guest boot times.

With UML, it's possibly to boot the UML guest, run a hello-world binary,and halt the UML guest in less than 0.02 second (sustained). For that speed,one needs a (guest) kernel patch (included and automatically applied inuevalrun) which skips the Calibrating delay loop kernel startup step,which usually takes 0.33 second. The memory overhead of UML is also quitelow (6 MB with a stripped-down kernel).

All software running inside and outside UML is written in C (for highperformance) and compiled with uClibcand linked statically to avoid depending on the libc on the host, and toavoid the need for installing a libc in the guest. (The total size of thesecustom-built binaries is less than 100kB.) There is no Linux distributioninstalled the guest – only a few custom binaries are copied, like acustom /sbin/init, which mounts /proc, sets upfile descriptors and resource limits, and starts the sandboxed program, andthen halts the guest. The /lib directory in the guest is empty,except for compilation, where /lib containslibc.a, libstdc++.a etc.

BusyBox (statically linked withuClibc) is used as a Swiss Army Knife tool for scripting and automation (ofthe build and the setup process), both inside and outside UML. Please note,however, that once all the binaries are built and the filesystem images arecreated, BusyBox is not used anymore for running sandboxed programs andevaluating their output (except for temporary filesystem creation) becauseof performance reasons, but work is done by the small and fast custom Ctools just compiled.

Sandboxed program sources can be written in C (using a large subset ofuClibc as the system library), C++(using uClibc andlibstdc++), Python (usingall the built-in Python modules), Ruby (using just the built-in Rubymodules and classes which are implemented in C), Perl (using only thefew dozen most common modules), or PHP (using just the built-in PHPfunctions which are implemented in C).

For C and C++ compilation, a GCC 4.1.2toolchain is used which is based onuClibc and produces statically linkedexecutables.

Interpreters for Python, Ruby, Perl and PHP are provided as precompiled,self-contained, single, statically linked, 32-bit (i386) Linux executables.It's easy to copy them to the /bin directory of the UML guest.

All tools used for building the software are either shipped with thesoftware, or the software downloads them during compilation. All binarytools are statically linked, 32-bit (i386) Linux executables. This providesmaximum host system compatibility and reproducible behavior across systems.

In our speed measurements, CPU-intensive programs running inside UML run1% slower than outside UML, but I/O-intensive programs (such as C++compilation with GCC) can be about 6 times slower.

The Minix filesystemis used in the UML guests for both read-only filesystems (e.g. theroot filesystem with the scripting language interpreters) and read-writefilesystems (such as /tmp for the temporary assembly filescreated by GCC). The Minix filesystem was chosen instead of ext2 and otherusual Linux filesystem, because it has less size overhead, it's codebase issmaller, and it has equally good performance for the small and mostlyread-only filesystems it is used for.

License

uevanrun has been released under the GNU GPL v2.

Bugs? Problems? Contact the author. Send feedback!

Please add your feedback to the issue tracker.All feedbacks are welcome.

How to download Skype without having to register

2010-11-19T11:06:00.001+01:00

Recently Skype has introduced the requirement of registration or logging in on its web page before downloading the Windows version of their Skype software. This blog post explains how to download Skype without registration or logging in.

Follow the link relevant to your operating system:

Remote desktop sharing for Linux without software installation and firewall configuration

2010-11-15T16:52:00.002+01:00

This blog post explains how to share two desktops (screen, keyboard and mouse) without software installation (on Linux) and firewall configuration. The instructions given have been verified on Ubuntu Lucid running on 32-bit (i386) and 64-bit (amd64) architecture -- but they should work on any modern Linux desktop.

TeamViewer is an excellent cross-platform application which provides desktop sharing, file transfer, chat and video conferencing, and it's free for personal use. Its use is straightforward. Installers for Linux, Mac OS X, Windows and iPhone are downloadable here.

If you don't want to install anything, you can try the QuickSupport edition of TeamViewer on Windows and the Mac OS X. (Get it from the same download page.) The QuickSupport edition doesn't let the user initiate a connection (to become a client), but it listens as a server waiting for connections.

As of now, there is no official Linux QuickSupport edition of TeamViewer, so I've created one; it downlads and starts the regular TeamViewer for Linux application, without installing it. To use it on a vanilla Ubuntu Lucid box, just press Alt-, and type (or copy-paste) the following command, and press Enter:

bash -c 'wget -O- goo.gl/k9imm | bash'

Please note that the O between the dashes is a capital O. All other characters are lowercase.

Remote desktop sharing for Linux without software installation and firewall configuration

2010-11-15T16:52:00.000+01:00

bash -c 'wget -O- goo.gl/k9imm | bash'

Please note that the O between the dashes is a capital O. All other characters are lowercase.

How to disable PulseAudio on Ubuntu Lucid and Oneiric without uninstalling it

2010-11-10T12:55:00.002+01:00

This blog post explains how to disable PulseAudio on Ubuntu Lucid without uninstalling it. The solution described here may work on other Linux distributions as well. Please note that you may lose volume controls integrated to GNOME (e.g. the volume control hotkeys), and you'll have to adjust the volume with alsamixer or it's GNOME equivalent, gnome-alsamixer.

Install the volume control programs:

$ sudo apt-get install alsa-utils gnome-alsamixer
$ sudo apt-get remove gamix  # Applications / Sound & Video / ALSA Mixer

Stop the PulseAudio server (pulseaudio) and remove temporary files:

$ pkill -STOP '^pulseaudio$'
$ rm -rf ~/.pulse

Make sure all application attempts to connect to the PulseAudio server and/or to start a new PulseAudio server will fail:

$ mkdir -p ~/.pulse
$ (if grep -q \=10 /etc/lsb-release; then
     echo 'default-server = 0.0.0.1'
   else
     echo 'daemon-binary = /dev/null/no-daemon'
   fi
   echo 'autospawn = no') >~/.pulse/client.conf

Kill the PulseAudio server:

$ pkill -KILL '^pulseaudio$'

Optionally, to disable PulseAudio for all users, add the following lines to /etc/pulse/client.conf for Ubuntu Lucid:

default-server = 0.0.0.1
autospawn = no

The corresponding lines for Ubuntu Oneiric are:

daemon-binary = /dev/null/no-daemon
autospawn = no

Restart your web browsers (for the Flash Player) and your Skype, and possibly other applications which use sound. To do so, it's best to log out and log in again.

How to get rid of the GNOME panel menu delay on Ubuntu Lucid

2010-11-03T18:30:00.000+01:00

This blog post gives instructions how to get rid of the GNOME panel menu delay, i.e. the small delay one experiences when moving the mouse pointer over the System menu, and the submenus like Preferences and Administration don't open immediately. The solution shown has been tested on Ubuntu Lucid, but it might work on other systems as well.

Run the following command to configure newly started applications to omit the delay:

$ echo "gtk-menu-popup-delay = 0" >> ~/.gtkrc-2.0

Run this command to restart the GNOME panel currently running:

$ killall gnome-panel

If you are interested speeding up the loading of icons, see more information on http://www.ubuntugeek.com/how-to-make-gnome-menus-faster-in-ubuntu.html.

pysshsftp: proof-of-concept SFTP client for Unix which uses the OpenSSH ssh(1) command-line tool

2010-10-30T22:23:00.001+02:00

This blog post is an announcement for pysshsftp, a proof-of-concept, educational SFTP client for Unix, which usesthe OpenSSH ssh(1) command-line tool (for establishing the secureconnection), but it doesn't use the sftp(1) command-line tool (so it cane.g. upload files without truncating them first).

Only very little part of the SFTP protocol has been implemented so far(initialization, uploading, and stat()ting files). The SFTP protocol wasreverse-engineered from sftp-client.c in the OpenSSH 5.1 source code.

The motivation behind writing pysshsftp was to have an SFTP client which

supports uploading files without truncating them (the OpenSSH sftp(1) always truncates the file before uplading data bytes);
can be easily scripted from Python;
uses the OpenSSH(1) command-line tool for establishing the secure connection (other Python libraries like pysftp and paramiko can't use the user's public key configuration properly by default: they don't support passphrase reading for passphrase-protected keys, they don't support reading keys from ssh-agent, and they don't support reading ~/.ssh/id_rsa and ~/.ssh/id_dsa exactly the same way as OpenSSH uses them).

Multilingual programs in Haskell, Prolog, Perl and Python

2010-10-29T13:11:00.002+02:00

This blog posts shows program source codes that work in multiple languages (including Haskell, Prolog, Perl, Perl and Python.

Here is a program which works in Haskell, Prolog and Perl:

$ cat >m1.prg <<'END'
foo( {-1/*_} ) if!'*/%'; print "Hello, Perl!\n" if<<'-- */'
}). :- write('Hello, Prolog!'), nl, halt.
/* -} x) = x
main = putStrLn "Hello, Haskell!"
-- */
END
$ perl m1.prg
Hello, Perl!
$ swipl -f m1.prg
Hello, Prolog!
$ cp m1.prg m1.hs
$ ghc m1.hs -o m1
$ ./m1
Hello, Haskell!

Here is a program which works in Haskell, Prolog and Python:

$ cat >m2.prg <<'END'
len( {-1%2:3} ); print "Hello, Python!"; """
}). :- write('Hello, Prolog!'), nl, halt.
/* -} x) = x
main = putStrLn "Hello, Haskell!"
-- """ # */
END
$ python m2.prg
Hello, Python!
$ swipl -f m2.prg
Hello, Prolog!
$ cp m2.prg m2.hs
$ ghc m2.hs -o m2
$ ./m2
Hello, Haskell!

How to upload a video to Picasa from Linux, Mac OS X and Windows

2010-10-26T21:38:00.002+02:00

This blog post explains how to upload a video to Google Picasa from Linux, Mac OS X and Windows.

It is not possible to upload a video to Picasa Web Albums from your web browser. (However, it is possible to upload images this way.)

Please note that the video you upload will be transcoded, most probably to lower quality than the original, and the original video file won't be available for download or watching on Picasa Web Albums.

On Windows, use the latest Picasa desktop application ([download]). Version 3.8 or newer works. Alternatively, you can use GoogleCL ([project page]).

On Mac OS X, use the Picasa Web Albums Uploader. ([download])

On Linux, use the Google command line tools (GoogleCL). [project page] Example command to upload photos and videos:

$ google picasa create --title album_name
$ google picasa post   --title album_name ~/Photos_and_videos/*

It might take a few minutes for the video to become transcoded and made available. In the meanwhile, you'll see this image on picasaweb instead:

On Linux, you may have to install the newest googlecl (0.9.11 works) and the newest python-gdata (2.0.12 or newer works), and remove your

~/.googlecl

directory to make it work.

Installation instructions for Ubuntu Lucid, before you can run the

google

command above:

$ sudo apt-get install python-setuptools
$ sudo easy_install gdata
$ sudo easy_install http://googlecl.googlecode.com/files/googlecl-0.9.11.tar.gz
$ mv -f ~/.googlecl ~/.googlecl.old

Panasonic Lumix DMC-FS30 battery information

2010-09-23T14:11:00.001+02:00

This blog post contains information about some digital cameras and their corresponding batteries and battery chargers.

The Panasonic Lumix DMC-FS30, FMC-FS11, DMC-FS10 and DMC-FS9 digital camera comes with the battery pack model CGA-S/106C, which is a 3.6V, 740 mAh, 2.7Wh Li-ion battery. The corresponding battery charger is the Panasonic DE-A60 battery charger.

How to emulate a WebSocket client in JavaScript using the Java plugin

2010-08-29T22:30:00.002+02:00

This blog post shows a proof of concept implementation which can be extended to emulate a WebSocket client in JavaScript in browsers where WebSocket support is not available, but there is a working Java plugin installed.

The motivation behind this blog post is to design drop-in replacements for WebSocket for older browsers (e.g. Internet Explorer 9 and earlier, Opera 9 and earlier, Safari 4 and earlier, Firefox 3.7 and earlier), where WebSocket client support is not available, but some browser plugin (like Java, Flash or Silverlight) could be used to emulate it. This blog post is a proof-of-concept demonstrating how the Java plugin can be used for such an emulation.

The trick is to use the java symbol exported to JavaScript. Here is the relevant HTML page with JavaScript code, which implements a simple, proof-of-concept HTTP client in JavaScript using Java:

<html><head>
<title>Java WebSocket proof-of-concept test</title>
<script type="text/javascript">
function output(str) {
  var output = document.getElementById("output")
  var text = document.createTextNode(str)
  var br = document.createElement('br')
  output.insertBefore(br, null)  
  output.insertBefore(text, null)
}
function init() {
  output("find java")
  java.lang
  var hostItems = window.location.host.split(':')
  var hostName = hostItems[0]
  var port = 80
  if (hostItems.length > 1) {
    port = parseInt(hostItems[1], 10)
    if (port == 443 && window.location.protocol == 'https:')
      port = 80
  }
  output("connecting to host " + hostName + ", port " + port)
  var socket = new java.net.Socket(hostName, port)
  output("connected")
  var osw = new java.io.OutputStreamWriter(socket.getOutputStream(), "UTF-8")
  var isr = new java.io.InputStreamReader(socket.getInputStream(), "UTF-8")
  var bis = new java.io.BufferedReader(isr)   
  output("sending request (GET /answer.html)")  
  osw.write("GET /answer.html HTTP/1.0\r\n\r\n")
  osw.flush()
  output("reading response")
  // SUXX: This blocks the whole browser (Firefox) until the response is
  // ready.
  var line
  while (true) {
    line = bis.readLine()    
    if (line == null) break
    if (line == '' || line == '\r') break
    output("got header line: " + line)
  }
  output("done with header")
  if (line != null) {
    while (true) {
      line = bis.readLine()  
      if (line == null) break
      output("got body line: " + line)
    }
  }
  output("done with body")
  output("closing")
  isr.close()
  osw.close()
  socket.close()
  output("done")
}
</script>
</head><body onload="init()">
<div id="output" style="border:1px solid black;padding: 2px">Welcome!</div>
</body></html>

The code above could be easily upgraded to use the WebSocket protocol.

Please note that the implementation above has a show-stopper bug: it locks the whole browser UI (all functionality in all tabs, in Firefox 3.6) until the server returns the HTTP response. It might be possible to solve it by using java.nio.SocketChannel, but we haven't investigated that yet.

Please note that it is possible to do WebSocket emulation with Flash instead of Java, see web-socket-js for the client-side code (JavaScript and Flash) and web-socket-ruby for the server-side code.

Please note that we haven't investigated if it is possible to do WebSocket emulation with Silverlight instead of Java.

How to try Stackless Python on Linux without installing it?

2010-08-11T16:18:00.001+02:00

This blog post gives instructions on trying Stackless Python without installing it on Linux systems (32-bit and 64-bit).

Use the Stackless version of the StaticPython binary, which is a portable version of Stackless Python 2.7 for Linux systems. It is linked statically, with most standard Python modules and C extensions included in the executable binary, so no installation is required. Here is how to try it:

$ wget -O stackless2.7-static \
  http://pts-mini-gpl.googlecode.com/files/stackless2.7-static
$ chmod +x stackless2.7-static
$ ./stackless2.7-static
Python 2.7 Stackless 3.1b3 060516 (release27-maint, Aug 11 2010, 13:55:35) 
[GCC 4.1.2] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import stackless
>>>

See more information about features, suggested uses and limitations on the StaticPython home page.

StaticPython released

2010-08-11T15:13:00.001+02:00

This blog post is an announcement of the StaticPython software distribution and its first release.

What is StaticPython?

StaticPython is a statically linked version of the Python 2.x (currently 2.7) interpreter and its standard modules for 32-bit Linux systems (i686, i386). It is distributed as a single, statically linked 32-bit Linux executable binary, which contains the Python scripting engine, the interactive interpreter with command editing (readline), the Python debugger (pdb), most standard Python modules (including pure Python modules and C extensions), coroutine support using greenlet and multithreading support. The binary contains both the pure Python modules and the C extensions, so no additional .py or .so files are needed to run it. It also works in a chroot environment. The binary uses uClibc, so it supports username lookups and DNS lookups as well (without NSS).

Download and run

Download the StaticPython 2.7 executable binary for version 0.1.

Here is how to use it:

$ wget -O python2.7-static \
  http://pts-mini-gpl.googlecode.com/files/python2.7-static
$ chmod +x python2.7-static
$ ./python2.7-static
Python 2.7 (r27:82500, Aug 11 2010, 10:51:33) 
[GCC 4.1.2] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>>

For what purpose is StaticPython useful?

for running Python scripts in chroot and other restricted environments (e.g. for running untrusted standard Python code)
for running CGI scripts on some web hosting providers where Python is not preinstalled (this is slow)
for trying the newest Python and running scripts on a very old Linux system or on a system where package installation is not possible or cumbersome
for deploying and running Tornado, Eventlet or Twisted-based networking applications on a machine without a working Python installation (please note that the framework itself has also to be deployed)

When isn't StaticPython recommended?

if dependencies need Python package installation (e.g. distutils, setuptools, setup.py, Pyrex, Cython) -- distutils is not included, there is no site-packages directory, loading .so files is not supported
if dependencies are or need shared libraries (.so files) -- StaticPython doesn't support loading .so files, not even with the dl or ctypes modules
if startup and module import has to be fast -- since StaticPython stores .py files (no .pyc) in a ZIP file at the end of the binary
for GUI programming -- no GUI or graphics library is included, loading .so files is not supported

How to extend, customize or recompile StaticPython?

Compiling python2.7-static was a one-off manual process. Automating this process has not been implemented yet. Please contact the author if you need this.

Feature details

Features provided

command-line editing with the built-in readline module
almost all standard Python 2.7 modules built-in
almost all standard C extensions built-in: _bisect, _codecs, _codecs_cn, _codecs_hk, _codecs_iso2022, _codecs_jp, _codecs_kr, _codecs_tw, _collections, _csv, _curses, _elementtree, _functools, _heapq, _hotshot, _io, _json, _locale, _lsprof, _md5, _multibytecodec, _multiprocessing, _random, _sha, _sha256, _sha512, _socket, _sockobject, _sqlite3 (added 1004307 bytes to the executable binary size), _sre, _struct, _symtable, _weakref, array, audioop, binascii, bz2, cPickle, cStringIO, cmath, crypt, datetime, errno, fcntl, fpectl, future_builtins, gc, grp, imageop, itertools, math, mmap, operator, parser, posix, pwd, pyexpat, readline, resource, select, signal, spwd, strop, syslog, termios, thread, time, timing, zipimport, zlib
greenlet integrated for coroutine support
multithreading (using thread and threading as usual)
line editing even without a terminfo definition or inputrc file(useful in chroot, provided by libreadline/libncurses by default)
the usual help, license used in interactive mode

Executable binary layout

Python 2.7 (r27:82500) on Linux i386, statically linkedcompiled and linked with uClibc, so it supports username lookups and DNS lookups as well (without NSS).
pure Python and C extensions integrated to a single, statically linked, i386 (i686) Linux executable
compiled with uClibc so it can do DNS lookups without /lib/libss_*so*
can run from any directory, even in chroot containing only the binary, even in interactive mode, even without /proc mounted

Features missing

missing: loading .so files (C shared libraries, Python C extensions)
missing: the ctypes module and the dl module (since no .so file loading support)
missing: the distutils module, custom extension installation
missing: !OpenSSL, SSL sockets
missing: IPv6 support
missing: GUI bindings (tkinter, GTK, Qt etc.)
missing: Stackless Python

How to try the latest MariaDB on Linux

2010-08-09T22:21:00.008+02:00

This blog post explains how to download and start the bleeding edge MariaDB on a fairly recent 32-bit or 64-bit Linux system, for trial and development, the most straightforward way, without overwriting an existing MySQL installation or its data.

MariaDB is an improved, backward compatible, drop-in replacement of the MySQL Database Server, by Michael "Monty" Widenius, the original author of MySQL.

Please note that parts of this blog post are obsolete. See also the new blog post.

The simplest download and startup instructions for MariaDB 5.2.1 for 32-bit (i386, i686) Linux systems is the following:

# Stop any MySQL server currently running.
$ wget -O /tmp/mariadb-compact.tbz2 \
  http://pts-mini-gpl.googlecode.com/files/mariadb-compact-5.2.1-beta-linux-i386.tbz2
$ cd /tmp
$ tar xjvf mariadb-compact.tbz2
$ cd mariadb-compact
$ ./bin/mysqld --datadir=$PWD/data --pid-file=$PWD/mysqld.pid \
  --socket=$PWD/mysqld.sock --language=$PWD/share/mysql/english \
  --log-error=/proc/self/stderr
...

To stop the server, press Ctrl-Backslash, Enter in the window mysqld_safe is running, and wait 15 seconds for the process to exit.

To connect to the server, install the MySQL client. (e.g. with $ sudo apt-get install mysql-client on an Ubuntu system), and then run

$ mysql --socket=/tmp/mariadb-compact/mysqld.sock --user=root --database=test

$ mysql --host=127.0.0.1 --user=root --database=test

to connect. Please note that mariadb-compact comes with insecure default settings: it lets anyone connect as user root (on TCP 127.0.0.1:3306 (but not on other IP addresses) and on the Unix socket as well) without a password. Please use the appropriate mysqladmin commands (or modify the tables mysql.user, mysql.host and mysql.db directly) to set up access restrictions. use the --skip-networking flag of mysqld to prevent it from listening on TCP ports (not even 127.0.0.1:3306).

FYI Here is how mariadb-compact.tbz2 was created from the official MariaDB Linux binaries. The official download was located on http://askmonty.org/wiki/MariaDB:Download http://askmonty.org/wiki/MariaDB:Download:MariaDB_5.2.1-beta, and then the following commands were executed:

$ wget -O /tmp/mariadb-5.2.1-beta-Linux-i686.tar.gz \
  http://ftp.rediris.es/mirror/MariaDB/mariadb-5.2.1-beta/\
  kvm-bintar-hardy-x86/mariadb-5.2.1-beta-Linux-i686.tar.gz
$ mkdir -p /tmp/mariadb-preinst
$ cd /tmp/mariadb-preinst
$ tar xzvf /tmp/mariadb-5.2.1-beta-Linux-i686.tar.gz mariadb-5.2.1-beta-Linux-i686/\
  {bin/mysqld{,_safe},bin/my_print_defaults,scripts/mysql_install_db,\
  share/mysql/english/errmsg.sys,share/fill_help_tables.sql,\
  share/mysql_fix_privilege_tables.sql,share/mysql_system_tables.sql,\
  share/mysql_system_tables_data.sql,share/mysql_test_data_timezone.sql}
$ cd mariadb-5.2.1-beta-Linux-i686
$ strip -s bin/mysqld
$ ./scripts/mysql_install_db --basedir="$PWD" --force --datadir="$PWD"/data
$ rm -rf scripts
$ rm -f share/*.sql
$ cd ..
$ mv mariadb-5.2.1-beta-Linux-i686 mariadb-compact
$ tar cjvf /tmp/mariadb-compact.tbz2 mariadb-compact
$ rm -rf /tmp/mariadb-5.2.1-beta-Linux-i686.tar.gz
$ rm -rf /tmp/mariadb-preinst
$ ls -l /tmp/mariadb-compact.tbz2 
-rw-r--r-- 1 pts pts 4194393 Aug  8 18:10 /tmp/mariadb-compact.tbz2

The script mysql_install_db creates the initial databases and tables (e.g. the mysql.user table used for authentication) in the data directory.

All the other commands above just extract the absolute minimum necessary files from the official binary distribution, strip the binaries, and create a small .tbz2 archive.

Please note that the binaries in the official binary distributions are dynamically linked:

$ ldd bin/mysqld
        linux-gate.so.1 =>  (0xb7727000)
        libnsl.so.1 => /lib/tls/i686/cmov/libnsl.so.1 (0xb76f9000)
        librt.so.1 => /lib/tls/i686/cmov/librt.so.1 (0xb76f0000)
        libpthread.so.0 => /lib/tls/i686/cmov/libpthread.so.0 (0xb76d6000)
        libwrap.so.0 => /lib/libwrap.so.0 (0xb76cd000)
        libdl.so.2 => /lib/tls/i686/cmov/libdl.so.2 (0xb76c9000)
        libresolv.so.2 => /lib/tls/i686/cmov/libresolv.so.2 (0xb76b5000)
        libcrypt.so.1 => /lib/tls/i686/cmov/libcrypt.so.1 (0xb7683000)
        libstdc++.so.6 => /usr/lib/libstdc++.so.6 (0xb7590000)
        libm.so.6 => /lib/tls/i686/cmov/libm.so.6 (0xb756a000)
        libc.so.6 => /lib/tls/i686/cmov/libc.so.6 (0xb7425000)
        /lib/ld-linux.so.2 (0xb7728000)
        libgcc_s.so.1 => /lib/libgcc_s.so.1 (0xb7407000)

It would be straightforward to create the 64-bit version of mariadb-compact.tbz2, by starting from a different official binary distribution (http://ftp.rediris.es/mirror/MariaDB/mariadb-5.2.1-beta/kvm-bintar-hardy-amd64/mariadb-5.2.1-beta-Linux-x86_64.tar.gz), and modifying the the commands above appropriately.

A newer version (5.2.9) of this precompiled 32-bit Linux MariaDB is new available: http://pts-mini-gpl.googlecode.com/svn/trunk/portable-mariadb.release/portable-mariadb.tbz2. Here is the shell script which generated it from the official MariaDB 5.2.9 sources: http://code.google.com/p/pts-mini-gpl/source/browse/trunk/portable-mariadb/build-mariadb.sh. Here is the newer blog post: http://ptspts.blogspot.com/2011/11/announcing-portable-mariadb-small.html.

How to create a bootable Ubuntu Lucid USB pen drive with persistent storage

2010-07-29T21:05:00.000+02:00

This blog explains how to create a bootable Ubuntu Lucid Lynx (10.04) USB pen drive with persistent storage. Persistent storage means that all changes made to the system settings and all files in the user's home directory are saved to the pen drive and are available after system reboot.

Connect your USB pen drive.

The Ubuntu Lucid live system supports persistent storage by default. On an existing Ubuntu Lucid system System / Administration / Startup Disk Creator on an existing Ubuntu Lucid system. If you have an existing Ubuntu Karmic system, run sudo apt-get install usb-creator to install the Startup Disk Creator program, and then run System / Administration / USB Startup Disk Creator.

Make sure that the Stored in reserved extra space is selected (it's selected by default).

Download the live CD ISO image to your hard drive from http://releases.ubuntu.com/lucid/ubuntu-10.04-desktop-i386.iso. If you want to download from a faster mirror, please find the ISO image link on http://releases.ubuntu.com/lucid/. For maximum compatibility, select the Intel x86 (or i386, or 32-bit) version. The download may take 30 minutes or more.

In the startup disk creator window, specify the downloaded .iso file by clicking on the Other... button.

For maximum compatibility, format your pen drive, by clicking on the Format button in the startup disk creator window.

Click on the Make Startup Disk button.

Eject (remove) the USB pen drive in the file manager. Disconnect the pen drive.

Boot the computer from your USB stick as usual. There is now configuration necessary. All files you create in the live system in the user's home directory, and all settings you change get retained through reboots.

How to add (generate) locales on Debian and Ubuntu

2010-07-27T11:28:00.001+02:00

This blog post gives instructions to add (and generate) locales on Debian and Ubuntu systems. The instructions given here work on Debian Etch and newer, and Ubuntu Hardy and newer (including Intrepid, Jaunty and Karmic).

To add the locales hu_HU.ISO8859-2 and hu_HU.UTF-8, run

$ echo 'hu_HU.ISO8859-2 ISO-8859-2' | sudo tee -a /var/lib/locales/supported.d/hu
$ echo 'hu_HU.UTF-8 UTF-8' | sudo tee -a /var/lib/locales/supported.d/hu
$ sudo dpkg-reconfigure locales

To verify that the locales are installed correctly, run the following commands and verify that they don't print anything:

$ LC_ALL=hu_HU.ISO8859-2 perl -e0
$ LC_ALL=hu_HU.UTF-8 perl -e0

An approximation of pi in bc

2010-07-17T18:17:00.004+02:00

This blog post shows an approximation of the constant π (3.14...) implemented as a bc program using Machin's formula.

Machin's formula is documented on http://en.wikipedia.org/wiki/Numerical_approximations_of_%CF%80#Machin-like_formulae. The implementation below computes acot using its Taylor series expansion, always rounding down for divisions. The implementation knows the number of good digits by computing an upper bound of the total rounding error.

#! /usr/bin/bc -q
# by pts@fazekas.hu at Sat Jul 17 17:38:25 CEST 2010

/** Return an approximation and lower bound for acot(x) * 10 ^ u. */
define acot(x, u) {
  auto sum, xpower, xx, n, term
  sum = xpower = u / x
  xx = x * x
  n = 3
  for (;;) {
    xpower /= xx
    term = xpower / n
    if (term == 0)
      return sum
    sum -= term
    n += 2
    xpower /= xx
    term = xpower / n
    if (term == 0)
      return sum
    sum += term
    n += 2
  }
}

/* Return an integer upper bound for the >= 0 error value pi * 10 ^ u - f(u),
 * where u is 10 ^ nd, f(u) is the integer 4 * (4 * acot(5, u) - acot(239, u)).
 *
 * The magic constants in the maxerr implementation were derived from analyzing
 * the acot implementation, taking into account the rounding (truncation) done
 * in each division.
 */
define maxerr(nd) {
  return (286135312 * nd + 41739380) / 10000000
}

/* Return a string of at most nd characters, prefix of pi,
 * assuming nd >= 4.
 */
define pi(nd) {
  auto u, y
  u = 10 ^ nd
  y = 4 * (4 * acot(5, u) - acot(239, u))
  y /= 10 ^ (length(maxerr(nd)) - 1)
  while (y % 10 == 0) {
    y /= 10
  }
  return y / 10
}

/* Print pi with increasing precision infinitely (until aborted). */
define pinfinite() {
  auto b, nb, nd, a, na
  print "3."
  b = 3
  nb = 1
  nd = 8
  for (;;) {
    a = pi(nd)
    na = length(a)
    /* Print digits not printed yet in previous iterations. */
    print a % (10 ^ (na - nb))
    b = a
    nb = na
    nd *= 3
  }
}

pinfinite()  /* Infinite loop. */

Here is the equivalent program in dc:

[lulx/dspsslxd*sw3snlDx]sC[ls2Q]sS[lplw/dspln/dst0=Slslt-ssln2+snlplw/dsp
ln/dst0=Slslt+ssln2+snlDx]sD[ly10/dsy10%0=Q]sQ[10ld^sz5sxlzsulCx4*239sxlz
sulCx-4*10ld286135312*41739380+10000000/Z1-^/dsy10%0=Qly10/]sP[10lksdlPxd
sgZdsilj-^lgr%nlgshlisjlk3*sklIx]sI[3.]n3sh1sj8sklIx

How to enable bitmap fonts on Ubuntu Karmic and Ubuntu Lucid?

2010-06-27T12:58:00.010+02:00

This blog post gives instructions to enable bitmap fonts for GNOME and other Xft X11 applications. The instructions given have been tested on Ubuntu Karmic and Ubuntu Lucid, but they might work on other Unix systems as well with minor modifications. The reason for enabling bitmap fonts is to get crisp font rendering in dialog boxes, menus and window titles.

Run these commands to install some bitmap fonts from package:

$ sudo apt-get install xfonts-mplus xfonts-bitmap-mule xfonts-base
$ sudo apt-get install xfonts-75dpi{,-transcoded} xfonts-100dpi{,-transcoded}

Run the command to install some Microsoft vector fonts (e.g. Times New Roman, Arial, Courier New, Verdana). Installation might take a few minutes, because the font files have to be downloaded form an external site.

$ sudo apt-get install msttcorefonts

If you are using Ubuntu Karmic or earlier (not Ubuntu Lucid or later), then run these commands to install some QT configuration tools (needed for Skype):

$ sudo apt-get install qt3-qtconfig qt4-qtconfig

Run these commands to install some useful fonts (fixedsc is the classic 6x13 xterm monospaced bitmap font helpfully renamed to FixedSC, and helxetica is the Helvetica bitmap font helpfully renamed to Helxetica, for QT):

$ wget -q -O - http://pts-mini-gpl.googlecode.com/svn/trunk/fonts/helxetica.tgz |
       (cd / && sudo tar xzv)
$ wget -q -O - http://pts-mini-gpl.googlecode.com/svn/trunk/fonts/fixedsc.tgz |
       (cd / && sudo tar xzv)

Enable bitmap fonts (this works in Ubuntu Hardy, Ubuntu Intrepid, Ubuntu Jaunty as well as in Ubuntu Karmic and Ubuntu Lucid):

$ sudo rm -f /etc/fonts/conf.d/70-{yes,no,force}-bitmaps.conf
$ if test -f /etc/fonts/conf.avail/70-force-bitmaps.conf
  then sudo ln -s {../conf.avail,/etc/fonts/conf.d}/70-force-bitmaps.conf
  else sudo ln -s {../conf.avail,/etc/fonts/conf.d}/70-yes-bitmaps.conf
  fi

The if above is needed, because /etc/fonts/conf.avail/70-force-bitmaps.conf has been introduced and /etc/fonts/conf.avail/70-yes-bitmaps.conf was made almost empty in Ubuntu Lucid. The correct (non-empty) contents are:

<?xml version="1.0"?>
<!DOCTYPE fontconfig SYSTEM "fonts.dtd">
<fontconfig>
<!-- Accept bitmap fonts -->
 <selectfont>
  <acceptfont>
   <pattern>
     <patelt name="scalable"><bool>false</bool></patelt>
   </pattern>
  </acceptfont>
 </selectfont>
</fontconfig>

Run these commands to rebuild the font filename cache (so it will find all bitmap fonts, including the newly installed fonts):

$ sudo rm -f /var/cache/fontconfig/*
$ rm -rf "$HOME/.fontconfig"
$ sudo fc-cache
$ fc-cache
$ fc-list | grep -E 'Helxetica|FixedSC' | sort
FixedSC:style=Bold
FixedSC:style=Regular
Helxetica:style=Bold
Helxetica:style=Bold Oblique
Helxetica:style=Oblique
Helxetica:style=Regular

To have crisp fonts in dialog boxes, menus an window titles, then in System / Preferences / Appearance / Fonts, set the following font settings (on Ubuntu Lucid, use Helxetica instead of Helvetica):

Application font: Helvetica | 9
Document font: Arial | 10
Desktop font: Helvetica | 9
Window title font: Helvetica Bold | 9
Fixed width font: FixedSC | 10

To make the GNOME Panel reload its menu font, you have to restart it. Run

$ killall gnome-panel

In Applications / Accessories / Terminal / Edit / Profile preferences, make sure the Use the system fixed width font is ticked.

On Ubuntu Karmic and earlier, press Alt-F2, type qtconfig-qt3, press Enter, then wait for the Qt Configuration window to appear. In the tab Fonts, change the Family: to Helxetica (sic, it's not Helvetica), and the Point Size: to 9. In the menu, choose File / Save, then File / Exit.

On Ubuntu Karmic and earlier, repeat the previous paragraph with qtconfig-qt4 instead of qtconfig-qt3.

Notes:

There is no need to log out, restart Skype, restart Firefox restart Nautilus, restart the GNOME Terminal or restart the GNOME Panel. All these applications pick up the font changes immediately.
The Fixed (6x13) had to be renamed to FixedSC, because the original 6x13 font has the tag semicondensed, which makes it impossible to select in the GNOME font selection dialog (or in any fontconfig-based app).
The Helvetica bitmap font had to be renamed to Helxetica, because QT4 applications (such as Skype 2.1) cannot render non-latin-1 characters properly (they will find a substitution font for those characters, even though the characters are available in the Helvetica font).

Does Btrfs survive silent disk data corruption in RAID1 mode?

2010-06-20T14:18:00.000+02:00

Some of my experimenting with Btrfs in Linux 2.6.34 yielded the following results:

If only one disk (out of two) contains corrupt data, then Btrfs detects some checksum failures, and then recovers (overwriting the corrupt data with the corresponding good data from the other disk). This works even if the corruption happened while the filesystem was mounted.
If both disks contain corrupt data (but not at the same location), then Btrfs detects some checksum failures, then recovers some data, but it won't recover all of it: some files continue to have I/O errors when reading, and the syslog will contain checksum failures again and again. The explanation for this behavior may be that in Btrfs RAID 1 mode the two copies of a block of data might be at a different offset on the two disks.

Here is how I did the experiment:

I had a Linux 2.6.34 system.
I had 2 partitions of size 2000061 KB each. (1 KB == 1 << 10 bytes.)
# mkfs.btrfs -m raid1 -d raid1 /dev/sdc1 /dev/sdb1
# mount /dev/sdb1 /mnt/p
I copied /var/lib/dpkg (7248 small files of 36.93 MB) recursively to /mnt/p.
I copied 4 large files of 1517.98 MB in total to /mnt/p. (So the filesystem became >75% full.)
I created 10000 empty files.
I calculated the checksum of all files in /mnt/p with a userspace tool.
I introduced the single-disk corruption by running dd if=/dev/zero of=/dev/sdb1 bs=1M count=1600 seek=200
I calculated the checksum of all files again. At this point the kernel reported some block checksum mismatches in the syslog, but eventually Btrfs has recovered all the data, and the file checksums matched with the previous run.
I introduced the non-ovarlapping multi-disk corruption by running dd if=/dev/zero of=/dev/sdb1 bs=1M count=800 seek=200 && dd if=/dev/zero of=/dev/sdc1 bs=1M count=800 seek=1000
I calculated the checksum of all files again. At this point the kernel reported some block checksum mismatches in the syslog, and it could reover some blocks, but not all, and some files yielded an I/O error, but the checksum of the non-erroneous files matched with the previous run.

How to use the ssh-agent programmatically for RSA signing

2010-06-18T18:47:00.000+02:00

This blog post explains what an SSH agent does, and then gives initial hints and presents some example code using the ssh-agent wire protocol (the SSH2 version of it, as implemented by OpenSSH) for listing the public keys added to the agent, and for signing a message with an RSA key. The motivation for this blog post is to teach the reader how to use ssh-agent to sign a message with RSA. Currently there is no command-line tool for that.

The SSH agent (started as the ssh-agent command in Unix, or usually as Pageant in Windows) is a background application which can store in its process memory some SSH public key pairs in unencrypted form, for the convenience of the user. When logging in, ssh-agent is usually started for the user; the user then calls ssh-add (or a similar GUI application) to add his key pairs (e.g. $HOME/.ssh/id*) to the agent, typing the key passphrases if necessary. Afterwards, the user initiates SSH connections, for which the keys used for authentication are taken from the currently running agent. The SSH agent provides the convenience for the user that the user doesn't have to type the key passphrases multiple time, plus that if agent forwarding is enabled (ForwardAgent yes is present in $HOME/.ssh/config), then the agent is available in SSH connections initiated from SSH connections (of arbitrary depth). The agent forwarding feature of ssh-agent is unique, because other in-memory key stores such as gpg-agent don't have that feature, so the keys stored there are available only locally, and not within SSH sessions.

The SSH agent stores both the public and the private keys of a key pair (and the comment as well), but it only ever discloses the public keys to applications connecting to it. The public keys can be queried (displayed) with the ssh-add -L command. But to the ssh-agent can prove to the external world that it knows the private keys (without revealing the keys themselves), because it offers a service to sign the SHA-1 checksum (hash) of any string. SSH uses public-key cryptography, which has the basic assumption that a party can sign a message with a key only he knows the private key; but anyone who knows the public key can verify the signature.

ssh-add uses the SSH_AUTH_SOCK environment variable (containing the pathname of a Unix domain socket) to figure out which ssh-agent to connect to. The permissions for the socket pathname are set up so that only the rightful owner (or root) can connect, other users get a Permission denied.

For more information about how SSH uses public-key cryptography and agents, please read this excellent article.

Below there is an example wire dump of an application using the SSH agent protocol to ask the list of public keys added to an ssh-agent (just like ssh-add -L), and to ask the ssh-agent to sign a message with an RSA key added to it. The sign request is usually sent when an SSH client (ssh(1)) is authenticating itself to an SSH server, when establishing a connection.

To understand the wire dump below, one has to know that in the RSA public-key cryptography system the public key consists of a modulus and a public exponent, the modulus being a very large integer (2048 bits or longer), the exponent being positive a small or large integer, smaller than the modulus. Verifying a signature consits of interpreting a signature as a large integer, exponentiating it to the public exponent, taking the result modulo the modulus, and comparing that result with the original message (or, in case of SSH, the SHA-1 checksum of the original message). Please read the article linked above for a full definition and the operation of RSA.

# The client connects to the agent.
client.connect_to_agent()

# The clients lists the key pairs added to the agent.
client.write("\0\0\0\1")  # request_size == 1
  client.write("\v")  # 11 == SSH2_AGENTC_REQUEST_IDENTITIES
agent.write("\0\0\3\5")  # response_size == 773
  agent.write("\f")  # 12 == SSH2_AGENT_IDENTITIES_ANSWER
  agent.write("\0\0\0\2")  # num_entries == 2
    agent.write("\0\0\1\261")  # entry[0].key_size == 433
      agent.write("\0\0\0\7")  # key_type_size == 7
        agent.write("ssh-dss")
      agent.write("...")  # 443-4-7 bytes of the DSA key
    agent.write("\0\0\0\25")  # entry[0].comment_size == 21
    agent.write("/home/foo/.ssh/id_dsa")
    agent.write("\0\0\1\25")  # entry[1].key_size == 275
      agent.write("\0\0\0\7")  # key_type_size == 7
        agent.write("ssh-rsa")
      agent.write("\0\0\0\1")  # public_exponent_size == 1
        agent.write("#")  # public_exponent == 35
      agent.write("\0\0\1\1")  # modulus_size == 257
        agent.write("\0...")  # p * q in MSBFirst order
    agent.write("\0\0\0\25")  # entry[1].comment_size == 21
    agent.write("/home/foo/.ssh/id_rsa")

# The client gets the agent sign some data.
data = "..."  # 356 arbitrary bytes to get signed.
client.write("\0\0\2\206")  # request_size == 646
  client.write("\r")  # 13 == SSH2_AGENTC_SIGN_REQUEST
  client.write("\0\0\1\25")  # key_size == 277
    client.write("\0\0\0\7")  # key_type_size == 7
      client.write("ssh-rsa")
    client.write("\0\0\0\1")  # public_exponent_size == 1
      client.write("#")  # public_exponent == 35
    client.write("\0\0\1\1")  # modulus_size == 257
      client.write("\0...")  # p * q in MSBFirst order
  client.write("\0\0\1d")  # data_size == 356
    client.write(data)  # arbitary bytes to sign
  client.write("\0\0\0\0")  # flags == 0
agent.write("\0\0\1\24")  # response_size == 276
  agent.write("\16")  # 14 == SSH2_AGENT_SIGN_RESPONSE  (could be 5 == SSH_AGENT_FAILURE)
  agent.write("\0\0\1\17")  # signature_size == 271
    agent.write("\0\0\0\7")  # key_type_size == 7
      agent.write("ssh-rsa")
    agent.write("\0\0\1\0")  # signed_value_size == 256
      agent.write("...")  # MSBFirst order

# The client closes the SSH connection.
client.close()
agent.read("")  # EOF

Below there is an example Python script which acts as a client to ssh-agent, listing the keys added (similarly to ssh-add -L), selecting a key, asking ssh-agent to sign a message with an RSA key, and finally verifying the signature. View and download the latest version of the script here.

#! /usr/bin/python2.4
import cStringIO
import os
import re
import sha
import socket
import struct
import sys

SSH2_AGENTC_REQUEST_IDENTITIES = 11
SSH2_AGENT_IDENTITIES_ANSWER = 12
SSH2_AGENTC_SIGN_REQUEST = 13
SSH2_AGENT_SIGN_RESPONSE = 14
SSH_AGENT_FAILURE = 5

def RecvAll(sock, size):
  if size == 0:
    return ''
  assert size >= 0
  if hasattr(sock, 'recv'):
    recv = sock.recv
  else:
    recv = sock.read
  data = recv(size)
  if len(data) >= size:
    return data
  assert data, 'unexpected EOF'
  output = [data]
  size -= len(data)
  while size > 0:
    output.append(recv(size))
    assert output[-1], 'unexpected EOF'
    size -= len(output[-1])
  return ''.join(output)

def RecvU32(sock):
  return struct.unpack('>L', RecvAll(sock, 4))[0]

def RecvStr(sock):
  return RecvAll(sock, RecvU32(sock))

def AppendStr(ary, data):
  assert isinstance(data, str)
  ary.append(struct.pack('>L', len(data)))
  ary.append(data)

if __name__ == '__main__':
  if len(sys.argv) > 1 and sys.argv[1]:
    ssh_key_comment = sys.argv[1]
  else:
    # We won't open this file, but we will use the file name to select the key
    # added to the SSH agent.
    ssh_key_comment = '%s/.ssh/id_rsa' % os.getenv('HOME')

  if len(sys.argv) > 2:
    # There is no limitation on the message size (because ssh-agent will
    # SHA-1 it before signing anywa).
    msg_to_sign = sys.argv[2]
  else:
    msg_to_sign = 'Hello, World! Test message to sign.'

  # Connect to ssh-agent.
  assert 'SSH_AUTH_SOCK' in os.environ, (
      'ssh-agent not found, set SSH_AUTH_SOCK')
  sock = socket.socket(socket.AF_UNIX, socket.SOCK_STREAM, 0)
  sock.connect(os.getenv('SSH_AUTH_SOCK'))

  # Get list of public keys, and find our key.
  sock.sendall('\0\0\0\1\v') # SSH2_AGENTC_REQUEST_IDENTITIES
  response = RecvStr(sock)
  resf = cStringIO.StringIO(response)
  assert RecvAll(resf, 1) == chr(SSH2_AGENT_IDENTITIES_ANSWER)
  num_keys = RecvU32(resf)
  assert num_keys < 2000  # A quick sanity check.
  assert num_keys, 'no keys have_been added to ssh-agent'
  matching_keys = []
  for i in xrange(num_keys):
    key = RecvStr(resf)
    comment = RecvStr(resf)
    if comment == ssh_key_comment:
      matching_keys.append(key)
  assert '' == resf.read(1), 'EOF expected in resf'
  assert matching_keys, 'no keys in ssh-agent with comment %r' % ssh_key_comment
  assert len(matching_keys) == 1, (
      'multiple keys in ssh-agent with comment %r' % ssh_key_comment)
  assert matching_keys[0].startswith('\x00\x00\x00\x07ssh-rsa\x00\x00'), (
      'non-RSA key in ssh-agent with comment %r' % ssh_key_comment)
  keyf = cStringIO.StringIO(matching_keys[0][11:])
  public_exponent = int(RecvStr(keyf).encode('hex'), 16)
  modulus_str = RecvStr(keyf)
  modulus = int(modulus_str.encode('hex'), 16)
  assert '' == keyf.read(1), 'EOF expected in keyf'

  # Ask ssh-agent to sign with our key.
  request_output = [chr(SSH2_AGENTC_SIGN_REQUEST)]
  AppendStr(request_output, matching_keys[0])
  AppendStr(request_output, msg_to_sign)
  request_output.append(struct.pack('>L', 0))  # flags == 0
  full_request_output = []
  AppendStr(full_request_output, ''.join(request_output))
  full_request_str = ''.join(full_request_output)
  sock.sendall(full_request_str)
  response = RecvStr(sock)
  resf = cStringIO.StringIO(response)
  assert RecvAll(resf, 1) == chr(SSH2_AGENT_SIGN_RESPONSE)
  signature = RecvStr(resf)
  assert '' == resf.read(1), 'EOF expected in resf'
  assert signature.startswith('\0\0\0\7ssh-rsa\0\0')
  sigf = cStringIO.StringIO(signature[11:])
  signed_value = int(RecvStr(sigf).encode('hex'), 16)
  assert '' == sigf.read(1), 'EOF expected in sigf'

  # Verify the signature.
  decoded_value = pow(signed_value, public_exponent, modulus)
  decoded_hex = '%x' % decoded_value
  if len(decoded_hex) & 1:
    decoded_hex = '0' + decoded_hex
  decoded_str = decoded_hex.decode('hex')
  assert len(decoded_str) == len(modulus_str) - 2  # e.g. (255, 257)
  assert re.match(r'\x01\xFF+\Z', decoded_str[:-36]), 'bad padding found'
  expected_sha1_hex = decoded_hex[-40:]
  msg_sha1_hex = sha.sha(msg_to_sign).hexdigest()
  assert expected_sha1_hex == msg_sha1_hex, 'bad signature (SHA1 mismatch)'
  print >>sys.stderr, 'info: good signature'

The wire dump above was created by attaching to a running ssh-agent with strace. The Python example code was written by understanding the ssh-agent.c in the OpenSSH codebase.

How to fix printing with HP LaserJet 1018 on Ubuntu Lucid and Karmic

2010-06-14T11:05:00.003+02:00

This blog post explains how to fix CUPS printing for HP LaserJet 1018. The instructions and the shell script presented have been tested on Ubuntu Lucid and Ubuntu Karmic, but they might work on other Linux systems as well with minor modifications.

The first method to make the HP LaserJet 1018 work is running


$ sudo apt-get install hplip
$ sudo hp-setup

, and following the instructions, and running hp-setup again if necessary. This method seemed to work for me (I could print a test page), but I haven't tested it excessively, and theprinter got the margins wrong, because it assumed that I'm printing to Letter paper, but I was printing to A4.

The second method (recommended) is the following. Download and run this script: http://pts-mini-gpl.googlecode.com/svn/trunk/pts-hplj1018/fix_hplj1018.sh.

The script download and install the printer firmware, it will ask you to install some packages (such as cups and foo2zjs), and to connect and disconnect the printer. It will also install the printer, remove (cancel) all pending jobs, and change the CUPS printer state to ready. Run this script each time you can't print anymore (e.g. because of a system upgrade).

The script has the A4 page size (used in Europe) hard-wired. If your printer supports the Letter page size (used in the USA), replace all occurrences of the word A4 in the script by the word Letter.

Setting up the printer is tricky because all settings have to be correct (otherwise the printer will just ignore jobs sent to it), and Ubuntu gets many settings wrong. The script fixes these settings:

The updated firmware (as required by foo2zjs) has to be present in /usr/share/foo2zjs/firmware/sihp1018.dl. Ubuntu doesn't have a package containing this file. The script downloads, extracts and installs the firmware.
The correct udev rules have to be in place so /usr/sbin/hplj1018 is run (and uploads the firmware) when the printer is connected. Ubuntu Lucid has a wrong (non-matching) udev rule in /lib/udev/rules.d/85-hplj10xx.rules. The script fixes to udev rule.
The correct PPD file must be present with the correct page size. The default file /usr/share/ppd/foo2zjs/HP-LaserJet_1018.ppd.gz in Ubuntu has the Letter size, which the script changes to A4 in the file /etc/cups/ppd/HP_LaserJet_1018_pts.ppd.
No printing jobs must be active or pending when the printer gets connected (so the firmware has a chance to upload). Ubuntu doesn't care about this. The script makes sure that CUPS is not running when the printer gets connected, and it also removes all pending jobs.
On Ubuntu Lucid there is a conflict between the usblp and the usb drivers: the newest CUPS uses usb to drive USB printers (while older CUPS versions used usblp), but usblp claims the printer first, so CUPS has no chance to actually use it. The solution is to modify the DeviceURI in /etc/cups/printers.conf to parallel:///dev/usb/lp0

How to create a custom keyboard layout for X.Org on Ubuntu

2010-06-12T23:36:00.004+02:00

This blog post gives an overview of the steps needed for creating and installing a new custom keyboard layout for X.Org (X11). The steps outlined here have been tested on Ubuntu Karmic and Ubuntu Lucid, but they should work on other Linux systems as well with minor modifications. The designer of a custom keyboard layout can assign (almost) Unicode characters and other keyboard symbols (such as F1 or Insert) arbitrarily to keys and key combinations (with modifiers like Shift and Ctrl). You are encouraged to create a custom keyboard layout for yourself if some useful symbols are missing from your favorite keyboard layout, or you can work with your own custom key mapping much faster with the default layouts, or you need a blend of two layouts with AltGr (right Alt) as the modifier key for selecting the alternative layout for 1 keypress.

All the keyboard layouts are managed by the XKB subsystem. For simplicity, we assume that you run both your X server and clients (at least GNOME and the setxkbmap command) on the same machine. The keyboard layouts are defined in the text files /usr/share/X11/xkb/symbols/*, e.g. the file /usr/share/X11/xkb/symbols/us contains the definition of the US English keyboard layout (and a few of its the variants). Edit one of these files (us. in the example) as root, adding your entry, e.g.

partial alphanumeric_keys modifier_keys
xkb_symbols "my_layout" {
    // A comment here.
    name[Group1]= "USA - my layout";
    key <....> { ... };
    ...
};

As soon as you save your changes, you are able to load the new layout with

$ setxkbmap 'us(my_layout)'

Please note that the command above is a bit dangerous, because if you load an incomplete or bad layout, you may have to log out (using the mouse) and log in again to make your keyboard useful. Another option is to copy the following command (which restores the US English keyboard layout) with its trailing newline to the clipboard before changing the layout, and then pasting it to a terminal emulator when things go wrong:

$ setxkbmap us

You can load multiple layouts and then use GNOME's keyboard switching applet to switch between them. For example, the following command loads the French layout by default, with an option to switch to your custom layout, and the default English layout as well:

$ setxkbmap 'fr,us(my_layout),us'

Once your layout is ready, you may want to make it selectable from the GNOME keyboard preferences dialog box (get it from System / Preferences / Keyboard / Layouts / Add, or start it with gnome-keyboard-properties, and then select Layouts / Add). To do so, you have to add your layout to the files /usr/share/X11/xkb/rules/*.{xml,lst}, close the GNOME keyboard preferences dialog box, reopen it, and then add a new layout which is yours. There is no need to log out or to restart the system. Ubuntu Karmic and Ubuntu Lucid look at evdev.xml and evdev.lst (as reported by running $ strace -e open -o key.log gnome-keyboard-properties), other systems might look at some other files, especially base.xml and base.lst.

As an example, see http://code.google.com/p/pts-mini-gpl/source/browse/trunk/pts-magyar/us.pts_magyar.sh, which is an installer shell script, which installs the us(pts_magyar) keyboard layout, modifying files /usr/share/X11/xkb/symbols/us and /usr/share/X11/xkb/rules/{evdev,base}.{xml,lst}. Here is how to install it easily:

$ wget -O- \
  http://pts-mini-gpl.googlecode.com/svn/trunk/pts-magyar/us.pts_magyar.sh | bash

FYI It is possible to load the config files from /usr/share/X11/xkb/symbols on one machine, and to apply it on any X server (as identified by the $DISPLAY variable). To do so, run this on the machine with the config file:

setxkbmap us -print | xkbcomp - "$DISPLAY"

How to determine if two line segments intersect (in 2D)?

2010-06-11T08:32:00.001+02:00

This blog post gives the C++ implementation of the fast schoolbook algorithm for determining if two (finite) line segments on a 2D plane intersect (cross).

static bool IsOnSegment(double xi, double yi, double xj, double yj,
                        double xk, double yk) {
  return (xi <= xk || xj <= xk) && (xk <= xi || xk <= xj) &&
         (yi <= yk || yj <= yk) && (yk <= yi || yk <= yj);
}

static char ComputeDirection(double xi, double yi, double xj, double yj,
                             double xk, double yk) {
  double a = (xk - xi) * (yj - yi);
  double b = (xj - xi) * (yk - yi);
  return a < b ? -1 : a > b ? 1 : 0;
}

/** Do line segments (x1, y1)--(x2, y2) and (x3, y3)--(x4, y4) intersect? */
bool DoLineSegmentsIntersect(double x1, double y1, double x2, double y2,
                             double x3, double y3, double x4, double y4) {
  char d1 = ComputeDirection(x3, y3, x4, y4, x1, y1);
  char d2 = ComputeDirection(x3, y3, x4, y4, x2, y2);
  char d3 = ComputeDirection(x1, y1, x2, y2, x3, y3);
  char d4 = ComputeDirection(x1, y1, x2, y2, x4, y4);
  return (((d1 > 0 && d2 < 0) || (d1 < 0 && d2 > 0)) &&
          ((d3 > 0 && d4 < 0) || (d3 < 0 && d4 > 0))) ||
         (d1 == 0 && IsOnSegment(x3, y3, x4, y4, x1, y1)) ||
         (d2 == 0 && IsOnSegment(x3, y3, x4, y4, x2, y2)) ||
         (d3 == 0 && IsOnSegment(x1, y1, x2, y2, x3, y3)) ||
         (d4 == 0 && IsOnSegment(x1, y1, x2, y2, x4, y4));
}

The algorithm above does 8 multiplications, 16 subtractions and 52 comparisons in the worst case. But it doesn't do any divisions or square roots. If the input coordinates are integers, and the multiplications and subtractions don't yield an overflow, then the whole algorithm can be performed only with integer arithmetic.

The algorithm above doesn't return the coordinates of the intersection point. To find the intersection point of two intersecting line segments, see the formula on http://en.wikipedia.org/wiki/Line-line_intersection.

If there are many line segments and we are interested in all intersections, calling the above algorithm for all pairs is slow (quadratic). See faster algorithms (O(n*log(n))) on http://en.wikipedia.org/wiki/Line_segment_intersection

How to disable the login sound and other sounds in Ubuntu Karmic

2010-06-07T00:16:00.005+02:00

This blog post explains how to disable the login sounds and other sounds for the default GDM and the GNOME session. The solution presented here was tested on Ubuntu Karmic Koala, but there is a little probability that some parts of it would work in other versions of Ubuntu or other Linux systems.

How to disable the login drum sound and the beep sound before logging in

Run this command in a terminal window (without the $ signs):

$ sudo -u gdm gconftool-2 --type=bool --set /desktop/gnome/sound/event_sounds false

Please note that many non-working solution attempts are available on the net, some of them involving the key /apps/gdm/simple-greeter/settings-manager-plugins/sound/active or /apps/gnome_settings_daemon/plugins/sound/active, but those don't work in Ubuntu Karmic.

The sudo -u gdm part in this solution is essential, because the sound is played by GDM, which looks at the settings for the user gdm. Without that part (or with only a simple sudo) it wouldn't have any effect.

The annoying drum sound file is /usr/share/sounds/ubuntu/stereo/system-ready.ogg . Some solutions on the net suggest removing this file (and also bell.ogg). There is no need to do so.

How to disable the welcome sound after logging in

In the menu bar on the top of the screen, select System / Preferences / Startup Applications. In the appearing dialog box, scroll down to find GNOME Login Sound. Untick it. Click on the Close button.

As an alternative, it is possible to disable this sound from the command-line as well:

$ perl -pi -0777 -e 's@^X-GNOME-Autostart-enabled=.*\n@@mg;
       s@\n*\Z(?!\n)@\nX-GNOME-Autostart-enabled=false\n@' \
       ~/.config/autostart/libcanberra-login-sound.desktop

How to disable the Skype login sound after logging in

Start Skype, find its Options dialog box. Select the Notifications tab. In the tab, select Skype Login. Untick Enable Event. Click on the Apply button.

How to disable system sounds (e.g. on mouse clicks)

Run this command in a terminal window (without the $ signs):

$ gconftool-2 --type=bool --set /desktop/gnome/sound/event_sounds false

The following commands don't seem to make any difference, but they wouldn't hurt:

$ gconftool-2 --type=bool --set /apps/gnome_settings_daemon/plugins/sound/active false
$ gconftool-2 --type=bool --set \
  /apps/gdm/simple-greeter/settings-manager-plugins/sound/active false
false

How to disable application sounds (e.g. notifications and game sounds)

Run this command in a terminal window (without the $ signs):

$ gconftool-2 --type=bool --set /apps/empathy/sounds/sounds_enabled false
$ gconftool-2 --type=bool --set /apps/gnobots2/preferences/enable_sound false
$ gconftool-2 --type=bool --set /apps/gnome-power-manager/ui/enable_sound false
$ gconftool-2 --type=bool --set /apps/aisleriot/sound false
$ gconftool-2 --type=string --set /apps/nautilus/preferences/preview_sound never
$ gconftool-2 --type=bool --set /apps/gnibbles/preferences/sound false
$ gconftool-2 --type=bool --set /apps/iagno/sound false
$ gconftool-2 --type=bool --set /apps/gnect/sound false
$ gconftool-2 --type=bool --set /apps/evolution/eplugin/mail-notification/sound-enabled false
$ gconftool-2 --type=bool --set /apps/evolution/eplugin/evolution_indicator/play_sound false

The following commands don't seem to make any difference, but they wouldn't hurt:

$ gconftool-2 --type=bool --set /apps/gnome_settings_daemon/plugins/sound/active false
$ gconftool-2 --type=bool --set \
  /apps/gdm/simple-greeter/settings-manager-plugins/sound/active false

Your system might have more sounds settings if you have non-default applications installed. To find these settings, start gconf-editor, select the Edit / Find menu item, tick the box to search in keys, type sound, and click on the Search button. At the bottom of the main gconf-editor window, you'll find the matches. Browse them and disable them.

How to disable the user list at the GDM login screen in Ubuntu Karmic

2010-06-06T23:06:00.000+02:00

This blog post explains how to disable the user list at the GDM login screen. The solution presented here was tested on Ubuntu Karmic Koala, but it might work on other Ubuntu versions as well (Hardy, Interpid, Jaunty, Karmic and Lucid).

To disable the user list, run this command in a terminal window (without the $ signs):

$ sudo -u gdm gconftool-2 --type=bool --set /apps/gdm/simple-greeter/disable_user_list true

Logout and login again. You have to type your user name from now on.

How to disable Compiz Fusion and enable Metacity on Ubuntu Karmic

2010-06-06T22:33:00.003+02:00

This blog post explains how to disable Compiz Fusion (3D desktop effects), and enable Metacity (2D) as the window manager for a GNOME session, for a single user. The solution presented here was tested on Ubuntu Karmic Koala, but it might work on other Ubuntu versions as well (Hardy, Interpid, Jaunty, Karmic and Lucid).

To disable Compiz and enable Metacity, run these commands in a terminal window (without the $ signs):

$ gconftool-2 --type=string --set \
  /desktop/gnome/applications/window_manager/current /usr/bin/metacity
$ gconftool-2 --type=string --set \
  /desktop/gnome/applications/window_manager/default /usr/bin/metacity

Logout and login again. Metacity should be running.

FYI the old setting was /usr/bin/compiz.

How to add the VirtualBox keyring to Debian and Ubuntu

2010-06-02T20:48:00.001+02:00

This blog post explains how to add the Sun and Oracle VirtualBox keyring to Debian and Ubuntu, to prevent apt-get update from complaining.

If apt-get update gives you a warning with NO_PUBKEY 54422A4B98AB5139, then do this to fix it:


$ wget -q http://download.virtualbox.org/virtualbox/debian/sun_vbox.asc -O- |
  sudo apt-key add -
$ wget -q http://download.virtualbox.org/virtualbox/debian/oracle_vbox.asc -O- |
  sudo apt-key add -
$ sudo apt-get update

How to implement a semaphore using mutexes

2010-05-29T20:49:00.005+02:00

This blog post shows an educational reference implementation of a semaphore using mutexes in Python.

Please note that the number of mutexes needed in this implementation is 1 + the number of threads waiting in the acquire operation while the value is 0. This is too much, 2 or 3 mutexes altogether would be enough, see the various implementations in Implementation of General Semaphores Using Binary Semaphores.

The implementation is available for download from http://pts-mini-gpl.googlecode.com/svn/trunk/script/semaphore.py.

import thread
from collections import deque


class Semaphore(object):
  def __init__(self, value=1):
    assert value >= 0, 'Semaphore initial value must be >= 0'
    self._lock = thread.allocate_lock()   # Create a mutex.
    self._waiters = deque()
    self._value = value

  def acquire(self, blocking=True):
    """Semaphore P primitive."""
    self._lock.acquire()
    while not self._value:
      if not blocking:
        break
      assert self._lock.locked(), 'wait() of un-acquire()d lock'
      waiter = thread.allocate_lock()
      waiter.acquire()
      self._waiters.append(waiter)
      self._lock.release()
      try:  # Restore state even if e.g. KeyboardInterrupt is raised.
        waiter.acquire()
      finally:
        self._lock.acquire()
    self._value -= 1
    self._lock.release()

  def release(self):
    """Semaphore V primitive."""
    self._lock.acquire()
    self._value += 1
    assert self._lock.locked(), 'notify() of un-acquire()d lock'
    _waiters = self._waiters
    if _waiters:
      _waiters.popleft().release()
    self._lock.release()


if __name__ == '__main__':
  s = Semaphore(3)
  def Reader():
    while True:
      raw_input()
      s.release()
  thread.start_new_thread(Reader, ())
  print 'Press <Enter> twice to continue.'
  for i in xrange(s._value + 2):
    print i
    s.acquire()
  print 'Done.'

Feature comparison of Python non-blocking I/O libraries

2010-05-18T00:10:00.007+02:00

This blog post is a tabular feature comparison of Syncless and the 6 most popular event-driven and coroutine-based non-blocking (asynchronous) networking I/O libraries for Python. It was inspired by http://nichol.as/asynchronous-servers-in-python (published on 2009-11-22), which compares the features and the performance of 14 Python non-blocking networking I/O libraries. We're not comparing generator-based (yield) solutions here. We haven't made performance measurements, so speed-related claims in this comparison are beliefs and opinions rather than well-founded facts. Here are the results:

	Concurrence	Eventlet	Tornado	Twisted	asyncore	gevent	circuits	Syncless
pure Python: can work without compiling C code	no	yes	yes	yes	yes	no	yes	no
pure Python: runs at full speed without compiling C code	no	yes	yes	yes	yes	no	yes	no
standard module in Python 2.6	no	no	no	no	yes	no	no	no
has asynchronous DNS resolver	no	yes, 1)	no	yes, 2)	no	yes, 3)	no	yes, 4)
supports running other tasklets while DNS resolving is in progress	no	yes	no	yes	no	yes	no	yes
has fully asynchronous and scalable DNS resolver	no	yes, 5)	no	yes	no	yes	no	yes
supports timeouts on individual socket send() and recv() operations	yes	yes	no	yes	no	yes	no	yes
has WSGI server	yes	yes	yes	yes	no	yes	yes--, 6)	yes
can work with CherryPy's WSGI server	no	yes--, 7)	no	no	no	yes--, 7)	no	yes
contains a custom, non-WSGI web framework	yes	no	yes	yes	no	no++, 8)	yes	no
can run external web frameworks using non-WSGI connectors	no	yes, 9)	no	no	no	no	no	yes++, 10)
runs with Stackless Python	yes	yes--, 11)	yes, 12)	yes, 12)	yes, 12)	no	yes, 12)	yes
runs with greenlet	yes--, 13)	yes	yes, 12)	yes, 12)	yes, 12)	yes	yes, 12)	yes
has fast non-blocking socket class implemented in C or Pyrex	no	no	no	no	no	no	no	yes
has fast read and write buffer code implemented in C or Pyrex	yes	no	no	no	no	yes, 14)	no	yes
uses fast (C or Pyrex) buffer code for its socket.socket.makefile()	yes	no	no	no	no	no	no	yes
uses fast (C or Pyrex) buffer code for its ssl.SSLSocket.makefile()	no, 15)	no	no	no	no	no	no	yes
has SSL support	no	yes, 16)	no	yes, 16)	no	yes, 16)	yes, 16)	yes, 16)
has monkey-patchig for socket.socket and other blocking I/O operations	no	yes	no, 12)	no, 12)	no, 12)	yes	no, 12)	yes
prints and recovers from an uncaught exception	yes, 17)	yes	yes	yes	yes	yes	no, 18)	no, 19)
can use libevent	yes	yes	no	yes	no	yes	no	yes
can use the libevent emulation of libev	yes	yes	no	yes	no	no, 20)	no	yes
works without libevent or an emulation installed	no	yes	yes	yes	yes	no	yes	yes
avoids C stack copying (which is slower than soft switching)	no	no, 21)	yes, 12)	yes, 12)	yes, 12)	no, 21)	yes, 12)	no, 22)
can use a high performance event notification primitive (e.g. epoll on Linux)	yes, 23)	yes	yes	yes	no, 24)	yes, 23)	yes	yes, 25)
nichol.as: What license does the framework have?	yes, 26)	yes, 26)	yes, 27)	yes, 26)	yes, 26)	yes, 26)	yes, 28)	yes, 29)
nichol.as: Does it provide documentation?	yes	yes	yes	yes++, 30)	no	yes	yes	yes--, 31)
nichol.as: Does the documentation contain examples?	yes	yes	yes	yes	no	yes	yes	yes
nichol.as: Is it used in production somewhere?	yes, 32)	yes, 33)	yes, 34)	yes	yes, 35)	yes, 36)	yes, 37)	no
nichol.as: Does it have some sort of community (mailinglist, irc, etc..)?	yes	yes	yes++, 38)	yes	no	yes	yes	no
nichol.as: Is there any recent activity?	no, 39)	yes	yes	yes	no	yes	yes	yes
nichol.as: Does it have a blog (from the owner)?	yes	yes	yes, 40)	yes++, 41)	no	yes	yes	yes--, 42)
nichol.as: Does it have a Twitter account?	yes	yes	yes	yes	no	yes	yes	yes--, 42)
nichol.as: Where can I find the repository? (without links)	yes, 43)	yes, 44)	yes, 43)	yes, 45)	yes, 46)	yes, 47)	yes, 44)	yes, 48)
nichol.as: Does it have a thread pool?	no	yes	no	yes	no	no	yes	yes--, 49)
nichol.as: Does it provide a HTTP server for WSGI applications?	yes	yes	yes--, 50)	yes	no	yes	yes--, 6)	yes
nichol.as: Does it provide access to a TCP Socket?	yes	yes	yes	yes	yes	yes	yes	yes
nichol.as: Does it have any Comet features?	no	no	no	no	no	no	no	no
nichol.as: Is it using epoll, if available?	yes, 23)	yes	yes	yes	no, 24)	yes, 23)	no++, 51)	yes, 25)
nichol.as: Does it have unit tests?	yes	yes	no	yes++, 52)	yes	yes	yes	yes, 53)
provides generic timeout for any block of code	yes	yes	yes	no	no	yes	no	yes
provides synchronization primitives (e.g. semaphore, codition variable)	yes--, 54)	yes	no	yes	no	yes	no	no
lets the programmer control event delivery order (e.g. with priorities)	no	no	no	no	no	no	no	no
provides callbacks (links) when some work is finished	no	yes	yes, 12)	yes, 12)	yes, 12)	yes	yes, 12)	no
has a high-level, comprehensive, consistent network programming framework	no	yes	yes--, 55)	yes	no, 56)	yes--, 55)	yes	no
has non-blocking select.select() implementation	no	yes	no, 12)	no, 12)	no, 12)	yes	no, 12)	yes
implements some application-level protocols beyond HTTP, WSGI and DNS	yes, 57)	no++, 58)	yes, 59)	yes++, 60)	no	no++, 58)	yes, 61)	no++, 62)
compatible with other non-blocking systems in the same process	no, 63)	yes, 64)	no, 63)	yes++, 65)	no, 63)	no++, 66)	yes, 67)	yes++, 68)

Comments:
1) thread pool or Twisted
2) built-in
3) evdns
4) evdns or its equivalents: minihdns or evhdns
5) if Twisted is available
6) does not implement the WSGI spec properly (no write or close)
7) by manually monkey-patching socket and thread
8) has its own simple HTTPServer class
9) BaseHTTPServer
10) BaseHTTPServer, CherryPy, web.py, Google webapp
11) has partial, incomplete and incomatible greenlet emulation
12) event-driven
13) has partial and incompatible Stackless Python emulation
14) evbuffer
15) no SSL support
16) client and server
17) in the Tasklet class
18) sliently ignores exceptions and keeps the connection hanging
19) the process exits
20) needs evdns
21) uses greenlet
22) calls stackless.schedule() from C code
23) uses libevent
24) but uses poll and epoll could be added asily
25) can use libev or libevent
26) MIT
27) Apache
28) GNU GPL v2 or later
29) Apache License, Version 2.0
30) exensive
31) README and docstrings
32) Hyves.nl (a Dutch social networking site)
33) Second Life (a virtual reality game)
34) FriendFeed (a social networking site)
35) pyftpdlib, supervisor [link] (via medusa)
36) many, [link] and [link]
37) Naali, TAMS, website-profiler, kdb IRC bot, python-brisa uPnP
38) huge
39) not since 2009-11-19
40) at facebook
41) lots
42) the author writes Syncless-related stuff to his blog
43) GIT on github
44) Mercurial on bitbucket
45) in its own Subversion + Trac repository
46) in Python's Subversion
47) Mercurial on bitbuckig
48) Subversion on Google Code
49) not tested in production for robustness and speed
50) limited
51) it can be enabled manually
52) extensive
53) unit tests cover only parts of the functionality
54) provides queues
55) not comprehensive
56) low-level
57) MySQL client
58) Python console backdoor, can monkey-patch external modules
59) storage server compatible with Amazon S3; OpenID
60) tons of protocols
61) pygame, GTK+, inotify and pygame
62) can monkey-patch external modules
63) not by default
64) has Twisted reactor
65) many including GTK+ and Qt
66) not by default, but there is the project gTornado
67) pygame and GTK
68) has monkey-patching for Concurrence, Eventlet, Tornado, Twisted, gevent, circuits and asyncore

Google Code Data API

2010-04-29T17:23:00.001+02:00

This blog post documents the options for programmatic (non-web-browser) access to open source project hosting at Google Code.

As of 2010-04-29, there is no full Google Code Data API, but there is an API for uploading files and an API for the issue tracker.

For uploading files, the following scripts can be useful:

Syncless: Non-blocking WSGI server and networking library for Stackless Python

2010-04-19T05:21:00.001+02:00

This blog post is a release announcement of the software named Syncless I've been developing recently for high-performance, low-memory, single-process, single-threaded, lock-free network communication (TCP/IP client and server) in Python, using coroutines for apperent parallel processing. I needed something which needs less memory than pre-forking worker instances, but doesn't require complicated and error-prone synchronization mechanisms (such as mutexes in a multi-threaded program).

Syncless is an experimental, lightweight, non-blocking (asynchronous) client and server socket network communication library for Stackless Python 2.6. For high speed, Syncless uses libevent, and parts of Syncless' code is implemented in C (Pyrex). Thus Syncless can be faster than many other non-blocking Python communication libraries. Syncless contains an asynchronous DNS resolver (using evdns) and a HTTP server capable of serving WSGI applications. Syncless aims to be a coroutine-based alternative of event-driven networking engines (such as Twisted and FriendFeed's Tornado), and it's a competitor of gevent, pyevent, Eventlet and Concurrence.

Version 0.03 of Syncless has been released. It is a complete rewrite so it has drop-in replacements for built-in Python socket.socket, socket.gethostbyname (etc.), ssl.SSLSocket, time.sleep and select.select. The replacement implementations... is easily monkey-patchable, so legacy (blocking) pure Python I/O code can be made non-blocking without a code change. Core functionality is now written in Pyrex/Cython for greater speed.

Samsung Story Station Plus external e-SATA hard drive speed test on Linux

2010-03-28T20:58:00.004+02:00

This blog post shows the results of the speed test I've done with my new Samsung Story Station Plus external e-SATA hard drive on a Linux nettop. Executive summary: the drive is large, fast and quiet; e-SATA with hot-plug works fine and full speed on Ubuntu Hardy and Ubuntu Karmic out-of-the box.

Speed results:

sustained local hard drive sequential read: 53 MB/s; dd if=/dev/sda of=/dev/null bs=1M
sustained local hard drive sequential write: (not tried to avoid possible damage to the existing system)
sustained local hard drive random seek + read 4 KB: 50 seeks/s, 206 kB/s
sustained external e-SATA hard drive sequential read: 112 MB/s: dd if=/dev/sdb of=/dev/null bs=1M
sustained external e-SATA hard drive sequential write: 108 MB/s: dd if=/dev/zero of=/dev/sdb bs=1M seek=32000
sustained external e-SATA hard drive random seek + read 4 KB: 90 seeks/s, 370 kB/s

System information:

Linux Ubuntu Hardy 64-bit desktop
dual Intel Atom 330 @ 1.60GHz
Nvidia ION
3.3GB RAM
internal hard drive: 2.5"; 250 GB; Hitachi HTS54322 FBEO
external e-SATA hard drive: Samsung Story Station Plus; 3.5"; 2TB; SAMSUNG HD203WI 1AN1
random seek benchmark implemented in Python, using 0.3% CPU

Other parameters of the external drive:

very quiet, I was not able to tell when it's reading during thesequential read or write benchmarks
very quiet, I had to lean with my ear 5 cm from the drive to hear itworking during the random seek test in a work environment. However, at home, when there is no other noise, I can hear the seeks if I concentrate.
low-power (0.1 Watt in standby, goes to standby quickly).

I've repeated the speed test with a Fujitsu Siemens Amilo 3560 notebook running Ubuntu Karmic 32-bit desktop, and the speeds are about the same as with the nettop.

How to cope with NIS (YP) server outages on Linux

2010-03-28T20:45:00.001+02:00

This blog post explains how to cope with NIS (YP) server outages on Linux (tested on Debian Lenny), i.e. how to give applications instant and valid response even when the NIS server is unreachable. When parts of the user (passwd) and group database is not stored locally, but on a NIS (YP) server, then each user and group lookup request (such as getpwnam(3) and id USERNAME) is sent to the NIS server. Should the NIS server be down or unreachable, each such request is blocked until a very long timeout (sometimes more than a minute). Also sometimes the client (ypbind) doesn't detect properly that the NIS server is back up again, so user lookups tend to fail or time out even when the NIS server has recovered. Another problem with communicating with the NIS server is that it's hard to write a strict firewall rule, because the UDP port number of the NIS service (ypserv) is not fixed, but dynamically assigned and managed by the RPC portmap service.

The solution presented here involves caching all NIS data locally, updating the cache from a cron job, and directing applications to always read the cache. This design gives instant response to applications, no matter whether the NIS server is reachable, and since cache updates are performed by a dedicated user, more precise firewall packet filters can be written.

The instructions below have been verified on Debian Lenny, but they should work similarly on other Linux systems. It is assumed that the NIS domain to connect to is called mydomain (the real domain name usually available in /etc/domainname), and the IP address of the NIS server is 1.2.3.4.

Make sure root can download the passwd database from NIS. Try this command: # ypcat -d mydomain -h 1.2.3.4 passwd This should print the the NIS user passwd database, including user names and (encrypted) passwords. If the 2nd field is x for all users than the passwords may be in the shadow file on the NIS server. Try this: # ypcat -d mydomain -h 1.2.3.4 shadow If this doesn't print any encrypted passwords, and you need password-based login on your machine for NIS users, then it wouldn't work for you (you may try complaining to the admin of the NIS server).
Install software (as root) to the client Linux system: # apt-get install sudo nis python2.4 libnss-extrausers
Create a user named nis-update: $ adduser --system --home=/tmp/blah --group nis-update (This disables login by default, and sets the user's login shell to /bin/false.)
Make sure that the line in /etc/shadow starting with nis-update: starts with nis-update:*: instead of nis-update-!:. This disables login for that user.

If you have a firewall which restricts outgoing packets on the local machine, you may have to add these rules to allow the nis-update user can download the NIS data:

# Allow portmap.
iptables -A OUTPUT -m owner --uid-owner nis-update \
    -p tcp --dport 111 -d 1.2.3.4 -j ACCEPT
# Allow all Sun RPC, including NIS (YP).
iptables -A OUTPUT -m owner --uid-owner nis-update \
    -p udp --dport 511:2999  -d 1.2.3.4 -j ACCEPT

Make sure that the nis-update user can download the passwd database from NIS. Try that this command prints all user records: # sudo -c nis-update ypcat -d mydomain -h 1.2.3.4 passwd

Add the following script as an executable /usr/local/sbin/nis-update.py:

#! /usr/bin/python2.4
# by pts@fazekas.hu at Sun Mar 28 18:21:20 CEST 2010

"""Script to download NIS (YP) users and groups for libnss_extrausers.so* .

This script should be run from a cron job.

Please also check /etc/nsswitch.conf for libnss_extrausers.so* . It reads
from /var/lib/extrausers/passwd etc.
"""

__author__ = 'pts@fazekas.hu (Peter Szabo)'

import pwd
import signal
import sys
import os

NIS_UPDATE_DOWNLOAD_DIR = '/var/cache/nis-update'
NIS_UPDATE_COPY_TARGET_DIR = '/var/lib/extrausers'
NIS_SERVER='1.2.3.4'   #### fix at install time
NIS_DOMAIN='mydomain'  #### fix at install time

def NisUpdate(uid, gid, nis_filename):
  assert nis_filename in ('passwd', 'group', 'shadow')
  download_filename = os.path.join(NIS_UPDATE_DOWNLOAD_DIR, nis_filename)
  target_filename = os.path.join(NIS_UPDATE_COPY_TARGET_DIR, nis_filename)
  fd = os.open(download_filename, os.O_WRONLY|os.O_TRUNC|os.O_CREAT, 0644)
  pid = os.fork()
  euid = os.geteuid()
  assert euid == uid or euid == 0, (euid, uid)
  if not pid:
    try:
      fdnull = os.open('/dev/null', os.O_RDONLY)
      if fdnull != 0:
        os.dup2(fdnull, 0)
        os.close(fdnull)
      if fd != 1:
        os.dup2(fd, 1)
        os.close(fd)
      if euid != uid:  # It's root
        os.setgroups([])
        os.setregid(gid, gid)
        os.setreuid(uid, uid)
      signal.alarm(5)
      # This doesn't need a running ypbind.
      os.execl('/usr/bin/ypcat', 'ypcat', '-d', NIS_DOMAIN, '-h', NIS_SERVER,
               nis_filename)
    except:
      exc_info = sys.exc_info()
      print >>sys.stderr, 'error in child: %s: %s' % (
         exc_info[1].__class__, exc_info[1])
      os._exit(1)
  os.close(fd)
  got_pid, status = os.waitpid(pid, 0)
  assert got_pid == pid
  st = os.stat(download_filename)
  if not status:
    status = 0
  if status or not st.st_size:
    print >>sys.stderr, 'warning: child %s failed with status 0x%x' % (
        nis_filename, status)
    return False
  os.rename(download_filename, target_filename)
  return True

if __name__ == '__main__':
  # Don't print anything, we're optimized for running as a cron job. 
  p = pwd.getpwnam('nis-update')
  NisUpdate(p.pw_uid, p.pw_gid, 'passwd')
  NisUpdate(p.pw_uid, p.pw_gid, 'group')
  # TODO: Download and os.chmod the shadow file if necessary.

In the script above, search for ####, and customize those settings.
As root, create the script output directories: # mkdir -p /var/cache/nis-update /var/lib/extrausers; chown root. /var/cache/nis-update /var/lib/extrausers; chmod 700 /var/cache/nis-update; chmod 755 /var/lib/extrausers
Run the script as root: # /usr/local/sbin/nis-update.py This should not print any error messages, and it should create the files /var/lib/extrausers/{passwd,group}.
If you also need the shadow file (because it contains passwords), then modify the script so it creates that file as well.
Add the script to your crontab so it runs every 5 minutes: # echo '0-55/5 * * * * root /usr/local/sbin/nis-update.py' | sudo tee /etc/cron.d/nis-update
Make sure that the nss-extrausers pacakge is installed. (There was an instruction above.) # apt-get install nss-extrausers
Edit /etc/nsswitch.conf, and set the following values:
```
passwd: files extrausers
group: files extrausers
shadow: files extrausers
```
(The previous values were probably compat or files nis.)
At this point, applications would use the cached files /var/lib/extrausers/{passwd,group}. Test this by checking that $ perl -e 'while(@L=getpwent){print join("+",@L),"\n"}' prints all local users followed by all NIS users.
Turn off the NIS client: # /etc/init.d/nis stop
Run the check above again and make sure that all NIS users are present: $ perl -e 'while(@L=getpwent){print join("+",@L),"\n"}'
Disable NIS client startup at boot time: # mv /etc/domainname{,.not}
Occasionally check root's e-mail for error messages from the nis-update.py cron job.

Python web framework of the day

2010-03-27T22:06:00.002+01:00

Since there are quite a lot of web application frameworks for Python, I think one more wouldn't hurt.

I was impressed by the compactness of Ruby's Sinatra:

require 'rubygems'
require 'sinatra'
get '/' do
  'Hello world!'
end

My goal was to create something similar for Python, as a proof-of-concept. Here is the equivalent Hello, world web application with my Python syntax:

from wfotd import *
@GET
def _():
  return 'Hello world!'

The Ruby solution looks much more clear and beautiful. Nevertheless, it was a nice Python coding experiment. I named it PYthon Web Framework Of The Day. The source is available at http://code.google.com/p/pts-mini-gpl/source/browse/#svn/trunk/pywfotd.

Here is a larger Python example, demonstrating the scoping and argument passing features of pywfotd:

from wfotd import *
@GET
def _(name=None):
  name = name or 'World'
  return 'Hello, %s!' % name

@GET
def foo():
  return 'This is /foo'

class bar:
  class baz:
    @GET
    def _():
      return 'This is /bar/baz'
    @GET
    def quux():
      return 'This is /bar/baz/quux'

How to try the GDM login screen in many resolutions

2010-02-20T11:26:00.001+01:00

This blog post explains how to try the GDM login screen and make screen shots in any, user-specified screen resolution. This can be useful when designing GDM themes. The instructions were tried on Ubuntu Hardy (8.04), but they should work on Ubuntu Intrepid (8.10) and Ubuntu Jaunty (9.04). They won't work on Ubuntu Karmic or Ubuntu Lucid, because they contain GDM 2, which doesn't have the --xnest feature implemented (** (gdmflexiserver:18758): WARNING **: Not yet implemented).Installation instructions:

(When asked to run a command, run it in a terminal window.)
Make sure GDM is your display manager. Since GDM is the default, chances are that it is. To make sure, run sudo dpkg-reconfigure gdm, and select GDM if it asks you.
Log in using GDM if you haven't already done so. To do that, run sudo /etc/init.d/gdm restart . (You will lose your X11 session doing so.)
Install Xephyr, a modern nested (embedded) X server, similar to Xnest: sudo apt-get install xserver-xephyr
Install ImageMagick for making screen shots in various image file formats: sudo apt-get install imagemagick
Try Xephyr: Xephyr :9 -screen 800x600 -extension RANDR . A new window should appear with the usual a black-and-white diagonal background and a diagonal cross-shaped cursor. Abort Xephyr in the terminal window. Try with different -screen sizes.
Find your GDM Xnest command line by running grep ^Xnest= /etc/gdm/gdm.conf . You should see something like Xnest=/usr/share/gdm/gdmXnestWrapper -br -audit 0.

Create a new xnest wrapper script:

echo '#!/bin/sh
GEOMETRY="`cat /tmp/new-xnest-geometry`"
GEOMETRY="${GEOMETRY:-800x600}"
exec Xephyr "$@" -screen "$GEOMETRY" -extension RANDR' |
sudo tee  /usr/local/sbin/gdm-xnest &&
chmod 755 /usr/local/sbin/gdm-xnest

Create the screenshot helper script:

echo '#! /bin/bash --
# by pts@fazekas.hu at Sat Feb 20 11:09:20 CET 2010
GOT=($(ps x | perl -ne '\''if (/[ ]Xephyr :/) {
  $c++;
  print"$1 $2\n" if (/[ ]Xephyr :(\d+)(?= ).* -auth (\S+)/)}
  END{die"Xephyr not found\n"if!$c;die"multiple Xephyrs found\n"if$c>1}'\''))
test "$?" = 0 || exit "$?"
if test ${#GOT} != 2; then
  echo "Xephyr detection failed" >&2
  exit 90
fi
export DISPLAY=":${GOT[0]}" XAUTHORITY="${GOT[1]}"
exec import -window root "$@"' |
sudo tee -a    /usr/local/bin/xephyr-import &&
sudo chmod 755 /usr/local/bin/xephyr-import

Change the Xrandr= setting in /etc/gdm/gdm.conf to use the new wrapper script: sudo perl -pi -0777 -e 's@^Xnest=.*@Xnest=/usr/local/sbin/gdm-xnest -br -audit 0@gm' /etc/gdm/gdm.conf
Make sure the Xrandr= setting is changed: check that running grep ^Xnest= /etc/gdm/gdm.conf displays Xnest=/usr/local/sbin/gdm-xnest -br -audit 0
Save your work in your X11 session, and restart GDM: sudo /etc/init.d/gdm restart . (You will lose your X11 session doing so.)

Usage instructions (how to show the GDM login dialog inside a window, and how to make a screen shot):

If not already logged in, login in at the GDM login screen.
Run gdmflexiserver --xnest to get the GDM login dialog inside a window of your regular session. You can specify the window size using e.g. echo 1024x768 >/tmp/new-xnest-geometry && gdmflexiserver --xnest
To make a JPEG screen shot, run xephyr-import -quality 75 screenshot.jpg while the Xephyr window with the GDM login screen is still open.
To make a PNG screen shot, run xephyr-import screenshot.png .