Syntax highlighted Python

Example syntax highlighted Python code: (please ignore)

def Hello(self, x, y): 
  return x * y + "foo"  # bar 

The syntax highlighting was done in the borland style on http://pygments.org/demo/.

Ruby on Rails 2.3 unit tests

The blog post Ruby on Rails Unit Tests explains why it is a bad idea to give the unit tests access to the database, and how to set up your Rails project so that you get an exception if you try to access the database with ActiveRecord model objects from a unit test. Unfortunately, that blog post doesn't work with the latest Rails. This blog post explains how to set up a Ruby on Rails 2.3 project to disallow database accees in the unit tests. The instructions were tested with Rails 2.3.2.

Step 1. Create file lib/tasks/unit_tests.rake with the following contents:

Rake::Task[:'test:units'].prerequisites.delete('db:test:prepare')

Step 2. Create file test/unit_test_helper.rb with the following contents:

# similar to autogenerated test/test_helper.rb
#
fail "some of the unit tests has loaded test_helper.rb. Please change " +
     "(require 'test_helper') to (require 'unit_test_helper') in " +
     "tests/unit/**/*.rb" if $".include?('test_helper.rb')
fail "some of the unit tests has loaded test_help.rb. Please make sure that " +
     "the first line is (require 'unit_test_helper') in tests/unit/**/*.rb" if
     $".include?('test_help.rb')
require File.expand_path(File.dirname(__FILE__) + "/../config/environment")
HIDE_ActiveRecord = self.class.send(:remove_const, :ActiveRecord)
require 'test_help'  # rails 2.3.2 standard module
ActiveRecord = self.class.send(:remove_const, :HIDE_ActiveRecord)

class FakeConnection
  class InvalidActionError < StandardError
  end
  COLUMNS = {}
  def self.columns(table_name, name=nil)
    if COLUMNS.has_key?(table_name)
      COLUMNS[table_name]
    else
      raise InvalidActionError, "please create something like this first: " +
          "FakeConnection::COLUMNS[#{table_name.inspect}] = [ " +
          "ActiveRecord::ConnectionAdapters::Column.new(name=..., " +
          "default=nil, sql_type=\"text\", null=false), ...]"
    end
  end
  DB_ERROR_MSG = 'You cannot access the database from a unit test'
  def self.quote_table_name(*args)  # called from ActiveRecord::Base.find
    raise InvalidActionError, DB_ERROR_MSG, caller
  end
  def self.quote_column_name(*args)  # called from ActiveRecord::Base.delete
    raise InvalidActionError, DB_ERROR_MSG, caller
  end
  def self.select_all(*args)  # called from ActiveRecord::Base.find_by_sql
    raise InvalidActionError, DB_ERROR_MSG, caller
  end
  def self.transaction(*args)  # called from ActiveRecord::Base.save
    raise InvalidActionError, DB_ERROR_MSG, caller
  end
end
class << ActiveRecord::Base
  def connection
    FakeConnection
  end
end

Step 3. Make sure all your unit tests (i.e. =*.rb= files in =test/unit=, recursively) start with the line require 'unit_test_helper' instead of require 'test_helper'. Don't forget any of the =*.rb= files, otherwise you'll get an error message at test file load time.

Step 4. If you need access to your model objects, add the necessary column declaration to your test file. For example, for model class Foo:

FakeConnection::COLUMNS['foos'] = [
  ActiveRecord::ConnectionAdapters::Column.new(
      name="bar1", default=nil, sql_type="varchar(255)", null=false),
  ActiveRecord::ConnectionAdapters::Column.new(
      name="bar2", default=nil, sql_type="integer", null=false),
]

After this, you can do a Foo.new in your tests – but you won't be able to save the object, and Foo.find, Foo.find_by_sql and Foo.delete won't work either. If you try any of those, a FakeConnection::InvalidAction gets raised. To test such a functionality, use an integration test instead of a unit test. To do so, create your test file as test/integration/*.rb instead of test/unit/*.rb.

Once all steps are done, run your unit tests with rake test:units. This will run all tests in all *.rb files in the test/units directory (recursively). If you get an exception, and you need the full backtrace, rerun with BACKTRACE=1 rake test:units. For an even longer backtrace, run rake --trace test:units.

Java .class file converter which makes method and fields public available

ClassPublic.Java is a Java .class file converter which makes a Java class public and non-final, its fields public and its methods public and non-final. ClassPublic.java has a compact implementation, depending only on a J2SE 1.5.0 or newer. ClassPublic.java can be used to increase interoperability and code reusability of a .class file whose .java source is not available or it is not feasible to recompile.

Download classpublic-latest.zip.

Java class disassembler with offset display available

jdisasm.py is a Java class file disassembler implemented as a Python script. jdisasm.py displays a java .class file in a human readable form, showing the class name, the field names and types, the method names, types and codes (including instruction mnemonics). For each item shown, the file offset is prepended. (Neither javap or jad can display the file offset.)

Download jdisasm-latest.tar.gz.

How to deploy a Ruby on Rails application using Phusion Passenger and Rails Enterprise Edition on Debian Etch or Ubuntu Hardy

This post gives step-by-step instructions on deploying a Ruby on Rails application on a Linux-based server using Phusion Passenger and Ruby Enterprise Edition. The instructions are for a Debian Etch system, but they should work with minor modifications on any Linux system based on Debian or Ubuntu (including Debian Etch Mini, available as Minimal Gnome Desktop from http://www.visoracle.com/download/debian/.

It doesn't matter if you have Ruby installed on the server, because you'll compile and install Ruby Enterprise Edition (REE) 1.8.6 into its own directory, and run the deployed Rails application using REE. REE is Ruby with some garbage collection improvements to allow memory page reuse across fork()ed subprocesses.

So the Ruby interpreter, Ruby packages and Ruby gems already installed to the system won't be used or needed. But some libraries (such as SQLite client and MySQL client) will be used. You have to install those libraries including the development packages using the system's package manager. Example:

# apt-get install libsqlite3-dev
# apt-get install libpq-dev  # optional, needed if Rails app connects to PostgreSQL
# apt-get install libmysqlclient15-dev  # needed if app connects to MySQL

You also have to install some development packages in order to be able to compile and install REE. Do this:

# apt-get install wget gcc g++ make libc6-dev libreadline5-dev zlib1g-dev libssl-dev

Should other packages be missing, the installer script run below will tell you the apt-get install command to run.

You can get the REE installer tarball from http://www.rubyenterpriseedition.com/. Feel free to substitute any newer ruby-enterprise-*.tar.gz. filename to the command below. To download and install REE (including a bundled Ruby, rubygems, Rake, Rails and Ruby client modules for SQLite 3, MySQL and PostgreSQL), run this:

# cd /usr/src
# wget http://rubyforge.org/frs/download.php/55511/\
  ruby-enterprise-1.8.6-20090421.tar.gz
# tar xzvf ruby-enterprise-1.8.6-20090421.tar.gz
# ruby-enterprise-1.8.6-20090421/installer \
  -c--enable-pthread -a/usr/local/ruby-enterprise-1.8.6
# strip /usr/local/ruby-enterprise-1.8.6/bin/ruby

The installer is an interactive script, and it would have asked questions and waited for you to press Enter occasionally if you hadn't called it with the -a flag. Nevertheless, it show you some nice, colorful messages indicating progress. In a few minutes, it finishes compilation and installation to /usr/local/ruby-enterprise-1.8.6. Please note that the installer runs /usr/local/ruby-enterprise-1.8.6/gem install to download, compile and install some gems (Ruby modules). Please note that -c--enable-pthread is necessary to work around a bug in ruby-enterprise-1.8.6-20090421.tar.gz, which causes fork()ed subprocesses to exit early with SIGVTALRM.

Scroll up and check that the MySQL and PostgreSQL Ruby client modules were installed properly. If not (and you need those), then apt-get install the necessary libraries, and run the installer or gem install again. Example command lines (optional, needed only if you need MySQL or PostgreSQL and the installer failed the compile the relavant Ruby modules):

# apt-get install libmysqlclient15-dev
# /usr/local/ruby-enterprise-1.8.6/bin/gem install --no-ri --no-rdoc mysql

# apt-get install libpq-dev
# /usr/local/ruby-enterprise-1.8.6/bin/gem install --no-ri --no-rdoc postgres

If you have any other gems needed by your Rails application, install them now (similarly to the postgres gem above). Please note that all gems should be installed as root in the setup described in this tutorial, and gems are shared among Rails applications.

Create a user (non-root) and download your Rails application to the user's home (e.g. /home/myrails/myapp). The best way to download the application is the checkout function of a version control system (such as Subversion), because using that you can easily re-deploy the application later. (Please note that Capistrano can be used to fully automate Rails application deployment, but this is beyond the scope of this tutorial.) If you have multiple Rails applications, you can create as many users as you want, and map applications to users freely. The application would run in production as the UID of that user (myrails). As myrails, add these lines to /home/myrails/.bash_profile:

export GEM_HOME=/usr/local/ruby-enterprise-1.8.6/lib/ruby/gems/1.8
export PATH="/usr/local/ruby-enterprise-1.8.6/bin:$PATH"

Log in again, and check that type -p ruby gem rake rails prints a filename inside /usr/local/ruby-enterprise-1.8.6. Make sure that all scripts in /home/myrails/myapp/scrit/* start with /usr/bin/env ruby, and they don't have a specific Ruby interpreter (such as /usr/bin/ruby) hardcoded. Create the initial database of your Rails application (by running rake db:migrate etc.). Quickly try your application by starting

/home/myrails/myapp/script/server --environment=production

and visiting http://localhost:3000/ . Try some functionality which accesses the database. As soon as it works fine, stop the server script.

Now it is time to install Apache2 (please note that Apache 1.3 won't suffice, because Phusion Passenger, the Rails--Apache connector this tutorial uses needs Apache 2). Run this:

# apt-get install apache2 apache2-prefork-dev libapr1-dev
# /usr/local/ruby-enterprise-1.8.6/bin/passenger-install-apache2-module -a

The passenger-install script gives you nice, colorful instructions how to configure your Apache. Memorize those instructions, and feel free to use them in place of the instructions given in this tutorial. Configure your Apache2 as usual. (Setting up and populating a DocumentRoot, setting up SSL (https://) and setting up Apache VirtualHost entries is not covered in this tutorial.) Create file /etc/apache2/mods-available/passenger.conf containing the right paths, for example (type it without the line break):

PassengerRoot /usr/local/ruby-enterprise-1.8.6/lib/ruby/gems/1.8/gems
    /passenger-2.2.2
PassengerRuby /usr/local/ruby-enterprise-1.8.6/bin/ruby

Crete file /etc/apache2/mods-available/passenger.load containing the right paths, for example (type it without the line break):

LoadModule passenger_module /usr/local/ruby-enterprise-1.8.6/lib/ruby/gems/
    1.8/gems/passenger-2.2.2/ext/apache2/mod_passenger.so

Run this:

# ln -s ../mods-available/passenger.load /etc/apache2/mods-enabled/
# ln -s ../mods-available/passenger.conf /etc/apache2/mods-enabled/

Restart apache with

# /etc/init.d/apache2 restart

Wait about 60 seconds, and make sure you don't get any Passenger-related error messages at the end of /var/log/apache2/error.log.

If you want to deploy your Rails application to the root URI of an Apache VirtualHost, all you have to do is specifying

<VirtualHost server.name:80>
  ServerName server.name
  DocumentRoot /home/myrails/myapp/public
</VirtualHost>

in your /etc/apache2/sites-available/default. Do not specify RailsBaseUrl /, that wouldn't work. Restart Apache, and visit http://server.name/ . The first page download may take a few seconds, because Phusion Passenger starts the Rails application at that time. The latency of further downloads should be negligible. If you get a colorful, but unhelpful error message like The page you were looking for doesn't exist.; You may have mistyped the address or the page may have moved., then visit http://localhost/ instead, on which Phusion Passenger doesn't hide the exception raised by Rails. Examining /var/log/apache2/error.log can also help you diagnose the problem.

If you want to deploy your Rails application to a non-root URI (e.g. http://server.name/myapp), then first make sure that the necessary workarounds are applied. Follow the instructions on http://ptspts.blogspot.com/2009/05/how-to-fix-railsbaseuri-sub-uri-with.html of adding files /home/myrails/myapp/config/initializers/!fix_relative_url_root.rb and /home/myrails/myapp/config/initializers/!relative_url_for.rb. Make sure those files are added to your source code repository as well, if applicable. Then create a symlink to your application's public directory. Assuming that your Apache2 DocumentRoot is /var/www, create the symlink by running

# ln -s /home/myrails/myapp/public /var/www/myapp

Make sure you have these lines in your Apache2 configuration (most probably /etc/apache2/sites-available/default):

DocumentRoot /var/www
<Directory /var/www>
  Options +FollowSymlinks
</Directory>
RailsBaseUri /myrails

It's OK to have multiple RailsBaseUri directives, both for multiple or a single application. Restart Apache2 if needed. Visit http://server.name/myrails . If you see an unhelpful error message instead of your application's main page, then visit http://localhost/myrails to get the details. Examining /var/log/apache2/error.log can also help you diagnose the problem.

Please note that if you have mutiple Rails applications (i.e. multiple public directories (after following symlinks)), Phusion Passenger will not
reuse the same Ruby worker process for running multiple applications. Please also note that Ruby worker processes spawned by Phusion Passenger are single-threaded. Phusion Passenger treats two Rails applications the same if they have the same public directory (after following symlinks), so the http:// and https:// versions and aliases of your Rails application may run on the same Ruby worker process.

Rails applications running under Phusion Passenger's control in the production environment don't pick up file changes automatically. So if you change your application's code or config, you have to restart the application: run (as myrails)

# touch /home/myrails/myapp/tmp/restart.txt

and visit the application's URL. The first download should take a few seconds, because Phusion Passenger is restarting your Rails application.

Please refer to Phusion Passenger's documentation for more information about running your Rails application with Phusion Passenger with an Apache2 frontend.

How to fix RailsBaseUri (sub URI) with Phusion Passenger without setting relative_url_root

There is a bug which prevents correct routing with Phusion Passenger 2.2.2 and Rails 2.3.2 (>= 2.2.2) and RailsBaseUri. For example, if you have RailsBaseUri myapp, and you visit http://localhost/myapp/mycontroller/myview, then the request gets routed to controller myapp instead of mycontroller, and you most probably get the error message Routing Error; No route matches "/myapp/mycontroller/myview" with {:method=>:get}. If you visit http://myserver/myapp/mycontroller/myview, you'll get the error message The page you were looking for doesn't exist. You may have mistyped the address or the page may have moved.. This post gives instructions how to work around the bug.

The bug is mentioned in various places, such as
http://blog.ashchan.com/archive/2008/12/10/passengers-railsbaseuri-not-working/ and https://rails.lighthouseapp.com/projects/8994-ruby-on-rails/tickets/1946-setting-a-relative-root-url-via-a- and http://github.com/rails/rails/commit/a87462afcb6c642e59bfcd2e11e3351e2b718c38 .

The manual workaround (according to what is suggested on the pages above) is adding the line below to config/environments/production.rb:

config.action_controller.relative_url_root = '/myapp'

We propose a completely automatic workaround here, which doesn't need hardcoding URIs to the Rails application configuration. To fix this, create a file config/initializers/!fix_relative_url_root.rb with the following contents:

# automatic relative_url_root fix
# for Phusion Passenger 2.2.2 and Rails 2.3.2 (>= 2.2.2)
# by pts@fazekas.hu at Mon May  4 20:48:38 CEST 2009
# from http://ptspts.blogspot.com/2009/05/how-to-fix-railsbaseuri-sub-uri-with.html
fail unless ActionController::Request       # check loaded
module ActionController
  class Request
    def initialize(env)
      @env = env  # Rack::Request#initialize does only this
      path = request_uri.to_s[/\A[^\?]*/]
      sn = @env['SCRIPT_NAME']
      if (RAILS_ENV == 'production' and
          (sn.empty? or sn.starts_with?('/')) and
          path == sn + @env['PATH_INFO'])
        Base.relative_url_root = sn
      end
    end
  end
end

In addition to the fix above, here is another fix for url_for and redirect_to so they automatically prepend ActionController::Base.relative_url_root when they get a string starting with a slash. To apply the fix, create file config/initializers/!relative_url_for.rb with the following contents:

# fix url_for and redirect_to to use ActionController::Base.relative_url_for
# fix for Rails 2.3.2
# by pts@fazekas.hu at Mon May  4 22:38:44 CEST 2009
# from http://ptspts.blogspot.com/2009/05/how-to-fix-railsbaseuri-sub-uri-with.html
fail unless ActionController::Base          # check loaded
fail unless ActionView::Helpers::UrlHelper  # check loaded
module ActionController
  class Base
    alias url_for__ptsroot__ url_for
    def url_for(options = {})
      options = Base.relative_url_root.to_s + options if
          options.kind_of?(String) and options.starts_with?('/')
      url_for__ptsroot__(options)
    end
    alias redirect_to__ptsroot__ redirect_to
    def redirect_to(options = {})
      options = Base.relative_url_root.to_s + options if
          options.kind_of?(String) and options.starts_with?('/')
      redirect_to__ptsroot__(options)
    end
  end
end
module ActionView
  module Helpers
    module UrlHelper
      alias url_for__ptsroot__ url_for
      def url_for(options = {})
        return escape_once(
            ::ActionController::Base.relative_url_root.to_s + options) if
            options.kind_of?(String) and options.starts_with?('/')
        url_for__ptsroot__(options)
      end
    end
  end
end

How to fix a broken registry key if Windows XP is not booting

Recently I modified the Windows XP registry key HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control\Nls\CodePage\OEMCP from 437 to 65001, which resulted an immediate Windows XP system crash, and Windows didn't boot anymore, not even in safe mode. This post describes how I changed the key back without a working Windows on the machine.

• I downloaded SystemRescueCD 1.1.7 from http://www.sysresccd.org/Download, burnt it to CD, booted the system from it.
  
• I pressed Enter at the CD boot menu to boot with the default options.
  
• I waited a few minutes for the SystemRescueCD Gentoo system to boot.
  
• At the root@sysresccd /root % prompt (I pressed Alt-F2 at the help screen), I listed the partitions with the command fdisk -l. It turned out that my Windows XP partition was /dev/sda1.
  
• I created a mount point directory with mkdir /mnt/, and mounted the Windows XP partition with mount -t ntfs-3g /dev/sda1 /mnt/p .
  
• I edited the system registry with chntpw -e /mnt/p/WINDOWS/system32/config/system (see the transcript below).
  
• I ensured that changes are written to disk sync.
  
• I rebooted without the CD, and now my Windows XP worked.

Here is the transcript of the chntpw session:

chntpw version 0.99.5 070923 (decade), (c) Petter N Hagen
Hive  name (from header): 
ROOT KEY at offset: 0x001020 * Subkey indexing type is: 686c <lh>
Page at 0x54c000 is not 'hbin', assuming file contains garbage at end
File size 5767168 [580000] bytes, containing 1301 pages (+ 1 headerpage)
Used for data: 103727/5482832 blocks/bytes, unused: 2263/25616 blocks/bytes.

Simple registry editor. ? for help.

> ls
Node has 7 subkeys and 0 values
  key name
  <ControlSet001>
  <ControlSet002>
  <LastKnownGoodRecovery>
  <MountedDevices>
  <Select>
  <Setup>
  <WPA>

> cd \ControlSet001\Control\Nls\CodePage

\ControlSet001\Control\Nls\CodePage> cat OEMCP

Value <OEMCP> of type REG_SZ, data length 12 [0xc]
65001


\ControlSet001\Control\Nls\CodePage> ed OEMCP
EDIT: <OEMCP> of type REG_SZ with length 12 [0xc]
[ 0]: 65001

Now enter new strings, one by one.
Enter nothing to keep old.
[ 0]: 65001
-> 437
newkv->len: 8

\ControlSet001\Control\Nls\CodePage> q

Hives that have changed:
 #  Name
 0  <system>
Write hive files? (y/n) [n] : y
 0  <system> - OK

Please note that a similar procedure (with the exact same fdisk, mount, and chntpw comamnds) using the Knoppix 5.3.1 live CD instead of SystemRescueCD.

Please note that it is possible to edit any registry file with the Windows XP regedit. Here are some relevant links how to do this: http://smallvoid.com/article/winnt-offline-registry-edit.html; http://www.hardforum.com/showthread.php?t=1162302; http://www.911cd.net/forums//index.php?showtopic=7066.

Please note that it is possible to edit the registry with the regedit application shipping with the ERD Commander 2005 Windows XP live boot CD. Get the CD from http://www.fullandfree.info/software/erd-commander-2005/ . There is a Linux tool nrg2iso which can convert the .nrg file in the download for burning to CD. On Windows, it is possible to burn the .nrg file using ImgBurn.

Here are some methods which I tried, but they didn't work to edit the registry system registry file:

• Using kregedit – it crashed opening the system registry file.
  
• Running wine's /usr/bin/regedit – I wasn't able to specify which file to edit.
  
• Running wine regedit.exe with the Windows XP regedit.exe – it failed to start up because it hasn't found some DLLs. I haven't bothered forcing it.

How to fix Ubuntu Hardy and Intrepid boot hang on a Philips laptop

If you have a Philips laptop and you want to boot Ubuntu Hardy and Intrepid on it, the system might hang indefinitely while booting (not reacting to any keypress, not even the Num Lock key). The reason for the hang may be the non-working driver for the notebook's built-in memory card reader (MMC). This post describes the solution: how to disable the driver for the memory card reader and thus get Ubuntu boot. (But you won't be able to use the memory card in Ubuntu.)

If you have another Linux boot CD at hand (such as Knoppix) which boots, and you are familiar with it, then boot it, open file /etc/modprobe.d/local on your Linux root partition, and add the line blacklist mmc_core . Reboot the machine to Ubuntu. Now it should boot normally, without hanging. By doing this you ensure that that the memory card reader driver kernel module won't ever be loaded in the future at boot time. This is a long-term solution, it survives kernel and distribution upgrades.

If you don't have an alternative boot CD at hand, then do the fixing like this. Reboot the system (power-cycle it if necessary). In the GRUB boot menu (a list of long lines starting with Ubuntu 8, press key E, in the new menu, select the line starting with kernel, press key E, press key Space, type init=/bin/bash rw , press key Enter, press key B. This will start the system boot process, giving you a shell soon. In a minute, you'll get a prompt ending with #, and the cursor blinking after that. Type bash -c 'echo blacklist mmc_core >>/etc/modprobe.d/local', press Enter. Type sync , press Enter. Wait a few seconds. Reboot the machine by pressing Ctrl-Alt-Del (or by any other means). Now Ubuntu should boot normally, without hanging.

If the instructions above don't work for you, here is a brute force, temporary fix, which won't survive some system upgrades. Do exactly as above, up to the # prompt. Then type rm -rf /lib/mmc , press Enter. Type mv -fT /lib/modules/`uname -r`/kernel/drivers/mmc /lib/mmc , press Enter. Continue with sync etc.

How to set up an external monitor for watching movies with the nvidia proprietary x.org driver

This post describes how to set up your /etc/X11/xorg.conf on Linux if you have a laptop with an NVIDIA video card, and you want to attach an external monitor for watching movies with MPlayer. Please note that the external monitor is not for regular work, so normally you don't want to put windows there, or use the mouse or keyboard to interact with windows on the external monitor.

How to configure the X server

If you enable TwinView in /etc/X11/xorg.conf (possibly using the command nvidia-xconfig, you'll get one big X11 screen (DISPLAY=:0), with windows possibly spanning two two displays. This is not what you want now. What you want is to have two, independent X11 screens, the laptop's built-in LCD display (DISPLAY=:0.0) running your regular X11 session, and the external monitor (DISPLAY=:0.1) showing the movies played. (Sending audio to the external monitor's speakers is not described here.)

To configure the X.Org X11 server for this, you have to modify /etc/X11/xorg.conf the following way. You have to add a new "Monitor" section, a new "Device" section and a new "Screen" section. First make a backup of your config file: sudo cp /etc/X11/xorg.conf{,.single.good}. Then modify the config file like this:

# modifications to /etc/X11/xorg.conf

Section "Device"
  # This is the original Section "Device"
  Identifier      "..."
  Driver          "nvidia"
  Busid           ...
  Option          ...
  ...
  # ADD Screen 0.
  Screen          0
EndSection

Section "Device"
  Identifier      "nvidia1"  
  Driver          "nvidia"   
  # COPY Busid from original
  Busid           ...
  # COPY Option(s) from original
  Option          ...
  # ADD Screen 1.
  Screen          1
EndSection

# ADD this Monitor section.
# The specified HorizSync and VertRefresh ranges are good for most
# external LCD monitors. You may want to widen them for your monitor.
Section "Monitor"
  Identifier      "monitor1"
  Option          "DPMS"
  HorizSync       28-64 
  VertRefresh     43-60 
EndSection

# ADD this Screen section.
Section "Screen"
  Identifier      "screen1" 
  Device          "nvidia1" 
  Monitor         "monitor1"
  # If it doesn't work with 24, try changing to 32 (and below as well).
  DefaultDepth    24
  SubSection "Display"
    Depth           24
    # SET this to the preferred (maximum) resolution of your external monitor.
    Modes "1920x1080"
  EndSubSection
EndSection

# ADD or modify this ServerLayout section
Section "ServerLayout"
  Identifier  ...
  # Multiple InputDevice entries are OK
  InputDevice ...
  ...
  # SET ... to the name of your original Section "Screen".
  Screen 0 "..." 0 0
  # MAKE sure that the number you specify is larger than
  # the maximum width of your screens. Otherwise the mouse
  # pointer may accidentally wrap one the edge of one screen
  # to another.
  Screen 1 "screen1" 1300 0
EndSection

Make sure that the position you specify for Screen 1 in ServerLayout is large enough, i.e. it is larger than the maximum width of your screens. http://users.tkk.fi/spniskan/switchscreen/ gives the same instructions: Define in the ServerLayout section the second screen's position to be farther away than the first screen's width.. Doing this makes sure that the mouse pointer won't accidentaly wrap from one screen to another when you move it out at the edge of any of the screens.

After modifying /etc/X11/xorg.conf, make a backup copy of the new version (sudo cp /etc/X11/xorg.conf{,.twoscreens.try}), close all applications, connect the external monitor, then restart your X11 session by pressing Ctrl-Alt-<BackSpace>. If it doesn't start up again, or you get some display configuration dialog, then there was something wrong with your modifications to /etc/X11/xorg.conf, and you have to fix it. To do so, press Ctrl-Alt-<F1> to switch to text mode, log in, then stop GDM by running sudo /etc/init.d/gdm stop, then have a look at the startup log file /var/log/Xorg.0.log , and edit /etc/X11/xorg.conf accordingly (you may have to copy it back from /etc/X11/xorg.conf.twoscreens.try first). To try your changes, run X (or sudo X), which displays a black-and-white dotted background on both screens, and an X-shaped mouse cursor on screen 0 (the laptop LCD). Exit by pressing Ctrl-Alt-<BackSpace>. If that one works, you may want to try startx to get a more interesting X session. Exit again by pressing Ctrl-Alt-<BackSpace>. Make a backup copy of the config file (sudo cp /etc/X11/xorg.conf{,.twoscreens.good}, and restart GDM: sudo /etc/X11/gdm restart.

How to prevent the GNOME panel and the icons from appearing on the second screen

Sorry, I don't know the right answer. The panels are drawn by gnome-panel. You can remove all but one, and GNOME will remember it upon relogin. If you remove all panels from a screen, GNOME will recreate the panels with the default configuration :-(. The desktop icons are drawn by nautilus. I have no idea how to disable it on the second screen. The window manager is metacity or compiz (use ps x to find out which is running). You can disable compiz on the second screen by appending the line COMPIZ_OPTIONS="$COMPIZ_OPTIONS --only-current-screen" to ~/.config/compiz/compiz-manager .

How to change the background image

Install ImageMagick with sudo apt-get install imagemagick. Then run display -window root -display :0.1 background.jpg. Please note that this will tile (repeat) copies of the image. You may want to resize the image to fit the screen size: display -window root -display :0.1 -resize 1920x1080 background.jpg, but this may ruin the aspect ratio. Use your favorite image editor (such as GIMP) to create an image of the right size. If you use GNOME, this trick might be useful: http://gnome-hacks.org/hacks.html?id=6 (but it sets the background for multiple screens).

How to move the mouse and switch the keyboard focus to the other X11 screen

There is a handy tool for that named switchscreen. Apparently there are two programs named switchscreen, one by Sampo Niskanen (http://users.tkk.fi/spniskan/switchscreen/) and one by David Antliff (http://en.gentoo-wiki.com/wiki/X.Org/Dual_Monitors#Moving_focus_between_screens and http://unlogical.net/files/scripts/switchscreen-0.4.tar.gz; alternate download for switchscreen.c: http://www.math.bme.hu/~pts/switchscreen.c). We are going the use the latter. Compile it with gcc -s -O2 -W -Wall -L/usr/X11R6/lib -lX11 -lXtst -lXext -o switchscreen switchscreen.c. (On Debian Etch and Ubuntu Hardy, you'll have to install some packages first: sudo apt-get install gcc libc6-dev x11-proto-core-dev x11proto-xext-dev x11proto-xext-dev .) Install with sudo cp switchscreen /usr/local/bin/ .

If you use GNOME, run switchscreen by pressing Ctrl-<F2>, and typing switchscreen plus Enter. You can use your desktop environment's configuration or xbindkeys to bind a key combination to switchscreen.

How to play a movie with mplayer on the external monitor

Run mplayer -display :0.1 -osdlevel 3 -fs movie.avi . If you don't see the the number of seconds elapsed, you may have to configure the MPlayer OSD font. If the OSD text is two large, append lines subfont-text-scale = 3 and subfont-osd-scale = 3 to the file ~/.mplayer/config . If the aspect ratio is wrong on the external monitor, but it is good on the LCD display, specify the flag -monitorpixelaspect X in the mplayer command line above, experimenting with values between 0.5 and 2.0 for X.

How to prevent stderr log messages from interleaving on Linux

Let's suppose you have a program which forks subprocesses, and each subprocess writes log messages to the stderr they share. Since processes run concurrently, it may be possible that the bytes they write interleave (overlap). For example, process A calls write(2, "ABC\n", 4) == 4, and, at the same time process D calls write(2, "DEF\n", 4) == 4. You expect that stderr will contain "ABC\nDEF\n" or "DEF\nABC\n", depending on which write succeeds first. But this is not always the case (!): messages written can interleave, e.g. stderr might contain "ABDCEF\n\n" or something similar. This post gives some recepies how to prevent that, and thus always get either "ABC\nDEF\n" or "DEF\nABC\n".

The instructions given below were tested on Linux 2.6.22 and 2.6.24, but they may probably apply other Linux kernels and other Unices as well.

• Use the write(2) system call to write your log messages. Avoid anything which can buffer or split the message.


• In C, don't use printf(3), fwrite(3), fputs(3) etc.; use write(2)


• In Java 6, don't use System.err.println; use System.err.print, and add a "\n" by hand.


• In Python, don't use sys.stderr.write or print >>sys.stderr; use os.write(2, ...)
.

• In Ruby, don't use $stderr.write or $stderr <<; use $stderr.syswrite. Don't forget to check the return value.


• In Perl, don't use print(STDERR ...); use syswrite(STDERR, ...).


• If stderr is a pipe, make sure you don't write more than 4096 bytes at a time. If you have a longer log message, split it to multiple log messages of at most 4096 bytes (including the newline). This size limit doesn't apply if stderr is a regular file. This size limit has been verified for the kernels above.


• If stderr is a regular file, make sure that it is in append mode when you are writing it.


• Without append mode, even data loss is possible, e.g. instead of "ABC\nDEF\n" you may get "ADBEF\n" (losing "C\n").


• You can ensure append mode by using the append-redirection 2>>file.log instead of 2>file.log when invoking the program.


• In C you can use fcntl(2) to put a file descriptor to append mode: fcntl(2, F_SETFL, fcntl(2, F_GETFL, 0) | O_APPEND). (Don't forget about checking the return values to detect errors.)


• In Python you can use fcntl.fcntl to put a file descriptor to append mode: import fcntl; import os; fcntl.fcntl(2, fcntl.F_SETFL, fcntl.fcntl(2, fcntl.F_GETFL) | os.O_APPEND).


• In Ruby you can use IO#fcntl to put a file descriptor to append mode: require 'fcntl'; $stderr.fcntl(Fcntl::F_SETFL, $stderr.fcntl(Fcntl::F_GETFL) | Fcntl::O_APPEND).


• In Perl you can use Fcntl to put a file descriptor to append mode: use Fcntl; fcntl(STDERR, Fcntl::F_SETFL, fcntl(STDERR, Fcntl::F_GETFL, 0) | Fcntl::O_APPEND).

As a more generic solution (not specific to Unix), you may also consider using mutexes or semaphores to prevent concurrent writes to the same file.

How to convert some LaTeX text to a high resolution PNG image

Create the TeX source file dump.tex like this:

\documentclass{article}
\pdfpagewidth2cm 
\pdfpageheight1cm
\hoffset-2.3cm
\voffset-2.3cm
\begin{document}
\shipout\hbox{$\infty \sum \int$}
\end{document}

Convert it to PDF:

% pdflatex dump.tex

Make sure that pk files don't appear on the console output of pdflatex. If they happen to appear, choose a different font, e.g. \usepackage{lmodern}.

Convert the PDF to PNG with Ghostscript:

% gs -dBATCH -dNOPAUSE -sDEVICE=pngmono -r1000 -sOutputFile=dump.png dump.pdf

If you need a larger resolution image, increase the number after the -r flag above.

This solution generates a PNG with sharp edges (no smoothing). If you take a screenshot of the PDF in a PDF viewer, or you load the PDF directly into GIMP, you'll most probably get a smoothed (antialiased) image.

How to set up ruby gems on Debian Etch as non-root

This tutorial describes how to install Ruby gems to your home directory. The gems you install there will not be visible to other user, and they won't affect the system default configuration. You'll need root access in the first few steps to install Ruby and the rubygems packaging framework itself. It is possible to install those without root access as well, but that's beyond the scope of this tutorial.

You need a working ruby and rubygems. You have to install it as root:

$ su -
# apt-get update
# apt-get install ruby1.8 rubygems
(if you want to install gems which need C compilation, e.g. ruby-sqlite:)
# apt-get install ruby1.8-dev gcc libc6-dev
# ruby -v
ruby 1.8.5 (2006-08-25) [i486-linux]

This is optional, to have the latest rubygems (you can do this any time later):

$ su -
# gem install rubygems-update
...
# /var/lib/gems/1.8/bin/update_rubygems
# rm -f /usr/bin/gem
# ln -s gem1.8 /usr/bin/gem
$ gem -v
1.3.1

All the rest works as non-root:

$ export GEM_HOME=$HOME/gems
$ rm -rf $GEM_HOME
$ mkdir $GEM_HOME{,/cache,/doc,/gems,/specifications}
$ cp -a /var/lib/gems/1.8/cache/sources-*.gem $GEM_HOME/cache/
$ cp -a /var/lib/gems/1.8/gems/sources-* $GEM_HOME/gems/
$ cp -a /var/lib/gems/1.8/specifications/sources-*.gemspec $GEM_HOME/specifications/
$ gem update
Updating installed gems...
Bulk updating Gem source index for: http://gems.rubyforge.org
(this takes a few minutes)
Gems: [] updated

Try querying or installing something:

$ gem search rake --remote
$ gem install rake
Successfully installed rake-0.8.4
Installing ri documentation for rake-0.8.4...
Installing RDoc documentation for rake-0.8.4...

Try installing rails:

$ gem install rails --include-dependencies
(... takes some time)
$ ~/gems/bin/rails -v
2.3.2

Try installing something which needs C code. Please note that you have to install the C prerequisite (libsqlite3) as root:

$ su -c 'apt-get install libsqlite3-dev'
$ gem search sqlite3 --remote
$ gem install --platform ruby sqlite3-ruby
Building native extensions.  This could take a while...
Successfully installed sqlite3-ruby-1.2.4
1 gem installed
Installing ri documentation for sqlite3-ruby-1.2.4...
Installing RDoc documentation for sqlite3-ruby-1.2.4...

Please note that you need the GEM_HOME environment variable set for installing gems and for running applications requiring gems. To have this variable set for you, do this:

$ echo 'export GEM_HOME=$HOME/gems' >>~/.bashrc
$ echo 'export GEM_HOME=$HOME/gems' >>~/.bash_profile

If you need the rails command without having to specify ~/gems/bin/rails, make sure you have an export PATH=$HOME/gems/bin:$PATH in your .bashrc and/or .bash_profile.

How to make Flash full screen work in Ubuntu Hardy and Firefox

We were using the Flash applet on Livescribe's web site in Firefox running on Ubuntu Hardy. We were using Flash 9.0 r124, and Firefox 3.0.7. It had a full screen button, but nothing happened. It was strange, because the full screen button worked a week ago. I've found a solution on http://ubuntulinuxtipstricks.blogspot.com/2007/12/fullscreen-youtube-now-available.html :

If you're using Compiz Fusion, turn off "Unredirect Fullscreen Windows" in the General section and turn off "Legacy Fullscreen Support" in the Workarounds plugin. If you don't, the controls for the player get cut off.

We've found the settings above in our Gnome desktop > System > Preferences > Advanced desktop effects > General options. Turning both settings off solved the problem for us.

ASCII isdigit, isalpha and isxdigit macros in ANSI C with arithmetic operations

The naïve way of defining a C macro which tests whether a character is a digit (assuming an ASCII-based character set) is #define ISDIGIT(c) ((c) >= '0' && (c) <= '9'). There is a fundamental problem with this naïve definition: it evaluates its argument c more than once, so for example ISDIGIT(x++) will increment x by 2 in some cases. The question naturally arises if there is a macro definition #define ISDIGIT(c) ..., which uses c exactly once. Indeed, there is:
#define ISDIGIT(c) ((c) - '0' + 0U <= 9U). The capital Us in the expression enforce unsigned calculation, so if c is less than '0', then (c) - '0' + 0U becomes a large positive number instead of a negative number with small absolute value, so the comparison will (correctly) return false.

The following code contains solutions for ISDIGIT, ISALPHA and ISXDIGIT, the latter not being practical because of the excessive use of arithmetic operations.

/* by pts@fazekas.hu at Thu Mar 26 01:10:56 CET 2009
*
* 0123456789 ISDIGIT
* ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz ISALPHA
* 0123456789ABCDEF ISCAPITALHEX
* 0123456789ABCDEFabcdef ISXDIGIT
*/
#include <stdio.h>

#define ISDIGIT(c) ((c) - '0' + 0U <= 9U)
#define ISALPHA(c) (((c) | 32) - 'a' + 0U <= 'z' - 'a' + 0U)
#define ISCAPITALHEX(c) ((((((c) - 48U) & 255) * 23 / 22 + 4) / 7 ^ 1) <= 2U)
#define ISXDIGIT(c) (((((((((c) - 48U) & 255) * 18 / 17 * 52 / 51 * 58 / 114 \
     * 13 / 11 * 14 / 13 * 35 + 35) / 36 * 35 / 33 * 34 / 33 * 35 / 170 ^ 4) \
     - 3) & 255) ^ 1) <= 2U)

int main(int argc, char **argv) {
  int i;
  (void)argc; (void)argv;
  for (i = 0; i < 256; ++i) if (ISDIGIT(i)) putchar(i);
  printf(" ISDIGIT\n");
  for (i = 0; i < 256; ++i) if (ISALPHA(i)) putchar(i);
  printf(" ISALPHA\n");
  for (i = 0; i < 256; ++i) if (ISCAPITALHEX(i)) putchar(i);
  printf(" ISCAPITALHEX\n");
  for (i = 0; i < 256; ++i) if (ISXDIGIT(i)) putchar(i);
  printf(" ISXDIGIT\n");
  return 0;
}

How to select an ALSA sound card and have concurrent, simultaneus playback using dmix

This blog post is tutorial which describes how to select a default sound card and run multiple playbacks simultaneously, using ALSA on Linux, without a sound server.

The quick trick is having

defaults.pcm.!card Headset
defaults.ctl.!card Headset
defaults.pcm.!device 0
defaults.ctl.!device 0

in your ~/.asoundrc. (Replace Headset with the name of the card on which you want to hear sound. Get the list of available sound cards with aplay -l | awk '/^card/{print$3}'|sort|uniq. To apply this setting for all users, add the lines above to /etc/asound.conf instead.) If this doesn't work for you, please continue reading.

Let's suppose you have a Linux system and many sound cards with an ALSA (>= 0.9) driver, and you don't use any sound server (e.g. pulse, ESD == esound or artsd). The dmix feature of ALSA does software sound mixing, so it makes it possible to run multiple playbacks simultaneously on the same card. ALSA, by default, enables dmix for all cards which don't support concurrent playback of more than one sound stream.

So you just run mplayer -ao alsa file1.mp3, and simultaneously (possibly in another terminal window) mplayer -ao alsa file2.mp3. You should hear both playbacks at the same time. (Please note that you can omit =-ao alsa= from the mplayer command line if you add ao=alsa to your ~/.mplayer/config file.) If the second mplayer doesn't start playback, but exists with an error message containing Device or resource busy, this means there is something wrong with your settings – this tutorial will help to fix that.

If even the first mplayer produces the Device or resource busy error, then close or kill all applications that might play sound. This includes the web browser (with flash), pulse, esd, artsd, mplayer, Skype, MPD and system sounds (disable everything in Gnome / System / Preferences / Sound / Sound). After that, mplayer -ao alsa file1.mp3 should start the first playback, and simultaneously, mplayer -ao alsa file2.mp3 should start the second playback.

If you have multiple sound cards (possibly the sound card in your computer, and an external USB headset), you can set the environment variable ALSA_CARD to direct playback to a specific card. Example 1: ALSA_CARD=Headset mplayer -ao alsa file1.mp3. Example 2: start Firefox as ALSA_CARD=Headset firefox to have the sound Flash movies played on the card named Headset.

You can get a list of sound cards (with some extra information) with aplay -l. For example, on my system, I get

$ aplay -l
card 0: Intel [HDA Intel], device 0: CONEXANT Analog [CONEXANT Analog]
  Subdevices: 1/1
  Subdevice #0: subdevice #0
card 0: Intel [HDA Intel], device 1: Conexant Digital [Conexant Digital]
  Subdevices: 1/1
  Subdevice #0: subdevice #0
card 1: TuxDroid [TuxDroid], device 0: USB Audio [USB Audio]
  Subdevices: 1/1
  Subdevice #0: subdevice #0
card 1: TuxDroid [TuxDroid], device 1: USB Audio [USB Audio #1]
  Subdevices: 1/1
  Subdevice #0: subdevice #0
card 2: Headset [Plantronics Headset], device 0: USB Audio [USB Audio]
  Subdevices: 0/1
  Subdevice #0: subdevice #0

Here is how I can select each card and device:

card 0: Intel [HDA Intel], device 0: CONEXANT Analog [CONEXANT Analog]
  ALSA_CARD=0
  ALSA_CARD=Intel
card 0: Intel [HDA Intel], device 1: Conexant Digital [Conexant Digital]
  ALSA_CARD=0 ALSA_PCM_CARD=1
  ALSA_CARD=Intel ALSA_PCM_CARD=1
card 1: TuxDroid [TuxDroid], device 0: USB Audio [USB Audio]
  ALSA_CARD=1
  ALSA_CARD=TuxDroid
card 1: TuxDroid [TuxDroid], device 1: USB Audio [USB Audio #1]
  ALSA_CARD=1 ALSA_PCM_CARD=1
  ALSA_CARD=TuxDroid ALSA_PCM_CARD=1
card 2: Headset [Plantronics Headset], device 0: USB Audio [USB Audio]
  ALSA_CARD=2
  ALSA_CARD=Headset

Once you have the proper ALSA_CARD and ALSA_PCM_CARD setting, you can add them to your ~/.bashrc and ~/.gnomerc and ~/.xprofile (and possibly to /etc/environment and /etc/X11/Xsession.d/* and /etc/gdm/Xsession). If you log out and log in to your graphic session, you'll have these environment variables by default.

It is possible to select the default sound card without changing setting the ALSA_CARD or ALSA_PCM_CARD environment variables. To do so, add these lines to your ~/.asoundrc:

defaults.pcm.!card Headset
defaults.ctl.!card Headset
defaults.pcm.!device 0
defaults.ctl.!device 0

Each !card line corresponds to the ALSA_CARD value, and each !device line corresponds to the ALSA_PCM_CARD value.

If you set the default in ~/.asoundrc and set the environment variables as well, then the environment variables take effect.

Please note that if you specify any specific ALSA device to any program (other than default), and the program starts playback, then automatic dmix would not work until that program finishes playback and closes the device. This implies that no other program will be able to start playback (but will yield Device or resource busy) until that happens. For example, Skype keeps the sound card open while it is running. So if you want to play sound not coming from Skype while Skype is running, you have to select Default device (default) in Options / Sound Devices for both Sound Out and Ringing. A similar restriction applies to music players and other software: if you specify the playback device for them in their command line or preferences, then the software will lock the sound card, and you lose dmix and concurrent playback. The only dmix-safe ways to select a sound card are the ALSA_CARD etc. environment variables and ~/asoundrc.

To get information about the ALSA cards, run aplay -l and aplay -v -v -L.

ALSA provides OSS emulation (i.e. /dev/dsp, /dev/dsp1), but dmix doesn't work with OSS emulation. To get it work, please run the software which needs OSS using the aoss wrapper, e.g. aoss mplayer -ao oss file1.oss, and concurrently, aoss mplayer -ao oss file2.oss or mplayer -ao alsa file2.oss . If you get the error message /dev/dsp: Device or resource busy from a program, then you'll either have to change it to use ALSA, or run it within aoss.

Here are some sine wave sound generators if you don't have an MP3 ready to test playback with:

perl -e 'print pack("v",32000*sin($_/34))."\0\0"; ++$_ while 1' | aplay -f dat  # Left ear
perl -e 'print "\0\0".pack("v",32000*sin($_/34)); ++$_ while 1' | aplay -f dat  # Right ear

If you have the asoundconf utility, you can use it to set up the default sound card in your ~/.asoundrc. For example, after removing ~/.asoundrc and running
asoundconf set-default-card Headset, you'll get a line <:/home/USERNAME/.asoundrc.asoundconf> in file ~/.asoundrc, and the file ~/.asoundrc.asoundconf would contain more than 50 config lines, the essential ones being

!defaults.pcm.card Headset
defaults.ctl.card Headset 
defaults.pcm.device 0
defaults.pcm.subdevice -1
defaults.pcm.nonblock 1  
defaults.pcm.ipc_key 5678293
defaults.pcm.ipc_gid audio  
defaults.pcm.ipc_perm 0660
defaults.pcm.dmix.max_periods 0
defaults.pcm.dmix.rate 48000   
defaults.pcm.dmix.format S16_LE
defaults.pcm.dmix.card defaults.pcm.card
defaults.pcm.dmix.device defaults.pcm.device
defaults.pcm.dsnoop.card defaults.pcm.card  
defaults.pcm.dsnoop.device defaults.pcm.device
defaults.namehint.extended off

This seems to be too much compared to the 4 lines the beginning of this tutorial suggests.

Random notes

With ALSA 1.0.15, only plughw: works for tuxdroid (so it cannot be used with ALSA_CARD, which implies hw:). Here is how to play: mplayer -ao alsa:device=plughw=TuxDroid file1.mp3 or aplay -D plughw:TuxDroid </dev/urandom. Please note that this restriction applies to both mplayer and aplay. (Maybe that's because dmix was busy when the tuxdroid was connected -- and if we reload ALSA, maybe it will work if I connect the tuxdroid first, and then load the ALSA modules?) The reason why it doesn't work seems to be that the U8 sample format needed by the tuxdroid was introduced only in ALSA 1.0.16. I've verified with libasound 1.0.16 installed (with the same old ALSA kernel) in a chroot, and dmix works with the tuxdroid.

The setting ALSA_CARD=Foo ALSA_PCM_CARD=2 corresponds to aplay -D hw:Foo,2 and mplayer -ao alsa:device=hw=Foo.2. Please note that there is no corresponding environment variable setting for plughw instead of hw. Please also note that mplayer won't use dmix (thus it won't be able to run multiple playbacks concurrently) if you specify any other ALSA setting than mplayer -ao alsa or mplayer -ao alsa:device=default . A similar restriction applies to aplay -D: if you specify any device other than default there, it won't use dmix.

An example full mplayer error message when dmix didn't work:

[AO_ALSA] alsa-lib: pcm_hw.c:1099:(snd_pcm_hw_open) open /dev/snd/pcmC2D0p failed: Device or resource busy
[AO_ALSA] Playback open error: Device or resource busy
Could not open/initialize audio device -> no sound.
Audio: no sound

Using pytz to display a timestamp in a time zone

Download pytz from http://pytz.sourceforge.net/. You can try out the code snippet below without installing pytz. Just create your Python script in the directory where pytz's setup.py lives.

import time  
import pytz  
import datetime  
dt = datetime.datetime.fromtimestamp(time.time(), pytz.utc)  
#tz = pytz.timezone('EST')  
tz = pytz.timezone('CET')  
print tz.normalize(dt.astimezone(tz)).strftime('%Y-%m-%d %H:%M:%S %Z(%z)')