2011-11-18

Measure your reaction time using Python

Run this in a terminal window (without the leading $):

$ python -c 'import random, time; time.sleep(2 + 8 * random.random()); \
  print 1; t = time.time(); raw_input(); print time.time() - t'

Press Enter as soon as the number 1 appears. Your reaction time (in seconds) will be printed. For best results, don't do it over an SSH connection. If you can go below .2 second, then you are most probably not human.

2011-11-13

Announcing Portable MariaDB: Small, portable binary MariaDB distribution for Linux

Portable MariaDB is a small, portable binary distribution of the SQL server MariaDB (Monty's fork of MySQL) for Linux i386 (32-bit). Only the mysqld binary and a versatile init script are included. Portable MariaDB can be run by any user in any directory, it doesn't try to access any mysqld data or config files outside its directory. Portable MariaDB can coexist with regular mysqld (MySQL or MariaDB) and other instances of Portable MariaDB on a single machine, as long as they are not configured to listen on the same TCP port. The only dependency of Portable MariaDB is glibc 2.4 (available in Ubuntu Hardy or later).

The most up-to-date documentation of Portable MariaDB is here.

The sources are here.

Why use Portable MariaDB?

  • It's small (not bloated). Fast to dowload, fast to extract, fast to install. Quick size comparison: mariadb-5.2.9-Linux-i686.tar.gz is 144 MB, the corresponding Portable MariaDB .tbz2 is less than 6 MB.
  • It's portable: does not interfere with other MySQL server installations on the same machine.
  • It's self-contained and consistent: copy the database and the configuration in a single directory from one machine to another.

Installation

To run Portable MariaDB, you need a Linux system with glibc 2.4 (e.g. Ubuntu Hardy) or later. 32-bit and 64-bit systems are fine. For 64-bit systems you need the 32-bit compatibility libraries installed. You also need Perl.

  $ cd /tmp  # Or any other with write access.
  $ BASE=https://raw.githubusercontent.com/pts/portable-mariadb/master/release
  $ #OLD: wget -O portable-mariadb.tbz2 $BASE/portable-mariadb-5.2.9.tbz2
  $ wget -O portable-mariadb.tbz2 $BASE/portable-mariadb-5.5.46.tbz2
  $ tar xjvf portable-mariadb.tbz2
  $ chmod 700 /tmp/portable-mariadb  # For security.
  $ /tmp/portable-mariadb/mariadb_init.pl stop-set-root-password

Usage

For security, don't do anything as root.

  $ cd /tmp/portable-mariadb
  $ ./mariadb_init.pl restart
  Connect with: mysql --socket=/tmp/portable-mariadb/mysqld.sock --user=root --database=test --password
  Connect with: mysql --host=127.0.0.1 --user=root --database=test --password

Feel free to take a look at /tmp/portable-mariadb/my.cnf, make modifications, and restart mysqld so that the modifications take effect.

Security

By default, connections are accepted from localhost (Unix domain socket and TCP) only, all MySQL users are refused (except if a password has been set for root above), and root has unrestricted access. Unix permissions (such as the chmod 700 above) are protecting against data theft and manipulation on the file level.

It is strongly recommended to change the password of root to a non-empty, strong password before populating the database.

Java support

Java clients with JDBC (MySQL Connector/J) are fully supported. Please note that Java doesn't support Unix doman socket, so make sure in my.cnf that mysqld listens on a TCP port. Please make sure you have ?characterEncoding=UTF8 specified in your JDBC connection URL, otherwise some non-ASCII, non-Latin-1 characters would be converted to ?.

Unicode support

Just as with MariaDB. All encodings and collations are supported. The latin1 encoding is the default, which can be changed in my.cnf.

Language support

All natural languages (of MariaDB) are supported for error messages. Set the `language' flag in my.cnf accordingly. English is the default.

2011-11-10

How to simply compress a C++ string with LZMA

This blog post explains how to simply compress C++ with LZMA compression, using liblzma.

Use the following functions:

#include <stdlib.h>
#include "lzma.h"
#include <string>

// Level is between 0 (no compression), 9 (slow compression, small output).
std::string CompressWithLzma(const std::string& in, int level) {
  std::string result;
  result.resize(in.size() + (in.size() >> 2) + 128);
  size_t out_pos = 0;
  if (LZMA_OK != lzma_easy_buffer_encode(
      level, LZMA_CHECK_CRC32, NULL,
      reinterpret_cast<uint8_t*>(const_cast<char*>(in.data())), in.size(),
      reinterpret_cast<uint8_t*>(&result[0]), &out_pos, result.size()))
    abort();
  result.resize(out_pos);
  return result;
}

std::string DecompressWithLzma(const std::string& in) {
  static const size_t kMemLimit = 1 << 30;  // 1 GB.
  lzma_stream strm = LZMA_STREAM_INIT;
  std::string result;
  result.resize(8192);
  size_t result_used = 0;
  lzma_ret ret;
  ret = lzma_stream_decoder(&strm, kMemLimit, LZMA_CONCATENATED);
  if (ret != LZMA_OK)
    abort();
  size_t avail0 = result.size();
  strm.next_in = reinterpret_cast<const uint8_t*>(in.data());
  strm.avail_in = in.size();
  strm.next_out = reinterpret_cast<uint8_t*>(&result[0]);
  strm.avail_out = avail0;
  while (true) {
    ret = lzma_code(&strm, strm.avail_in == 0 ? LZMA_FINISH : LZMA_RUN);
    if (ret == LZMA_STREAM_END) {
      result_used += avail0 - strm.avail_out;
      if (0 != strm.avail_in)  // Guaranteed by lzma_stream_decoder().
        abort();
      result.resize(result_used);
      lzma_end(&strm);
      return result;
    }
    if (ret != LZMA_OK)
      abort();
    if (strm.avail_out == 0) {
      result_used += avail0 - strm.avail_out;
      result.resize(result.size() << 1);
      strm.next_out = reinterpret_cast<uint8_t*>(&result[0] + result_used);
      strm.avail_out = avail0 = result.size() - result_used;
    }
  }
}

Please note that in some use cases there may exist a solution which uses less memory.

See the liblzma/container. for documentation of lzma_easy_buffer_encode().

The decompression code was based on xzdec.c in xz-utils.