My favorite android apps

I have been using android for a month, and I do think it a great OS, from a geek perspective, of cause ;) Since Google does not provide paid app in Chinese market, I got only free apps, however some of them are already great, let me name some of them:

• AndFTP is a powerful yet easy to use FTP client.
• Android Terminal Emulator is definitely essential on rooted box (oops, can I call a mobile phone as box just like a PC?) Without it customizing your system would be much tougher.
• Barcode Scannner by ZXing Team is the fastest scanner I found in market, and recognizing URI with camera is such a great invention!
• Batteryminder tells me in percentage how much battery left, which is a must on a phone with a power-hungry large screen just like my milestone...
• diviClock is a cute desktop clock widget offering shortcuts to your alarm clock and calendar
• NetCounter is used to track how much traffic is spend in a day/week/month because CMCC's traffice is so expensive - 7MB for 1 USD!!!
• Jorte provides various widget and calendar view, and connects to both Google calendar and a Japanese(?) calendar service.
• TaskOS To Do List: writing TODO list in voice is great, hopefully web syn feature will be available soon.
• Vplayer, though still in alpha stage, plays a great range of format of video.

At last I would be very happy if Google brings paid games, though I can get robo defense free and Angry Birds from Chinese market, and they are very interesting (and very time-consuming, too).

web is a mess…where is the hope?

Recently I am helping designing and coding a web site, which I thought would be a small project… I did know about the difference between old IE and other “standard” browsers and some hacks and work-arounds, but I failed to realize it would be so painful to do all things “right”.

In stead of insisting everything should look exactly the same in all browsers, I think it is natual that some design details could degrade in older browsers, like round corners and text shadow. The real troublemaker turns form elements, each browser (depending on the operating system and even system theme) displays them differently – an awful nightmare :( There aren’t any standard to style them, the only solution is to employ JavaScript to hide the original form and render a fake one with div and spans. Thus select element becomes an unsorted list with a CSS’s and scripts, which seems very bizarre…

Maybe it is time to re-invent the web. People have been talking about this for decades, but I see little progress. I don’t believen HTML5 is a solution, though it’s really hot. Semantics is important, but I don’t believe it the key issue. Web pages are no longer simply documents, but applications, and we are distributing service instead of content. JavaScript has many bad parts but is blessed as well, devil or angel, it should be the core of web. HTML should be fundamentally changed to be a better present layer of JavaScript. A lot of functions carried out by browser natively should be given to JavaScript or other future client-side code. With more power and freedom of client code, coding for web could certainly bring more good things to users.

Serving static file in nginx with a some check

Recently I developed  a small web site for my lab, and we are hosting it on Google App Engine (GAE), as Google provides quite enough resources for a site of several pages ;)

However we also need to provide download of huge files, which can not be store in GAE, thus we had to host it in one of our severs. Initially we do not want our content directly accessible, so we want to use dynamic URL but we don’t want to loose the performance of nginx.

My final solution is use rsa module by Sybren A. Stuvel (which is pure python) to encrypt the real url (with fixed private key) in GAE, and give the encrypted url to user. In the file host, I use nginx’s X-Accel-Redirect feature and a tiny cherrypy server to decrypt the url (with corresponding pub key), and simply set the X-Accel-Redirect header to the real internal path of nginx.

This solution works well in my case :)

Mount qemu harddisk image in host system

Recently I got to copy a lot of files from a qemu guest, and I think using network is too slow, so I googled for solutions to mount qemu image in host system (when guest is shutdown). And it turns out that only raw images are supported, therefore the first step is to convert the image format:

qemu-img convert –f qcow2 origin.qcow –O raw temp.raw

Then I fount in archlinux wiki that I should mount with –loop,offset=32256 option, but I fail to figure out where 32256 comes. Luckily I remenbered how I delt with dd images. Just get the offset with fdisk:

fdisk –u –l temp.raw

Then I have the sector size (512 in my case) and the offset (2048, start sector of my partation), so the offset should be 512*2048=1048576. (btw, you can calculate with bc: echo ‘512*2048’ | bc)

mount –t ext4 –o loop,offset=1048576 temp.raw /mnt/some-path

That’s it :)

Win 7 Knows Every Mouse?

Even my scope node?

Cool!

Link for X, HAL & udev

It seems we could thank HAL for all these years of great work and say goodbye in a short period of time (source below).

To be honest I still feel confused now and then about what (fill this blank) is doing for me. My understanding is:

linux kernel ─>─ udev ─>─ HAL ─>─ X server
                   │                │
                   └────────>───────┘
                         libudev

So I just try to find more about X/Hal/udev thing, and keep some links here. (Remember the best source of information is always the official web sites and the mailing list).

(Not) blaming HAL by Keith Packard (explains what X.org X server needs from HAL before 1.8)

Manage Linux Hardware with udev by Carla Schroder (tells a little history of udev and HAL, and how /dev, /sys works.)

* source: Free X.org Server 1.8.0 with udev to replace HAL

How to delete last 10 lines in a file with script

I find this question asked very frequently... So here is the present of day(not for the April 1st, it's already 2nd in China now ;)

$ tac file.in | sed '1,10d' | tac > file.out

Obviously, you could replace 10 with any number you like, Cheers!

Use LibSVM in WEKA in an Archlinux box

I found both WEKA and libSVM in AUR, however, neither maintainer considered that a user might use them together... It seem we Archie got to work it out ourselves, so here is what I did:

Dowload WEKA tarball from AUR, and Change weka.sh to

#! /bin/sh
# Note: the '-Xmx256m' flag is used to set the maximum memory that the java
# program is allowed. When working with large data sets, this number may need
# to be increased based on how much memory you wish to allow weka to have.
# Specifying command line arguments will override the defaults shown here.
# Example: '-Xmx512m'
DEFAULT_ARGS='-Xms32m -Xmx256m'
if [ $# -gt 0 ]; then
ARGS="$@"
else
ARGS="$DEFAULT_ARGS"
fi
java -classpath
"${CLASSROOT+CLASSROOT:}/usr/share/java/weka/weka.jar:/usr/share/java/libsvm.jar"
weka.gui.GUIChooser $ARGS

Do not forget to change the MD5SUM of weka.sh in PKGBUILD file :) Modify LibSVM PKGBUILD to:

# Contributor: Pierre Gueth
# Modified by: Yuanjie Huang
pkgname=libsvm
pkgver=2.9
pkgrel=2
pkgdesc="A library for Support Vector Machines classification, include
binaries and python bindings."
arch=(x86_64 i686)
url="http://www.csie.ntu.edu.tw/~cjlin/libsvm/"
license=('GPL')
groups=()
depends=()
makedepends=()
optdepends=()
provides=()
conflicts=()
replaces=()
backup=()
options=()
install=
source=("http://www.csie.ntu.edu.tw/~cjlin/cgi-bin/libsvm.cgi?+http://www.csie.ntu.edu.tw/~cjlin/libsvm+tar.gz")
noextract=()
md5sums=('c48109c825d8326d71c8c8564589736d')
build() {
cd "$srcdir/$pkgname-$pkgver"
make || return 1
install -D -m755 svm-train $pkgdir/usr/bin/svm-train
install -D -m755 svm-predict $pkgdir/usr/bin/svm-predict
install -D -m755 svm-scale $pkgdir/usr/bin/svm-scale
install -D -m644 java/libsvm.jar $pkgdir/usr/share/java/libsvm.jar
cd "python"
python setup.py build || return 1
python setup.py install --root $pkgdir
}

Then makepgk and install, have fun.

Where is mendeley user config stored?

Mendeley Desktop is a nice academic research resource management tool. However I messed its configuration up, and got to reset it. But there is no such a directory as .mendeley as I expect in my home directory, so I got to find it like this:

for f in `ls -a1F | grep -E '^\.[A-Za-z]+/$'`; \
do find $f -print | grep -E '[Mm]endeley'; done

Then simply delete everything I got, and bring Mendeley back happily ;)

Linux Hibernation in short

In order to hibernate in Linux, you need a user space script and an backend. Two User space scirpts widely used are:

• pm-utils : used in OpenSuse, Ubuntu, etc
• hibernation-scirpt : used by gentoo, etc

The linux kernel comes with hibernate backend named swsusp but you can also try other tool, so your options are:

• uswsusp : need to be installed
• tuxonice : you got to patch the kernel
• swsusp : comes without extra effords

My choise is pm-utils + uswsusp for I’m too lazy to patch and compile a kernel, and I’ve been using pm-utils for a long time.

And one more thing to mention, even hibernation-script comes with tuxonice, it can use uswsusp as backend.

GAE Hello world error

Google App Engine supports SSL now, however Python 2.5 does not come with SSL socket in stock :(

gae$ /usr/local/bin/dev_appserver.py \
~/projects/hello/
Traceback (most recent call last):
File "/usr/local/bin/dev_appserver.py", line 50, in <module>
execfile(script_path, globals())
File "/usr/local/google_appengine/google/appengine/tools/dev_appserver_main.py", line 338, in <module>
sys.exit(main(sys.argv))
File "/usr/local/google_appengine/google/appengine/tools/dev_appserver_main.py", line 297, in main
server = MakeRpcServer(option_dict)
File "/usr/local/google_appengine/google/appengine/tools/dev_appserver_main.py", line 259, in MakeRpcServer
host_override=option_dict[ARG_ADMIN_CONSOLE_HOST])
File "/usr/local/google_appengine/google/appengine/tools/appcfg.py", line 114, in __init__
self.opener = self._GetOpener()
File "/usr/local/google_appengine/google/appengine/tools/appcfg.py", line 317, in _GetOpener
opener.add_handler(urllib2.HTTPSHandler())
AttributeError: 'module' object has no attribute 'HTTPSHandler'

To resovle the problem you got to recomile Python with SSL support as instructed by Patrick Altman and then follow a regular  configure – make – make install routine.

Then you need to Edit the Modules/Setup.dist to uncomment a couple of lines:

# Socket module helper for SSL support; you must comment out the other
# socket line above, and possibly edit the SSL variable:
SSL=/usr/local/ssl
_ssl _ssl.c 
-DUSE_SSL -I$(SSL)/include -I$(SSL)/include/openssl 
-L$(SSL)/lib -lssl -lcrypto

links for customizing GNOME menu

• Customizing Menus
• Desktop Menu Specification
• Placing Entries in the Applications Menu
• Themable Application-specific Icons
• Icon Theme Specification / Directory Layout
• Tango Icon Theme Guidelines (to make 24x24 icons from 22x22 ones)

libpng12.so.0 missing problem in Archlinux

Archlinux has been offering libpng14 recently, however if some self-compiled packages are used in the system, it may turn out an disaster :(

I had cario, freetype2 & libxft compiled from abs so as to enable sort of cleartype effect in type rendering, however, after I updated libpng recently from version 12 to 14, I cannot boot into gdm correctly. I cannot even run vim in a console for it complains that libpng12.so.0 cannot be found.

So I have to:

1. boot with run level 3 (adding 3 at the end of grub's kernel line);
2. login as root and remove self-compiled packages that require libpng12, cario, freetype2 & libxft in my case (I found -d option useful since I will soon reinstall these packages);
3. recompile new versions with libpng14;
4. install those packages.

Using English fonts in Windows 7 with Chinese local setting

Windows 7 comes with Lucida Console and Consolas as its default console fonts, but I find no option to enable either of them when my location is set to China. After a few tries, I managed to get them via the following steps:

1. Right click the console icon and go to the default menu item;
2. Set the default code page to '437 OEM';
3. Switch to the font tab, and change the font to Lucida Console, press OK button;
4. Restart the console and go to the code page setting back to '936 Chinese'.

Idea for a simple bib tool

I was using Zotero for my research readings, however, it is a bit too heavy for me, and I want a tool that can be portable across machines. So I am planning to write a bib tool for myself when I am a bit free sometime next spring.

The most important concept in this tool:

• Keep it Simple: there will NOT be fancy setup interface, but a config XML; there will not be many export styles for that is latex’s business.
• Depend less: there is already many advanced tools for bib organization, but they depend on this and that. I plan to implement all this tool with Python and sqlite3, so it can be put into U-key/Dropbox, and used anywhere.

The basci senarios is:

1. User start the tool, which set up a local web sever at a high port (eg: 8080), and open any browser to land on the interface.
2. To add a new paper, user have to
   1. upload a pdf file
   2. add bibtex info by
      1. upload a crospending a bibtex file
      2. fill a form to generate bibtex info
   3. add optional tags
   4. add optional note
3. To view paper:
   1. start a search (empty field means any)
   2. a list of reference is returned
   3. (optional) open pdf file
4. To export
   1. start a search
   2. select paper and add to “basket”
   3. go to export folder and get a sorted bib for all and BIB’s for each files and copies of PDF’s

Suggestions are always welcomed :)

SumatraPDF is really FAST

I have reading a lot of pdf files recently, and I often open a file, read a handful paragraphs and close the file. While Adobe Reader provides great render quality, it is a bit slow.

So I turned to SumatraPDF, which is an open source pdf reader based on MuPDF and brings several useful functionalities, such as searching and copy.

What in_lto_p has _p suffix in GCC 4.5

Abstract: Curiosity killed the cat XD

Usually we use _p suffix to pointers in C language, but it is not the case for in_lto_p in GCC 4.5, so I searched for the reason, and according to <http://www.stanford.edu/~blp/writings/blp-stds/blp-stds_4.html> :

p-convention

A _p suffix indicates a boolean test for the condition described by the rest of the name. (This comes from Lisp.)

And it can be supported by this wiki page<http://en.wikipedia.org/wiki/P_convention> :

This practice originated among users of the Lisp programming language, in which there is the convention of appending the letter “P” on elements to denote a predicate (a yes or no question). It is most commonly used at MIT and the University of California, Berkeley, or among computer scientists working in Artificial intelligence (which frequently uses Lisp).

So _p for predicate not pointer here :)

Bash environment memo: profile, bash_profile & bashrc

Abstract: I'm trying to explain the process of bash setup in this article.

Keywords:bash, shell, environment

It is usually hard to answer the question where this variable is set, because there is SO many different files involved. So let's start with some concepts:

interactive login shell

The shell started after a successful login, using /bin/login, by reading the /etc/passwd file.

interactive non-login shell

It is normally started at the command-line using a shell program (eg.$/bin/bash) or by the /bin/su command. It is also started with a terminal program within a graphical environment.

non-interactive shell

It's usually used to execute a shell script

An interactive login shell generally initializes with the following configuration files:

/etc/profile -> [~/.bash_profile] -> [~/.bash_login] -> [~/.profile]

In some systems, the /etc/profile script will look for additional settings in /etc/profile.d/*.sh, and the ~/.bash_profile usually sources ~/.bashrc.

When an interactive non-login shell starts, it first copies the environment of its parent shell and then reads its settings in the following order:

/etc/bash.bashrc -> [~/.bashrc]

Remember that only exported variables of its parent shell is copied, functions and aliases of its parent are NOT inherited.

A non-interactive shell simply copies environment from its parent, and does not read any configuration files.

Mount internal filesystem without passwd with udisk (formerly devicekit-disk)

Abstract: Use policykit to avoid root password for mount an internal filesystem.

Last week, I updated my Arch Linux laptop, and got the new deviekit-disk. Then I found that the authorizations set with gnome interface, which actually sets theorg.freedesktop.hal.storage..., stops work. I have to give the root passwd when I try to mount an internal filesystem, which is really bothering:(

Luckly it is not hard to solve this problem with Policykit, though there dosen't seem to be a sweet GUI to help. All you have to do is simply create a text file named anything with suffix .pkla in /etc/polkit-1/localauthority/50-local.d/ and fill it with the content below:

[filesystems mount internal privs]
Identity=unix-group:disk
Action=org.freedesktop.devicekit.disks.filesystem-mount-system-internal
ResultAny=no
ResultInactive=no
ResultActive=yes

In case you have udisks2 instead of udisks, the action should written like this:

Action=org.freedesktop.udisks2.filesystem-mount-system;org.freedesktop.udisks2.filesystem-mount

Here I allow all users in group disk to mount internal filesystem without root password. disk is the default group of storage devices, you should add yourself to this group. Otherwise you can specify your login name:

Identity=unix-user:yourname

To see all possible actions, check /usr/share/polkit-1/actions directory.
You can always man pklocalauthority for more info:)

---

Update (May 22nd 2012)

Udisks2 uses different action name.

---

Update (Oct 25th, 2011)

Latest udev will set storage devices of group disk. So I specify group disk instead of storage now.

PS: I moved to a new blog.

---

Update (April 3rd 2010)

People from freedesktop.org are making a great progress - by renaming devicekit-disk as udisk (Sounds talking about flash disk, isn't it?)... So you got to change Action=org.freedesktop.devicekit.disks.filesystem-mount-system-internal to Action=org.freedesktop.udisks.filesystem-mount-system-internal.

Dear sirs/madams from freedesktop.org, will you please stop renaming things? It's not fun, at all :(

---

Update (April 16th, 2010)

It seems /etc/polkit-1/localauthority/50-local.d/ is a better place to apply your own settings.

---

Update (Nov 2nd, 2010)

The configuration file has to be named with suffix .pkla to be recognized, for more details you can man pklocalauthority. And lots of thanks goes to Awebb who kindly points this out :)

looking for a web software for knowlege sharing

I am currently looking for a web-based software for knowledge sharing in our lab, I've investigated some blog, cms and also wiki softwares, but none seems to fit :(

We just want something which is easy to setup and maintain, and can organize different kind of materials. CMS's are powerful, but relatively hard to used...Wiki seems to lack some kind of structure. Blogs are easy to use, but does not organize resources well.

So what do you use in your lab, my dear readers?