Convert unixtime to date using just the shell

Here’s a simple trick that I use for converting unixtime values via the shell to a readable time format by using the date utility that may not be obvious to some. The idea is to use the date utility’s relative keywords and the difference between the current unixtime and the unixtime you’re interested in.

$ unixtime=$(date -d "2000/01/01" +%s)
$ echo $unixtime
946684800
$ date -d "$(($unixtime-$(date +%s))) seconds" +%Y/%m/%d
2000/01/01

Nice and easy for quick conversions and bash scripts involving unixtime.


Deadhouse Gates


Book #2 of the Malazan Book of the Fallen series by Steven Erickson. To put it plainly this book dragged on a heck of a lot that it almost makes one doubt whether the investment into this 10 book series is a good idea. Deadhouse Gates takes place in a completely separate location to Gardens of the Moon with a very select cast from the first book. The story continues on from the previous title, but focuses on the fork of Kalam et al heading back to the empire via the desert continent of Seven Cities. So for those who were wanting to read more of their favourite characters, such as Whiskeyjack, Ganoes Paran and Quick Ben, will be left wanting till the third novel.

Let’s get the complaints out of the way first. We’re introduced to a new cast of key characters with which to become familiar with (Felisin et al), most of whom are far form likeable. They are all distinct and interesting in their own right, but not endearing enough that we should care for them. To put it simply they are too flawed and antagonistic to garner any sort of attachment. The story itself revolves around a rebellion forming within Seven Cities against the Malazan empire. This sounds all fine and dandy, but there is no real conclusion, in fact the narrative is almost written in a prelude/prologue kind of way to set the stage for following the books. Seven Cities is a desert and the desert’s blandness really wears you down – there’s only so much imagery to be had here – sand, sand, sand… water and some ruins – yay! The final irk is probably the distinct lack of super powers at play this time around. Gardens of the Moon had gods, ascendants, immortals and powerful mages all mixing it up, in comparison Deadhouse Gates is much more focused on the lower beings of Erickson’s world struggling and struggling very hard to get by.

The cool stuff. Coltaine and his Chain of Dogs are just awesome and a real joy to read. This sub-story is probably the highlight of the novel and is what really keeps you interested. It basically follows a Malazan Fist’s struggle to lead his army and train of civilians across the desert continent constantly being hounded by the rebellious forces. Who doesn’t enjoy reading about the age old David versus Goliath struggle. Following Kalam, Apsalar and Crokus around on their little arcs is fun too, mostly because we’ve grown to like them from the first novel and generally because more interesting things happen to them throughout the novel. The other big plus is the introduction of many pivotal characters in Erickson’s world, although never fully explored in Deadhouse Gates, adds lots of reward when they appear again in the following books. It is at that point that things start to click and make sense, till then you’ll just have to take my word for it.

So to sum up, Deadhouse Gates is a necessary stop over in the series, albeit a very drawn out and mentally exhausting one. The chief benefit is that all the materials contained therein is needed to fully appreciate the subsequent novels. Consider it a bit of a history lesson or homework with some parts of good reading. Take solace in the idea that things get really interesting in Book #3 and #4 as a result. A very strange thing that Erickson did, but this is compulsory reading for very odd reasons – longer term gains.


Gardens of the Moon

Not too long ago I set about the task of starting the Malazan Book of the Fallen series after hearing about how it is really one of the diamonds of the fantasy genre, albeit not a series for the unprepared. To be fair, I’ve been aware of it for some time now and only decided on attempting it after being bored or not very captivated with the initial offerings from series by other authors. I was after something meaty, something with substance and something that grows on you the way the good things in life do after you get past the initial tang or bitterness.

Gardens of the Moon, the first in the series, the one that more or less decides whether a reader continues on with the series or drops it flat. Well I did get through it and let me just say, the aura surrounding it is not undeserved. It is definitely a book for people with the willingness to work at absorbing the world that’s thrust in front of them. There is a lot of information flying at you from the get go and the writing assumes you know whats going on. In many respects this is a rather large flaw in the narrative style, but for those willing to persist, there are hints that there is good stuff in the pages and by extension, books to come. Under the surface, behind the sometimes dry prose, you really do get the feeling that there is something special here.

For me that specialness is the promise of a carefully crafted world that will stay with you for more than the usual three books, a world whose innumerable characters get on with living however best they can. With a 10 book series, there’s a helluva amount of adventure awaiting and even from the first few pages you can tell the world is one packed with lore. For some that is enough to justify the investment, myself included.

The story itself is not one of those riveting tales in the sense that one page won’t guarantee to propel you to the next, in fact you need to bulldoze your way through it at times. The book almost reads like a history book that’s unfolding as you go along with many complex threads going on at any one time. A lot of different factions and parties come to play and there are references aplenty to previous events backing the world, all of which add a appreciable level of texture to the book, but at the same time detracts from the immediate enjoyment. To explain, you may be reading along and notice groups like the Malazans, T’lan Imass, Jaghut, Ascendants, Tiste Andii and more generally Gods thrown about the place without much explanation as to where they fit in the grand scheme of things. It is only indirectly that you begin to formulate where they sit based on events and dialogue as you read through. It may well be a hundred pages or more before you can make a fair guess as to how the puzzle (that is the world) all fits together and a further two hundred before something from way back at the beginning starts to make sense. It’s a very strange book in that respect. I believe in the author notes at the start, Steven Erikson himself mentions that he contemplated writing some sort of primer for the book’s world to help ease people into it. In the end he decided against it, simply telling people to read the book thoughtfully to give themselves a chance to absorb the information presented as they go along. In that light, there is a very steep learning curve here.

One other thing I will mention is that Erikson does a really good job creating and developing the innumerable characters that populate the his world. Each one feels very well defined and the core characters that the story revolves around get a lot of development as it moves along. It really feels like these characters are alive in Erikson’s world. It’s quite an achievement considering that there are so many characters in the book and that it is quite the task to keep track of everyone. The strange thing is, even the minor characters do not feel insignificant, rather they are simply just not central to the story.

Despite it’s obvious shortcomings I would still recommend this book, but only to fantasy lovers and only those who really are looking for something rather heavy, almost hardcore. My only advice is to approach it knowing what it is and be ready to work at it a bit, hopefully it’ll also be your cup of tea once things start to click :)


Linux, WINE, CDemu and Mixed-mode CDs – Specifically Total Annihilation

Finally!!! Total Annihilation

One of the things that has always made me almost pull hairs is how to get Linux to properly mount mixed-mode CD images so that the images would work the same as inserting the physical disc. In case you don’t know what mixed-mode CDs are, they are discs that contain both a data track and an audio track (usually several). They were quite a popular format in the 90′s with PC games for storing good quality soundtracks before the arrival of acceptable lossy audio compression ala mp3s/ogg.

A bit of background, but Windows was always blessed with image mounting tools, specifically Daemon Tools. On the Linux side however this was not the case. It was always possible to mount CD images to the filesystem, but this only worked well for purely data ISO images. In the case of mixed-mode discs the audio tracks would be forever hidden even if they are stored in the image :(

This has long been an irk of mine as one of my favourite games to resurrect every couple of years is Total Annihilation. This game is simply timeless. Having purchased a copy towards the late 90′s when it used to make my wee old Pentium 100 cry all the way up till now, it still looks and plays like a modern RTS. A true testament to its creator’s (Chris Taylor) vision. One of the highlights of the game is the orchestral soundtrack that provides a wonderful feeling of theater to the gameplay. After switching over to Linux as my main and usually sole OS in and around 2002, it was always a sad thing to play TA via WINE and not have any audio accompaniment. It’s like watching Star Wars without John Williams… seriously!

Having pulled out TA again just recently I set about doing investigations on how to bring back the music. As it turns out Linux has now got a proper CD image mounting tool in the form of CDemu – Hooray! What makes CDemu work is that it is a proper mounting tool whereby the image itself is presented as it’s own CD-ROM device under the /dev filesystem. For all intents and purposes, from the application’s point of view the image is CD-ROM device which means the images work exactly as their physical counterparts do! Does this mean our thousand unit TA battles now have music? YES, IT DOES!!!

We’re going to have to build the packages manually from source unfortunately since they’re not in the normal repositories. I would upload the deb files for everyone, but that would be too easy and WordPress doesn’t seem to provide a file upload feature. Here are the general steps on how to install CDemu on a Debian 6.x (x86_64) system with a little more detail than what’s hinted to by the CDemu notes.

  1. First we’re going to need to Debian package building tools,
    $ apt-get install dpkg-dev

  2. Next we’ll need the source files of the relevant packages from the CDEmu website. In particular we need to tarballs for:
    • vhba-module (20110416)
    • cdemu-daemon (1.3.0)
    • libmirage (1.3.0)
    • cdemu-client (1.3.0) (there’s also a graphical client, but I don’t really care about that)
  3. Once downloaded, we’ll need to unpack each tarball, they should all create their own directories, but it’s useful at this point to put all the tarballs into a CDemu directory before hand.
  4. First, build the vhba-module by going into the unpacked vhba-module directory and run the Debian build package command.
    $ cd vhba-module-20110416
    $ dpkg-buildpackage -b -tc
  5. At this point you may get an error about missing dependencies, not to worry, just go through and install all the missing dependencies and try again. once the build is done, you’ll end up at about the following place assuming you do not have the package maintainers gpg key (who would?) It’s okay though as the package did get built and stored one level above the vhba-module-20110416 directory
    $ apt-get install list-of-missing-deps
    $ dpkg-buildpackage -b -tc
    ...
    gpg: skipped "Henrik Stokseth (Henrik) ": secret key not available
    gpg: [stdin]: clearsign failed: secret key not available
     
    fakeroot debian/rules clean
    QUILT_PATCHES=debian/patches \
    quilt --quiltrc /dev/null pop -a -R || test $? = 2
    No patch removed
    rm -rf .pc debian/stamp-patched
    dh clean --before auto_clean
    dh_testdir
    dh clean --after auto_clean
    dh_clean
    dpkg-source --after-build vhba-module-20110416
    dpkg-buildpackage: binary only upload (no source included)
    dpkg-buildpackage: warning: Failed to sign .changes file
  6. At this point you should now have the following deb in your cdemu directory:
    vhba-dkms_20110416-0ubuntu1_all.deb. Congratulations, you’re one package closer to mixed-mode heaven. Let’s install it,
    $ dpkg -i vhba-dkms_20110416-0ubuntu1_all.deb
  7. Now we need to repeat the previous steps and build the packages and install for (in this order): libmirage, cdemu-daemon and cdemu-client.
  8. Once all those packages are installed we’re almost ready to rock and roll. To simplify things a cdemu user was created with the daemon installation. Let’s add our user account to that group so that we can mount/unmount images without needing super powers,
    $ usermod -a -G cdemu anh
  9. Let’s start up the CDemu daemon, no point in waiting!
    $ /etc/init.d/cdemu-daemon start
  10. Next we just need to log out of our X session and log back in firstly to let the group addition take effect, but also because the CDemu daemon seems to need it as it is using a session bus (no idea how that works to be fair).
  11. At this point, with any luck (fingerscrossed) you should now be able to run the client to query the daemon’s mount status. It should show that you have two possible emulation devices, both of which do not have anything mounted to them.
    $ cdemu status
    Devices' status:
    DEV LOADED FILENAME
    0 0 N/A
    1 0 N/A
  12. Let’s go ahead and mount something, for example our (my) TA disc image. My image is a Nero image (NRG), but CDemu supports it (it actually supports a lot of formats!)
    $ cdemu load 0 totala1.nrg
    $ cdemu status
    Devices' status:
    DEV LOADED FILENAME
    0 1 /srv/media/iso/totala/totala1.nrg
    1 0 N/A
    $ mount
    ...
    /dev/sr0 on /media/Total Annihilation type iso9660 (ro,nosuid,nodev,uhelper=udisks,uid=1000,gid=1000,iocharset=utf8,mode=0400,dmode=0500)
  13. Wow, it’s mounted the disk using it’s CD label and not only that, but it also appears in the dev filesystem as /dev/sr0. Our very own virtual SATA CD-ROM device, amazing! At this point my Gnome desktop environment picked up the TOTAL ANNIHILATION data disc, but also an audio disc and has placed links to it on my desktop. So far so good! The next step is, how to we make WINE aware of this new found goodness? Initially, I thought setting /dev/sr0 to be the D: dosdevice in winecfg to be the answer, but it didn’t work, which was weird. It seems all that we need to do is set it to the actual mount point of the virtual CD-ROM device, in this case /media/TOTAL ANNIHILATION. Once that was done, starting up TA in WINE allowed it to pick up the audio tracks! Success!
    $ winecfg

Well I hope that helps some of you fans of late 90′s gaming who still revisit them from time to time via WINE on your Linux computers :)


Windows KMS Activation Error – Timestamp is Invalid (0xC004F06C)

For those of you who are using a Windows KMS (key management) within your organisation, you may have come across the following message: The Key Management Service (KMS) determined that the request timestamp is invalid

What this slightly more informative than usual Windows message indicates is that there is a big enough time desync between the KMS host and the client that they get rejected. In an ideal world every host in a network would be synchronising with a common time server of some sort, whether it be external NTP or an domain time server.

The fix in this case is to resync up the client’s time, assuming the KMS server has good time. At minimal, just issue a resync and cross your fingers!
> w32tm /resync

Sometimes this fails because the Windows Time service that’s been configured is somehow broken, in which case we can fix it. The following sets the system to use an external NTP server,
> net stop w32time
> w32tm /unregister
> w32tm /register
> w32tm /config /syncfromflags:manual /manualpeerlist:time.nist.gov
> net start w32time
> w32tm /resync

If the resync finally succeeds, we can hopefully start celebrating pending the following activation command,
> slmgr.vbs /ato

Hope that helps somebody out as it took some digging and trial and error to get this thing working for a customer.

Update (29/Jul/11): I’ve also had success with this method when the following errors are thrown: 0xc004f00f and 0xc004f074

Update (11/Aug/11): Changed the preferred time service to time.nist.gov. It seems like sometimes the Windows Time service group policy may need to be reset as well as follows,
> gpedit.msc

Set the following to Not configured under Local Computer Policy->Computer Configuration->Administrative Templates->System->Windows Time Service:
Global Configuration Settings
Time Providers->Configure Windows NTP Client
Time Providers->Enable Windows NTP Client
Time Providers->Enable Windows NTP Server

Once done, force a group policy update
> gpupdate /force

Some combination of the Group Policy fix and the time service reset should fix this!!!


Windows Port Forwarding/Proxying

There is a little known feature of Windows that lets you do port forwarding within the system. I suppose this is mostly intended to be used when the Windows system happens to be the network gateway. The more useful use however is supporting badly written network applications that for one reason or another only binds on the one (wrong) interface without a means of reconfiguring. Thus, the strategy will be to do a local port forward for example to redirect traffic from the wrong socket to the right socket.

Simple one liner as follows to add a new IPv4 forward rule:
> netsh interface portproxy add v4tov4 listenport=1234 listenaddress=10.0.0.100 connectport=1234 connectaddress=192.168.0.100

Syntax to list the current rules:
> netsh interface portproxy show v4tov4
Listen on IPv4: Connect to IPv4:
Address Port Address Port
--------------- ---------- --------------- ----------
10.0.0.100 1234 192.168.0.100 1234

Syntax to delete a rule:
netsh interface portproxy delete v4tov4 listenport=1234 listenaddress=10.0.0.100

That’s about it really, obviously you can also use this to do port translation and proper port forwarding, etc.

Microsoft’s reference doc for this function can be found here.


Xen DomU pv-grub: “Kernel panic – not syncing”

Just a quick note for those attempting to use custom kernels within a Xen DomU. If you’ve ever gotten the dreadful Kernel Panic – not synching message it usually means the kernel can’t find your root device likely because the initrd image is set to reference a device that may have changed. A sample boot trace as follows:
Creating root device.
Mounting root filesystem.
mount: could not find filesystem '/dev/root'
Setting up other filesystems.
Setting up new root fs
setuproot: moving /dev failed: No such file or directory
no fstab.sys, mounting internal defaults
setuproot: error mounting /proc: No such file or directory
setuproot: error mounting /sys: No such file or directory
Switching to new root and running init.
unmounting old /dev
unmounting old /proc
unmounting old /sys
switchroot: mount failed: No such file or directory
Kernel panic - not syncing: Attempted to kill init!

This happened in my case because the disk device node name changed when the kernel was booted by the Hypervisor (/dev/sda1) compared to how it was configured when it was booted using pv-grub (/dev/xvda1). This is a problem only because when I installed kernel-xen using yum in CentOS the package manager rewrote the initrd image to match the device nodes it saw at the time, not the device that will be present later on.

So where is the problem exactly?
$ cp /boot/initrd-2.6.18-238.12.1.el5xen.img /tmp
$ cd /tmp
$ mv initrd-2.6.18-238.12.1.el5xen.img initrd-2.6.18-238.12.1.el5xen.img.gz
$ gunzip initrd-2.6.18-238.12.1.el5xen.img.gz
$ mkdir initrd-2.6.18-238.12.1.el5xen
$ cd initrd-2.6.18-238.12.1.el5xen
$ cpio -id < ../initrd-2.6.18-238.12.1.el5xen.img
$ tail init
echo Scanning and configuring dmraid supported devices
resume /dev/sda2
echo Creating root device.
mkrootdev -t ext3 -o defaults,noatime,acl,ro /dev/sda1
echo Mounting root filesystem.
mount /sysroot
echo Setting up other filesystems.
setuproot
echo Switching to new root and running init.
switchroot

As can be seen there are two references to /dev/sda in the init script, hence our little problem! So how to fix? A quick way would be to modify the init script to reference /dev/xvda in the appropriate places, repackage and replace the original. The nicer solution is to switch to using device labels on the DomU.
$ e2label /dev/sda1 root
$ mkswap -L swap /dev/sda2
$ yum remove kernel-xen

First we just need to edit the /boot/grub/grub.conf and /etc/fstab to use the device labels.
# /etc/fstab
LABEL=root / ext defaults,noatime,acl 1 1
LABEL=swap swap swap defaults 0 0

# /boot/grub/grub.conf
default=0
timeout=5
 
title=Centos 5.5 2.6.18-238.12.1.el5xen
root (hd0)
kernel /boot/vmlinuz-2.6.18-238.12.1.el5xen ro root=LABEL=root panic=0
initrd /boot/initrd-2.6.18-238.12.1.el5xen.img

Once that’s all done, reinstall the Xen kernel and the initrd should get generated to reference the labels rather than the device nodes making the device not found issue moot.
$ yum install kernel-xen
...
$ tail init
echo Scanning and configuring dmraid supported devices
resume LABEL=swap
echo Creating root device.
mkrootdev -t ext3 -o defaults,noatime,acl,ro LABEL=root
echo Mounting root filesystem.
mount /sysroot
echo Setting up other filesystems.
setuproot
echo Switching to new root and running init.
switchroot

Now when we reboot the system to use pv-grub the system should boot into our shiny internal kernel. This allows the VM’s administrator more control over their Linux instance without really needing to interact with the host system. Horray!


Getting a Remote Desktop on a Headless Debian Server

Sometimes it’s nice to get a nice remote desktop when working on a Linux system. Not all the time, but sometimes. On those rare sometimes, it’s nice to have something nicer than twm. So here are some relatively minimal steps to get a working Gnome remote desktop on a Debian Lenny server that you don’t actually want to install an X server on.

Install the required packages. Notice we’re using gnome-core since we really do not need the entire Gnome environment… this is true even for dedicated desktops, bloat anyone? ;)
apt-get update
apt-get install vnc4server
apt-get install gnome-core

Now we just need to quickly start and stop a vncserver session so we’ll have the required .vnc scripts and configuration in our home directory.
vncserver
vncserver -kill :1

Next, we add the following to the end of our user’s xstartup script to include loading of a session manager,
echo "x-session-manager &" >> ~/.vnc/xstartup

The resulting xstartup script should read like the following,
#!/bin/sh
 
# Uncomment the following two lines for normal desktop:
# unset SESSION_MANAGER
# exec /etc/X11/xinit/xinitrc
 
[ -x /etc/vnc/xstartup ] && exec /etc/vnc/xstartup
[ -r $HOME/.Xresources ] && xrdb $HOME/.Xresources
xsetroot -solid grey
vncconfig -iconic &
x-terminal-emulator -geometry 80x24+10+10 -ls -title "$VNCDESKTOP Desktop" &
x-window-manager &
x-session-manager &

Lastly, we start up our VNC server and that’s it!
vncserver

These steps should work for other desktop environments as well, we just need to update the alternatives to the relative window and session (if one exists) managers. For Gnome it looks like the following,
$ ls -l /etc/alternatives/x-{session,window}-manager
lrwxrwxrwx 1 root root 22 2011-05-24 13:39 /etc/alternatives/x-session-manager -> /usr/bin/gnome-session
lrwxrwxrwx 1 root root 17 2011-05-24 13:39 /etc/alternatives/x-window-manager -> /usr/bin/metacity

Well that’s that really. Goes without saying however that real admins use screen or twm ;)


The Wise Man’s Fear

The Wise Man's Fear cover

Note: I read this book back in March 2011, so this review is looking back a few months after.

My fantasy book choices seem to always fall towards the grittier mature themed novels, the obvious example that everyone should have read being George R R Martin’s A Song of Ice and Fire series. More recently I’ve taken up the challenging and exciting task of chewing through Steven Erickson’s Malazan Book of the Fallen series starting from the very beginning.

No one can doubt the absolute pleasure it was to read Patrick Rothfuss’s The Name of the Wind. It was literally a breath of fresh air. The prose of Rothfuss is a dream to read, his words seem to have a duty of care that carefully carries the reader from page to page. Another nicety of Rothfuss is that he doesn’t dwell on gore, risque themes or morally disconcerting events to push along his narrative. At the same time however, the reader doesn’t feel like the novel is dumbed down for less mature audiences, it is still very much a book for adults. This style sits in stark contrast to some of his modern day peers who seem to enjoy pushing the boundaries of what is bearable and although for the most part they do provide interesting reading, it usually requires a mental fortitude and a certain mood to actually be enjoyable. So it is in this way that The Name of the Wind has started this supposed trilogy and we arrive at the much anticipated second book, The Wise Man’s Fear.

Firstly, I must confess that I didn’t catch onto the first book until about eight months before the second book was due to be released. I feel a little spoilt in this regard, being able to enjoy Kvothe’s tale unfolding in rapid succession (unlike a certain of series whose continuation has finally managed a release date only after about five years of delays!)

So how was it? Well consider that I purchased the book one cold London day in the middle of March and basically spent the next four days (or just over) huddled up indoors keeping warm and reading the book from start to finish. It was pretty good, but the long and short of it is that The Wise Man’s Fear does not truly feel like a sequel. The main reason was that nothing much really happened to Kvothe that actually advances the story forward, at least not as far nor as fast as the pace set by the first book. So that’s the bad news. The good news is that the second book brings Rothfuss back again and again we are treated to more of his enjoyable writing. Granted that the events in the second book seem almost episodic, but they’re still a fun romp to read and the pages really do fly if you let them, my example being a case in point (though it did help that I was in between jobs at the time).

My recommendation is to grab a copy, especially if it’s a bargain bin print a year or more later. Heck, I managed to snag the first book for $13AUD, my best value novel purchase so far! Just keep in mind that Kvothe’s life isn’t as much of a roller coaster this time around, which was much of the charm of the first book’s tale, in a way he has become more settled in comparison. The Wise Man’s Fear is definitely still the Rothfuss trademark enjoyable read however, so no reason not to read it if you have nothing else on your plate.

Cover courtesy of Wikipedia


Hello world!

Obligatory first post titled from a programmer’s context.

Wee… here we go!!!


Follow

Get every new post delivered to your Inbox.