Thursday, March 24, 2011

Not as constant as one might think

One of those Ruby gotchas that has to lead to a bug report before it finally burns itself to memory:

irb(main):001:0> TMP="1234"
=> "1234"
irb(main):002:0> t = TMP
=> "1234"
irb(main):003:0> t.succ!
=> "1235"
irb(main):004:0> TMP
=> "1235"

Eh? Isn't TMP a constant, after all?

Apparently this has to do with t being assigned a reference to a (shared) String:

irb(main):005:0> a="456"
=> "456"
irb(main):006:0> b=a
=> "456"
irb(main):007:0> b.succ!
=> "457"
irb(main):008:0> a
=> "457"

Needless to say, this will do the trick:

t = TMP.dup

This will raise an error on t.succ!, for obvious reasons :


Moral: "Constants" are NOT frozen and their values are not copied on assignment!

Wednesday, March 23, 2011

Using git to synchronize and backup home directories

With git-home-history gone, and nothing suitable to take its place, one must handroll a decent solution to version controlling a home directory.  Since git can synchronize repositories, it should also be possible to use it to synchronize the contents of a home directory on multiple machines.

This example will assume that a desktop computer (hostname 'desktop')contains the master ('remote') repository, and that a laptop (hostname 'laptop') contains the slave (local) repository.

NOTE: It is important to be aware of which files should be left out of the repo. Because a laptop and a desktop will have different graphics hardware, the settings for GUI applications such as web browsers and window managers should not be in the repo. Also, files which change a lot (cache files, .bash_history, etc) should be kept out of the repo as they will always cause merge conflicts. Finally, private keys should be left out of the repo.

Finally, need it be said that the home directories of both machines should be backed up before trying this?

Desktop: Create and fill the repository

To begin with, create an empty git repo:

bash$ cd ~
bash$ git init .
bash$ touch .gitignore
bash$ git add .gitignore

Next, modify the .gitignore file to select which files or directories to leave out of the repo:

bash$ vi .gitignore

This example ignores backup, swap, and core files, as well as directories that probably shouldn't be shared between the two machines (Desktop, Downloads, Templates, tmp, mnt). Note that all hidden files are left out of the repo by default (.*).

Now add all allowed files into the repo:

bash$ git add .

Now add exceptions to the .gitignore file. These will be config files that are shared between the two machines:

bash$ git add -f .bashrc .xsessionrc .vimrc .gvimrc .gdbinitrc  .ssh/config .local/share/applications

If Firefox was configured correctly (i.e. by making a tarball of ~/.mozille/firefox on the desktop machine and extracting it to ~ on the laptop machine, instead of letting Firefox generate its own config), then the bookmarks file can be added as well:

bash$ git add -f .mozilla/firefox/*.default/bookmarks.html

This of course holds true for non-config data in the Firefox dir, such as .mozilla/firefox/*.default/ReadItLater (UPDATE: but not zotero, as it updates itself even while it is not being modified) .

Finally, commit all of the contents to the repo:

bash$ git commit -m 'Initial home dir checkin'

The git directory now has a starting version of the home directory checked in. It can be reviewed with a tool such as QGit to  ensure nothing is missing or unwanted:

bash$ qgit &

NOTE: To make the following operations go smoothly, the following line must be added to .git/config :

    denyCurrentBranch = false

Desktop : Create a script to auto-commit

At this point. it is useful to create a shell script that performs a commit in the background.

bash$ mkdir -p bin
bash$ vi bin/
cd ~
git add .
git commit -m 'automated backup' . 
bash$chmod +x bin/

Laptop : Clone the repository

On the laptop, clone the repository from the desktop:

bash$ cd ~
bash$ mkdir -p tmp/git-repo
bash$ cd tmp/git-repo
bash$ git clone desktop:/home/$USER

Note that the repo was cloned to a temporary directory so that it will not overwrite any local files. This is important!

Move the git metadata directory to the home directory:

bash$ cd $USER
bash$ mv .git ~

Retrieve any missing files (i.e. that exist on the desktop but not on the laptop, such as .gitignore) from the repository:

bash$ cd ~
bash$ git checkout \*

Laptop: Add local changes

Create a branch for the changes that will be made next:

bash$ git checkout -b laptop

Add any local exclusions to the .gitignore file:

bash$ echo .pr0n >> .gitignore

Add any additional files to git:

bash$ git add TODO NOTES

Commit the branch:

git commit -m 'laptop additions'

Now merge the branch into master:

bash$ git checkout master
bash$ git merge laptop

Verify that the changes are suitable:

bash$ qgit &

Finally, push the changes to the desktop:

bash$ git push

Desktop: Generate canonical file versions

The desktop will now have all its files set to the laptop versions.

At this point, files that have been modified should be reviewed and editted, so that a canonical version will be stored in the repo and used by both the desktop and the laptop. QGit makes the review process fairly simple.

Note that some config files will have to source local config files that lie outside the repository (i.e they are excluded in /gitignore). For example, .bashrc might have a line like

[ -f ~/.bash_local.rc ] && . ~/.bash_local.rc

...and .vimrc might have a line like

if filereadable(expand("$HOME/.vim_local.rc"))
    source ~/.vim_local.rc

The files .bashrc_local.rc and .vim_local.rc will be listed in .gitignore, and will have machine-specific configuration such as custom prompts, font size (e.g. in .gvimrc), etc.

Once the canonical versions of the files have been created, they are committed :

bash$ git commit -, 'canonical version' .
bash$ git tag 'canonical'

Laptop: Pull canonical versions

The canonical versions can now be pulled down to the laptop. Note that any supporting files (e.g. .bash_local.rc) will have to be created on the laptop.

bash$ git pull

Laptop & Desktop : Add cron job

In order for git to automatically track changes to the home directory, both the laptop and the desktop will need to add a cron job for running .

The following crontab will run git every two hours:

bash$ crontab -e
0 */2 * * * /home/$USER/bin/ 2>&1 > /dev/null

...of course $USER must be replaced with the actual username.

Note: Some provision must be made for pushing the laptop repo to the desktop. This can be done in a cron job, but is probably better suited to an if-up (on network interface up) script.

UPDATE: Be careful when pushing; the desktop must be forced to update its working tree, or its next commit will delete files on the laptop. The following script will do the trick:

cd $HOME

git push && ssh desktop 'git reset --merge `git rev-list --max-count=1 master`'

Of course passwordless ssh should be set up for this to work. A similar problem exists when pulling from the server: a "git checkout \*" must be performed to create any missing files.

Desktop: Add backup script and cron job

At this point, a backup script and cron job can be added to the desktop server. The directory ~/.git is all that needs to be backed up; a shell script can rsync it to a server.

Thursday, March 10, 2011

Running .desktop files from the command line

After making a few .desktop files with custom commands in them (e.g.RXVT settings), it is useful to be able to run those same commands from the occasional terminal.

When KDE is installed, the kioclient can be used for this purpose:

kioclient exec file:/PATH_TO_DESKTOP_FILE

Custom desktop files are stored in ~/.local/share/applications, so this is easy to wrap in a shell function for inclusion in ~/.bashrc :

xdg-exec () {
    kioclient exec file:${HOME}/.local/share/applications/${1}.desktop


bash$ xdg-exec rxvt-unicode

Thursday, March 3, 2011

Ruby, Qt4, and AbstractItemModel

There are no good examples of using QAbstractItemModel in Ruby. For most purposes, QStandardItemModel will suffice. In this particular case, which calls for a lazily-loaded tree, QStandardItemModel will not cut it.

What follows is a simple implementation. The number of columns is limited to 1, items are read-only, there is no DnD support, and the model only supports the Display role for item data. Needless to say, these features are easy enough to implement, and would only distract from the example of subclassing Qt::AbstractItemModel.

The Model

=begin rdoc
A basic tree model. The contents of all tree nodes are determined by the ModelItems, not the Model, so they may be loaded lazily.
class Model < Qt::AbstractItemModel
  signals 'dataChanged(const QModelIndex &, const QModelIndex &)'

=begin rdoc
The invisible root item in the tree.
  attr_reader :root

  def initialize(parent=nil)
    @root = nil

=begin rdoc
Load data into Model. This just creates a few fake items as an example. A full implementation would create and fill the top-level items after creating.
Note: @root is created here in order to make clearing easy. A clear() method just needs to set root to'').
  def load()
    @root ='',nil)'First', @root)'Second', @root)

=begin rdoc
This treats an invalid index (returned by as the index of @root.
All other indexes have the item itself stored in the 'internalPointer' field.

See AbstractItemModel#createIndex.
  def itemFromIndex(index)
    return @root if not index.valid?
=begin rdoc
Return the index of the parent item for 'index'.
The key here is to treat the invalid index (returned by as the index of @root. All other (valid) indexes are generated by AbstractItemModel#createIndex. Note that the item itself is passed as the third parameter (internalPointer) to createIndex.

See ModelItem#parent and ModelItem#childRow.
  def index(row, column=0,
    item = itemFromIndex(parent)
    if item
      child = item.child(row)
      return createIndex(row, column, child) if child

=begin rdoc
Return the index of the parent item for 'index'.
This is made a bit complicated by the fact that the ModelIndex must be created by AbstractItemModel.

The parent of the parent is used to obtain the 'row' of the parent. If the parent is root, the invalid Modelndex is used as usual.
  def parent(index)
    return if not index.valid?

    item = itemFromIndex(index)
    parent = item.parent
    return if parent == @root

    pparent = parent.parent
    return if not pparent

    createIndex(pparent.childRow(parent), 0, parent)

=begin rdoc
Return data for ModelItem. This only handles the case where Display Data (the text in the Tree) is requested.
  def data(index, role)
    return if (not index.valid?) or role != Qt::DisplayRole
    item = itemFromIndex(index)
    item ? :

=begin rdoc
Set data in a ModelItem. This is just an example to show how the signal is emitted.
  def data=(index, value, role)
    return false if (not index.valid?) or role != Qt::DisplayRole
    item = itemFromIndex(index)
    return false if not item = value.to_s
    emit dataChanged(index, index)
  alias :setData :data=

=begin rdoc
Delegate rowCount to item.

See ModelItem#rowCount.
  def rowCount(index)
    item = itemFromIndex(index)
    item ? item.rowCount : 0

=begin rdoc
Only support 1 column
  def columnCount(index)

=begin rdoc
All items can be enabled only.
  def flags(index)

=begin rdoc
Don't supply any header data.
  def headerData(section, orientation, role)

The ModelItem

=begin rdoc
An example of a ModelItem for use in the above Model. Note that it does not need to descend from QObject.

The ModelItem consists of a data member (the text displayed in the tree), a parent ModelItem, and an array of child ModelItems. This array corresponds directly to the Model 'rows' owned by this item.
class ModelItem
  attr_accessor :data
  attr_accessor :parent
  attr_reader :children

  def initialize(data, parent=nil)
    @data = data
    @parent = parent
    @children = []
    parent.addChild(self) if parent

=begin rdoc
Return the ModelItem at index 'row' in @children. This can be made lazy by using a data source (e.g. database, filesystem) instead of an array for @children.
  def child(row)

=begin rdoc
Return row of child that matches 'item'. This can be made lazy by using a data source (e.g. database, filesystem) instead of an array for @children.
  def childRow(item)

=begin rdoc
Return number of children. This can be made lazy by using a data source (e.g. database, filesystem) instead of an array for @children.
  def rowCount

=begin rdoc
Used to determine if the item is expandible.
  def hasChildren
    childCount > 0

=begin rdoc
Add a child to this ModelItem. This puts the item into @children.
  def addChild(item)
    @children << item

This should serve as a basic implementation of an AbstractItemModel.

Realistically, ModelItem would be subclassed to represent different types of items in the data source, each of which would also (likely) be subclassed from ModelItem. This allows a browsable tree to be created for navigating data hierarchies.