QScintilla and getSelection()

The recent Ubuntu upgrade and subsequent Ruby woes were caused, of course, by the desire to install libqscintilla-ruby, a package only available on 10.10 (and even then, only for 1.8).

For the most part, QScintilla works fine in Ruby... until one encounters methods like this:

void getCursorPosition(int *line, int *index) 

void getSelection(int *lineFrom, int *indexFrom, int *lineTo, int *indexTo)

The docs provide some hint as to the problem:

If there is a selection, *lineFrom is set to the line number in which the selection begins and *lineTo is set to the line number in which the selection ends. (They could be the same.) *indexFrom is set to the index at which the selection begins within *lineFrom, and *indexTo is set to the index at which the selection ends within *lineTo. If there is no selection, *lineFrom, *indexFrom, *lineTo and *indexTo are all set to -1.

How does one pass an integer by reference in Ruby?

The answer: one doesn't. These functions take integer arguments and return nil, making them entirely useless in Ruby.

The Python guys did it right:

line_fro, idx_fro, line_to, idx_to = getSelection

The Ruby guys, of course, were lazy, and routed all calls directly to libqscintilla.so regardless of the sanity of their argument lists.


There is a way to make things work, however, thanks to ScintillaBase.


This, the base class of the Scintilla widget, provides the following method:

long SendScintilla(unsigned int msg, unsigned long wParam=0, long lParam=0)

At the top of  the base class documentation are a bunch of enums that look promising:

...

 SCI_SETSELECTIONSTART = 2142,   

 SCI_GETSELECTIONSTART = 2143,
 SCI_SETSELECTIONEND = 2144,  

 SCI_GETSELECTIONEND = 2145, 

...


Sure enough, these turn out to be the values for the msg parameter.


It makes for short work to add the following methods to the Scintilla object:

def get_sel_start

    # SCI_GETSELECTIONSTART

    self.SendScintilla(2143), 0, 0)

end

def get_sel_end

   # SCI_GETSELECTIONEND
    self.SendScintilla(2145), 0, 0)
end

def get_current_pos
    # SCI_GETCURRENTPOS
    self.SendScintilla(2008), 0, 0)
end

Yes, the messages have to be passed by number, as the symbols for the bulk of the messages have not been defined as constants in Ruby (lazy! bad! lazy!), as can be determined by examining Qsci::ScintilaBase.constants.

Ubuntu getting crappier and crappier

Stupidly made the decision to upgrade to 10.10 (for a single package!) on the main laptop (mbpro 5,5) and poof! No wireless!

Seriously, it's been three releases since an ethernet cable was required to do an upgrade.

Piece.
Of.
Shit.

Only question now is how many hours of productivity are going to be lost fixing what *was* a perfectly working system before the upgrade.

The sad fact is that os x and windows are just as unreliable for upgrades (while being less usable), and FreeBSD refuses to even glance at this hardware.

A decade ago, this shit used to *work*!

Running log of the fixes:

* apt-get install gnome-icon-theme to get NetworkManager running again. There is no fix for Wicd; it has apparently "stopped working".

* apt-get install gnome-alsa-mixer to un-mute the sound. Kmixer has been reduced from 7 or so channels to 1 (without a mute option).

* uninstall ruby 1.9 via apt-get. Download and install rvm (system-wide as this is a workstation), then do 'sudo rvm --default use 1.9.2' to make 1.9 the system-wide default (might roll that back later).

* reinstall all 1.9 gems.

* rebuild the passenger apache module ('sudo passenger-install-apache2-module') and update the config files to point to the new location.

* fix all rack-based webapps to include the line
    $: << File.dirname(__FILE__)
in config.ru and in each Sinatra::Base application file, as something got screwed in the ruby-passenger-rack environment, and PassengerRoot is no longer in the Ruby module path.

See? A smooth, seamless upgrade! You almost don't even notice that it happened!

Apache + Passenger + Sinatra

OK, this seems like a pretty normal situation, and like most normal situations it apparently never occurs in nature (judging by the available docs).

An existing Apache webserver is to have a new webapp added to it, in a subdirectory of the DocumentRoot. Sinatra is the framework to be used, and Passenger is going to route requests from Apache to Sinatra without going through any mod_proxy or mod_rewrite business.

Assume the following: Apache2, an Ubuntu system, and all of the relevant gems have been apt-get installed. The web server root directory is /var/www, and the webapp will be in /var/www/timon.

First, update the Apache config file (/etc/apache2/apache2/conf):

<VirtualHost *:80>
  ServerName Apemantus
  DocumentRoot /var/www

  <Directory />
    Options -Indexes
    AllowOverride None
  </Directory>

  <Directory /var/www/>
    AllowOverride AuthConfig
    Order allow,deny
  </Directory>

  SetEnv RUBYLIB '/var/www/timon'
  PassengerEnabled on
  PassengerAppRoot /var/www/timon
  RackBaseURI /timon
</VirtualHost>

The bold lines indicate what must be added.

• SetEnv : This just allows the code in subdirectories of the webapp to be easily required.
• PassengerEnabled : Seems fairly obvious.
• PassengerAppRoot : The full filesystem path to the webapp directory.
• RackBaseURI : The relative (to DocumentRoot) path to the webapp directory.

Next, verify that the Passenger options (/etc/apache2/mods-enabled/passenger.conf) are correct:


<IfModule mod_passenger.c>

  PassengerRoot /usr
  PassengerRuby /usr/bin/ruby
  PassengerMaxPoolSize 10
  PassengerDefaultUser www-data

</IfModule mod_passenger.c>

These options are pretty straightforward. The only thing to note is that PassengerRuby can be set to a specific version of Ruby, e.g. jruby or ruby1.9.

Now, create an empty Rack-friendly directory structure for the webapp:

bash# cd /var/www

bash# mkdir timon

bash# mkdir timon/public

bash# mkdir timon/tmp

The use of public for static pages and tmp for restart.txt is well-documented.

Next, a Rack config file must be provided. This will be named config.ru (/var/www/timon/config.ru) and will have the following contents:

#!/usr/bin/env ruby

require 'rubygems'

require 'sinatra'

# Disable Sinatra's default Webrick instance

set :run, false

# Include timon.rb, the main webapp script

require 'timon'

run Sinatra::Application

Finally, the app itself must be provided. This will be named timon.rb (/var/www/timon/timon.rb) and will have the following contents:

#!/usr/bin/env ruby

require 'rubygems'

require 'sinatra'

get '/' do

  "<b>TIMON!</b>"

end

# Local 404 handler

not_found do

  "Timon NotFound exception"

end

# Local error handler

error do

  "Timon Error: " + env['sinatra_error'].name

end

Debugging can be made a bit more straightforward by adding some basic logging code to config.ru:

#!/usr/bin/env ruby

require 'rubygems'

require 'sinatra'

# Disable Sinatra's default Webrick instance

set :run, false

# Local logging

FileUtils.mkdir_p 'log' unless File.exists?('log')

log = File.new('log/sinatra.log', 'a')

$stdout.reopen(log)

$stderr.reopen(log)

# Include timon.rb, the main webapp script

require 'timon'

run Sinatra::Application

That's it.

Nothing much to it, really, but the lack of RackBaseURI in the relevant examples really makes debugging this kind of thing difficult.

Ruby CGI and Javascript

After a brief perusal of the Ruby CGI module documentation, it doesn't seem like there is a good way to generate Javascript from within it.

Due to the method_missing way that the CGI module handles HTML tags, however, it turns out to be quite simple: invoke cgi.script, passing all tag parameters as a hash, and pass the Javascript code as a string in the block:

  cgi.out {
    cgi.html {
      cgi.head {
        cgi.title {
          "Test"
        } +

        # Javascript library to include
        cgi.script( 'src' => 'dygraph-combined.js',
                    'type' => 'text/javascript') {
        }
      } +

      cgi.body {
        cgi.br +
        cgi.div( 'id' => 'graphdiv' ) +
        cgi.br +
        cgi.script( 'type' => 'text/javascript') {

          # Javascript to execute
          'g = new Dygraph(
                  document.getElementById("graphdiv"),
                  "Date,Temperature\n" +
                  "2008-05-07,75\n" +
                  "2008-05-08,70\n" +
                  "2008-05-09,80\n"
              );
          '
        }

      }

    }

internationalization regex

Just a quick vim regex that's useful when adding internationalization to a file:

:%s/\("\([^\\]\\"\|[^"]\)\+"\|'\([^\\]\\'\|[^']\)\+'\)/gettext(\1)/

Wraps all quoted strings in a gettext() call, preserving inner (and escaped) quotes. Doing this in vim instead of sed allows a cursory review of the changes in case any of them * shouldn't* be changed (e.g. log strings).

Indeed.

I was astonished to learn how accomplished both of my children are in programming. It is a skill that an entire generation of adolescents is learning underground, much the way we used to pick up dirty words.

-- S. Milgram, 1982

Controlling a command-line interpreter in Ruby

There seem to be a lot of posts to mailing lists, forums, etc in regards to controlling a child process in Ruby via STDIN and STDOUT. Most of the discussion (with this notable exception) ends with "use open3!" or "use open4!".

Anyone who has ever tried to control (i.e., tried to send more than one command to) an interpreter using either of these recognise their shortcomings immediately: the child process STDOUT cannot be read until STDIN has been closed.

Fortunately, the pty module (helpfully mentioned in the notable exception) allows the interpreter to be controlled properly as long as the OS supports psuedo-terminals -- that is, as long as it is a UNIX-like OS (i.e. Linux, OS X, *BSD... basically every desktop/server OS but Windows).

It is a bit tricky to get working well, due to the problem of not knowing for sure whether the child process is preparing more data to write to STDOUT, or is waiting for another command on STDIN.

The following implementation wraps the praat speech analysis software. It uses the praat prompt ('Praat > ') to determine when the child process is ready for more input (i.e., when it can stop reading from STDOUT).

#!/usr/bin/env ruby

require 'pty'

module Praat

  class Interpreter
    attr_reader :stdout, :stdin, :pid

    PROMPT='Praat > '

    def initialize(program='praat')
      @stdout, @stdin, @pid = PTY.spawn( 'praat', '-' )

      # Read initial prompt from pipe
      read_until_prompt

      if block_given?
        yield self
        @stdin.close
        @stdout.close
        @stdin = @stdout = @pid = nil
      end

    end

    def read_until_prompt
      outbuf = buf = ''

      # Read from child STDOUT until > 0 bytes have been read
      # (i.e. wait for child process to finish reading input)
      while buf.length == 0
        begin
          IO.select([@stdout])  # block until child process is ready
          @stdout.read_nonblock( 1024, buf )
        rescue Exception => e
          buf = ''              # READ failure! Try again.
        end
      end

      # Read from child STDOUT until 0 bytes are read or a line ending in a
      # prompt (i.e. next input prompt) was encountered.
      while buf.length > 0
        outbuf << buf

        # complete read if next input prompt is encountered
        break if outbuf =~ /#{PROMPT}$/

        begin
          buf = ''
          @stdout.read_nonblock( 1024, buf )
        rescue Errno::EAGAIN => e
          IO.select([@stdout])  # block until child process is ready
          retry
        rescue Exception => e
          buf=''                # READ failure. Exit loop.
        end

      end

      # Return output of interpreter as an array of lines
      return outbuf.split("\n").each { |x| x.chomp! }
    end

    # Send a command to the interpreter. Returns an array of the output.
    # If include_prompts is true, lines beginning with a prompt will NOT be
    # stripped from the output.
    def send( command, include_prompts=false )
      @stdin.write(command + "\n")

      outbuf = read_until_prompt

      # Ignore all ECHOed lines before the first (input) prompt
      first_prompt = outbuf.find_index { |x| x =~ /^#{PROMPT}/ }
      result = outbuf.slice(first_prompt, outbuf.length-first_prompt)

      # Return full results, or the results with prompt lines removed
      result = result.select {|x| x !~ /^#{PROMPT}/} if not include_prompts

      yield result if block_given?

      return result
    end

  end

end

if __FILE__ == $0

  puts 'Testing block implementation'
  Praat::Interpreter.new() { |p| puts p.send('echo BLOCK TEST') }

  Praat::Interpreter.new() do |p|
    p.send('echo Full Output', true).each { |line| puts "\t" + line }
  end

  puts 'Testing object implementation'
  praat = Praat::Interpreter.new()

  lines = []
  lines.concat praat.send('echo OBJ TEST 1' )
  lines.concat praat.send('echo OBJ TEST 2' )
  lines.concat praat.send('echo OBJ TEST 3' )

  puts lines.inspect
end