Dispatches from the X1 front

The first-gen Thinkpad X1: possibly the best laptop I've ever owned. Lasted three years, compared to the usual 12-18 months. It still runs, but the battery is down to about 40 minutes, and the (second!) keyboard is starting to lose keys.

The third-gen Thinkpad X1 Carbon: a little annoying at first: no SD slot, no h/w kill switch for the wireless, no spill-through keyboard (wtf guys, that was brilliant!). About 6 months in, the video glitches started: there is a short which corrupts video memory while using wifi. The fix, of course, is a replacement system board, which is great if you have a spare third-gen X1 Thinkpad Carbon lying around and can just up and change workstations like it's no big. Workaround: rfkill block wifi, or just get used to the strobe effect.

The latest: Thinkpad X1 Yoga. And let's just put it out there: this doesn't work out of the box like the previous two did. Lenovo and Intel seem to have backed off of their Linux commitment, though that's probably half their users these days (errybody else on the mobile, right?)

Here's the rundown on getting the X1 Yoga up and running with a Linux install.


Basic Linux

Debian 8.3.0 AMD64 netinst ISO if you were wise enough to buy the OneLink+ RJ45 ethernet adapter; otherwise, use the regular (non-netinst) ISO.

Use unetbootin to make a USB flash drive image for this. Go into the BIOS on the X1 Yoga, enable F12 for boot device selection (just because), turn off secure boot, maybe enable diagnostic boot as well. No need to allow legacy BIOS installs, Debian will work just fine with uEFI (unlike FreeBSD where you need to use the 10.3-Beta or 11 memstick). Go through the install like you usually do. Your ethernet (if you have it) will be detected (again, unlike in FreeBSD).

Sound, APCI (suspend, hibernate, and battery status) and video card should all be working out of the box.


Wireless

Upgrade your kernel to 4.4.1 from the ubuntu PPA:
  http://kernel.ubuntu.com/~kernel-ppa/mainline/v4.4.1-wily/
This is per notes about a typo in the iwlwifi driver at
 https://bbs.archlinux.org/viewtopic.php?id=206831
Looking at the kernel source, 4.2.7 has the fix as well, but might as well go with the latest stable release of the kernel if we're upgrading.

You'll also need to upgrade the firmware, or your Wifi will not be recognized. There is a debian package in backports for this:
 https://packages.debian.org/jessie-backports/firmware-iwlwifi


Skylake P-State Fix

If you rebooted excitedly into your new kernel without following the rest of these steps, you're likely looking at a blank screen. That's because of a skylake p-state bug reported here:
 https://www.reddit.com/r/thinkpad/comments/410haw/yoga_260_wont_boot_ubuntu_with_kernel_4244_but/

This is because the kernel needs a command line parameter to not totally crap itself when it encounters Skylake's poor implementation of pstate.

To fix this, change the cmdline in /etc/default/grub to:
  GRUB_CMDLINE_LINUX_DEFAULT='intel_pstate=no_hwp quiet'

Don't forget to run update-grub when you're done.

Hopefully this will go away in 4.5; according to a commenter, that's what the kernel guys are trying for.


i915 Xorg Video fix

Of course, the fancy new kernel hosed your X windows, didn't it?

Not sure why there's such a tight coupling there, but never mind. This one's an easy fix, per http://comments.gmane.org/gmane.linux.debian.user/508779 . Download (and install) a new version of the xserver-xorg-video-intel package from backports:

https://packages.debian.org/jessie-backports/xserver-xorg-video-intel


Touchscreen

The touchscreen is recognized and configurable in X Windows, but does not respond to touch or stylus events. A bug report is open at the linuxwacom project.

The touchscreen (+ stylus) works out of the box once the firmware update is installed (via windows, sadly,so don't pave the drive until you've updated firmware). Make sure to perform a hard reset (60 sec with a paperclip in the bottom of the laptop) after the upgrade.

To disable the touchpad when entering tablet mode (note: the keyboard will be disabled by  the BIOS), create the file /etc/acpi/events/thinkpad-tablet-touchpad-off with the following contents:

event=ibm/hotkey LEN0068:00 00000080 000060f0
action=/usr/local/bin/touchpad_disable.sh

You can verify the hotkey parameter by running acpi_listen and folding the display back to tablet position.

Next, create the file /usr/local/bin/touchpad_disable.sh with the following contents:

export XAUTHORITY=`ls -1 /home/*/.Xauthority | head -n 1`

export DISPLAY=":`ls -1 /tmp/.X11-unix/ | sed -e s/^X//g | head -n 1`"

tp_dev='SynPS/2 Synaptics TouchPad'

state=`xinput list-props "$tp_dev" | grep 'Device Enabled' | cut -d':' -f2 | tr -d '\t '`

if [ $state -eq 1 ]

then

  xinput disable "$tp_dev"

else

  xinput enable "$tp_dev"

fi

You will need to give root access to the X server session for this. The quickhack version is to add xhost + & to ~/.xsessionrc, but it's better to do something like +root@localhost..

The Acer BMA150 accelerometer is detected as a joystick device, so various /sys/bus/iio -based screen orientation scripts like thinkpad-yoga-scripts won't work. An alternative is to define a Window Manager screen edge event that invokes a script to cycle through screen orientation rotations:

#!/bin/sh
# script to rotate screen (counter-clockwise) based on current orientation

conn_line=`xrandr | grep ' connected'`
dev=`echo "$conn_line" | cut -d' ' -f 1`
orient=`echo "$conn_line" | cut -d' ' -f 5`

case $orient in
  '(normal' )
    xrandr --output $dev --rotate left
    ;;
  'left' )
    xrandr --output $dev --rotate inverted
    ;;
  'inverted' )
    xrandr --output $dev --rotate right
    ;;
  'right' )
    xrandr --output $dev --rotate normal
    ;;
  * )
    xrandr --output $dev --rotate normal
    ;;
esac

Some other approaches are discussed in this Manjaro Linux post.

Fingerprint reader

Is not supported by fprint. USB ID: 138a:0090 (Validity, no product string in usb-devices).


Camera

Works out of the box. Driver is uvcvideo. VLC mirrors it without any problems:

vlc v4l:// :v4l-dev="/dev/video0" :v4l-adev="/dev/audio"


FreeBSD

You need to use a 10.3 (BETA3 or more current) UEFI memstick image to do the install with, of course, the OneLink ethernet adapter connected. Presently, no support for wifi or Xorg support (perhaps in 11?). To boot, add an entry in /etc/grub.d/40_custom as follows:

menuentry "FreeBSD 10.3-BETA3" {
  insmod chain
  set root=(hd0,gpt10)
  chainloader (hd0,gpt10)/boot/loader.efi
}

...replacing gpt10 with the partition number where FreeBSD was installed. This starts counting from 1, so the example here is the tenth partition on the first hard drive (hd0).

Note that for update-grub to correctly recognize the FreeBSD partition (and therefore find loader.efi), you should provide an fstab entry to specify the UFS partition type:

/dev/sda10      /home/bsd       ufs    ro,noauto,ufstype=ufs2       0       0

Using a webapp as a local application GUI

This is just a quick proof-of-concept for using a local webserver and a browser instance as a standalone GUI application. The idea is to use a browser and an embedded webserver as your UI toolkit, instead of hoping that the user was able to get qtbindings or shoes installed.


Http Application

The main application spawns two child processes: a local web server and a web browser instance.

In an ideal world, the application would wait for both processes to exit, and perhaps offer to restart one or the other in the event of a crash. Because of the way modern browsers work -- opening URLs in an existing process instead of starting a new one -- this has to be simplified to waiting only for the web server process to exit, and hoping that the browser process can successfully manage things.

The HttpApplication class provides a run() method that starts a webserver on localhost using the first available port, then opens a browser window/tab connected to the webserver:

class HttpApplication

def run(host=nil, port=nil, browser=nil, path=nil)

@server = ServerProcess.new(host, port, @controller)

@server.start

uri = server.uri.dup

if path

path = '/' + path if (! path.start_with? '/')

uri.path = path

end

Browser.open(uri, browser)

Process.waitpid(@server.pid)

end

end

When the webserver process exits, the application will exit. Note that this means the webapp itself must close the webserver process, either through an explicit signout, or through a Javascript on_close or on_unload event handler.


ServerProcess

The webserver process is simply a Webrick instance with a servlet acting as a controller. This is pretty straightforward: the only interesting part is detecting the next available port so that it can be passed to the Webrick constructor.

require 'uri'

require 'webrick'

class ServerProcess

attr_reader :host, :port, :controller, :pid, :uri, :webrick

def initialize(host=nil, port=nil, controller=nil)

@port = port || get_avail_port(host)

@host = host || IPSocket.getaddress(Socket.gethostname)

@controller = controller

end

def start

@pid = Process.fork do

@webrick = WEBrick::HTTPServer.new( :Port => @port )

@webrick.mount('/', @controller) if @controller

trap('HUP') { @webrick.stop; @webrick.start }

trap('INT') { @webrick.shutdown }

@webrick.start

end

trap('INT') { Process.kill 'INT', @pid }

trap('TERM') { Process.kill 'INT', @pid }

@uri = URI::HTTP.build( {:host => @host, :port => @port} )

self

end

def stop

@webrick.shutdown if @webrick

Process.kill('INT', @pid) if @pid

end

private

def get_avail_port(host)

host ||= (Socket::gethostbyname('')||['localhost'])[0]

inf = Socket::getaddrinfo(host, nil, Socket::AF_UNSPEC,

Socket::SOCK_STREAM, 0, Socket::AI_PASSIVE)

fam = inf.inject({}) { |h, arr| h[arr[0]]= arr[2]; h }

sock_host = fam['AF_INET'] || fam['AF_INET6']

sock = sock_host ? TCPServer.open(sock_host, 0) : TCPServer.open(0)

port = sock.addr[1]

sock.close

port

end

end

Browser

The browser process  is a simple launcher that opens a URL in a browser instance , in what is hopefully a platform-independent manner. One could use the launchy gem or some other suitably over-engineered solution, but there's really not much need to support more than Windows, OS X, and Linux. Let's face it, anyone who doesn't have xdg-open installed probably doesn't want to run your webserver-based app on their ideologically-pure system.

require 'shellwords'

class Browser

if RUBY_PLATFORM =~ /darwin/

URL_OPEN_COMMAND = 'open'

elsif RUBY_PLATFORM =~ /linux/ or RUBY_PLATFORM =~ /bsd/

URL_OPEN_COMMAND = 'xdg-open'

else

# assume windows

URL_OPEN_COMMAND = 'start'

end

def self.open(uri, cmd=nil)

pid = Process.fork do

uri_s = Shellwords.shellescape uri.to_s

`#{Shellwords.shellescape(cmd || URL_OPEN_COMMAND)} #{uri_s}`

end

Process.detach(pid)

end

end

RestContoller

The meat of the UI is in the controller object which HttpApplication uses to handle webserver routes. This particular one is based on Webrick's AbstractServlet class, and uses regex patterns to determine the handler for a particular route.

Each route consists of a pattern and a block. The pattern is used to match the URL; for best results, the ^ and $ anchors should be used, and the URL path should be absolute (i.e., starting with '/').

The block receives three arguments: query, request, and server. The query is a Hash extracted from the Webrick HTTPRequest object, the request is the complete HTTPRequest object, and the server is the Webrick HTTPServer object for the webserver (used mainly to shutdown the server). The block must return either the response body or an array [body, content_type, status]; these return values will be written to the Webrick HTTPResponse object.

class RestController < WEBrick::HTTPServlet::AbstractServlet

DEFAULT_STATUS = 200

DEFAULT_CONTENT_TYPE = 'text/html'

DEFAULT_BODY = '404 : not found' # this should really be html

@routes = []

def self.route(pat, &block)

@routes << [pat, Proc.new(&block)]

end

def self.routes; @routes; end

def initialize(server)

@webrick = server

super server

end

def fill_response(response, body, content_type=DEFAULT_CONTENT_TYPE, status=DEFAULT_STATUS)

response.status = status

response['Content-Type'] = content_type

response.body = body

response

end

def route_request(request)

content_type = DEFAULT_CONTENT_TYPE

status = 404

body = DEFAULT_BODY

self.class.routes.each do |pat, proc_obj|

if request.path =~ pat

arr = [ proc_obj.call(request.query, request, @webrick) ].flatten

body = arr[0]

content_type = arr[1] || DEFAULT_CONTENT_TYPE

status = arr[2] || DEFAULT_STATUS

break

end

end

[body, content_type, status]

end

def do_GET(request, response)

body, content_type, status = route_request(request)

fill_response(response, body, content_type, status)

end

def do_POST(request, response)

body, content_type, status = route_request(request)

fill_response(response, body, content_type, status)

end

end

Example

Here's a quick example that defines a few handlers.

def create_rest_controller   cls = RestController   # /test   cls.route(/^\/test$/) { |q,r| ["test", "text/plain"] }   #/quit   cls.route(/^\/quit$/) { |q,r,s| s.shutdown; ["shutting down", "text/plain"] }    #/   cls.route(/^\/$/) { |q,r,s| s.shutdown; ["root!", "text/plain"] }   cls end app = HttpApplication.new( create_rest_controller ) app.run

The content returned by the routes is all plaintext because hey, the blogger interface sucks for entering < and  > characters.

Note the /quit handler: this is necessary to shutdown the webserver process (otherwise it will continue to run in the background). If you are certain your users permit Javascript, you can have an event handler hit this URL when the user closes the browser tab/window.

An alternative is to have the web server manage a PID file, so that subsequent invocations of the application will connect to the webserver process running in the background (much like the browsers that forced us into this mess).

As usual, the code is available on github.

why create two jobs when one will do?

NoSQL : "We don't want to pay for a DBA."

Devops : "We don't want to pay for a sysadmin."

Say goodbye to your weekends, suckaz.

Death to anticipatory design

Thinkpad wants to stop its Great Descent.

Providing users with status LEDs and hardware switches, instead of pretending nobody uses them in order to achieve a more aerodynamic design?

Is sanity finally returning to the industry? Perhaps the house of cards that is the advertising-funded business model will be next.

What decade is it, again?

bash$  find /opt/android/sdk/ -exec file \{\} \; | cut -f 2 -d ':' | grep ELF |cut -b1-15 | sort | uniq
 ELF 32-bit LSB
 ELF 64-bit LSB

bash$ find /opt/processing/ -exec file \{\} \; | cut -f 2 -d ':' | grep ELF | cut -b1-15 | sort | uniq
 ELF 32-bit LSB
 ELF 64-bit LSB  

Processing only has a couple of these, but it's enough:

/opt/processing/modes/java/application/launch4j/bin/windres: ELF 32-bit LSB  executable, Intel 80386, version 1 (SYSV), dynamically linked (uses shared li
bs)
/opt/processing/modes/java/application/launch4j/bin/ld: ELF 32-bit LSB  executable, Intel 80386, version 1 (SYSV), dynamically linked (uses shared libs)
/opt/processing/modes/java/libraries/serial/library/linux32/libjSSC-2.8.so: ELF 32-bit LSB  shared object, Intel 80386, version 1 (SYSV), dynamically linked

Android sins heavily:

/opt/android/sdk/tools/emulator: ELF 32-bit LSB  executable, Intel 80386, version 1 (SYSV), dynamically linked (uses shared libs), for GNU/Linux 2.6.15, not stripped
/opt/android/sdk/tools/lib/gles_mesa/libGL.so.1: ELF 32-bit LSB  shared object,Intel 80386, version 1 (SYSV), dynamically linked, not stripped
/opt/android/sdk/tools/lib/gles_mesa/libGL.so: ELF 32-bit LSB  shared object, Intel 80386, version 1 (SYSV), dynamically linked, not stripped
/opt/android/sdk/tools/lib/gles_mesa/libosmesa.so: ELF 32-bit LSB  shared object, Intel 80386, version 1 (SYSV), dynamically linked, not stripped
/opt/android/sdk/tools/lib/monitor-x86/plugins/org.eclipse.equinox.launcher.gtk.linux.x86_1.1.200.v20120913-144807/eclipse_1502.so: ELF 32-bit LSB  shared object, Intel 80386, version 1 (SYSV), dynamically linked, not stripped
/opt/android/sdk/tools/lib/monitor-x86/libcairo-swt.so: ELF 32-bit LSB  shared object, Intel 80386, version 1 (SYSV), dynamically linked, stripped
/opt/android/sdk/tools/lib/monitor-x86/monitor: ELF 32-bit LSB  executable, Intel 80386, version 1 (SYSV), dynamically linked (uses shared libs), for GNU/Linux2.2.5, not stripped
...
bash$ find /opt/android/sdk -exec file \{\} \; | grep 'ELF 32-bit' | wc -l
81

Guys, it's been TEN YEARS since 64-bit chips started shipping by default on new hardware. Stop compiling to goddamn 32-bit.

Creating an R list using RsRuby

For the most part rsruby, works as advertised. Where things blow up unexpectedly is when using a Ruby Hash to create an R List object.

irb > require 'rsruby'
irb > ENV['R_HOME'] ||= '/usr/lib/R' 
irb > $R = RSRuby.instance

irb > $R.assign('test.x', { :a => 1, :b => "abc", :c => [8,9] } )
RException: Error in (function (x, value, pos = -1, envir = as.environment(pos), inherits = FALSE,  : 
  unused arguments (a = 1, b = "abc", c = 8:9)

This happens because rsruby treats a trailing Hash as a collection of keyword arguments to the R assign() function. All that metaprogramming magic ain't free, y'know?

The solution is to wrap the Hash argument into an actual Hash storing keyword arguments to the R function.

A quick look at the R help file for assign() shows that it has the following signature:

assign(x, value, pos = -1, envir = as.environment(pos),

            inherits = FALSE, immediate = TRUE)

This means that the Hash containing the R List data will have to be passed as the value argument to the assign() call.

$R.assign('test.x', { :value => { :a => 1, :b => "abc", :c => [8,9] } } )

ArgumentError: Unsupported object ':a' passed to R.

Of course, R cannot handle Symbols unless they are the names of function keyword arguments. This is easy to fix.

irb >$R.assign('test.x', { :value => { 'a' => 1, 'b' => "abc", 'c' => [8,9] } } )

 => {"a"=>1, "b"=>"abc", "c"=>[8, 9]} 

irb > $R.eval_R("print(test.x)")

$a

[1] 1

$b

[1] "abc"

$c

[1] 8 9

 => {"a"=>1, "b"=>"abc", "c"=>[8, 9]}

All's well that ends well!

Custom tick labels in R perspective plots

In R, the persp() is a built-in function to create surface plots. The basic usage is straightforward: create a matrix of values

# plot a 10x10 matrix of random values in the range -100..100:
persp( matrix(runif(100, min=-100, max=100), nrow=10, ncol=10) )

All well and good, until it's time to prepare the plots for presentation -- and suddenly it becomes apparent that plots created with persp() do not work well with axis(), text(), mtext(), par(), and other standard graphics device functions.

The trans3d documentation refers the reader to the persp documentation for examples; those examples are too convoluted to serve any useful educational purpose. A quick note to documentation writers: always include an example showing the simplest possible use of your function on trivial data sets (usually the array of integers from 1 to 10, or sin(x) if a function is required). Do not use only edge cases and exciting demos as examples.

The discussion that follows will demonstrate how to construct a perspective plot with custom labels using persp() and trans3d(). The data to be plotted is a 10x10 matrix of values in the range -100:100.


The first thing to understand is that persp() does not just draw a plot; it also returns a perspective matrix (or pmat) which can be used to translate 3-dimensional coordinates to the 2-dimensional coordinate system used in the image of the plot.

The function that performs this translation is trans3d(). Its arguments are the x, y, and z coordinates to be translated, followed by the pmat. The return value is a list with two elements: x and y, the two-dimensional coordinates in the image.

If one of the x, y, and z arguments is a vector, then the vector is considered to be a line at the other two coordinates. Thus, a line along the X axis from (0, 10, 10) to (10, 10, 10) would be translated using trans3d(0:10, 10, 10, pmat); a line along the Y axis from (0, 3, 10) to (0, 7, 10) would be translated using trans3d(0, 3:7, 10, pmat).


Enough background; time for an example.


Basic Perspective Plot

First, some definitions of the data ranges to keep things clear:

x.axis <- 1:10
min.x <- 1
max.x <- 10
y.axis <- 1:10
min.y <- 1
max.y <- 10
z.axis <- seq(-100, 100, by=25)
min.z <- -100
max.z <- 100

Pay particular attention to z.axis: in addition to specifying the range of each axis, the *.axis variables also specify the tick marks of each axis.

Next, a draw the initial perspective plot, saving the pmat:

pmat <- persp( x=x.axis, y=y.axis,
               matrix(runif(100, min=-100, max=100), nrow=10, ncol=10), 
               xlab='', ylab='', zlab='', 
               ticktype='detailed', box=FALSE, axes=FALSE, 
               mar=c(10, 1, 0, 2), expand=0.25,
               col='green', shade=0.25, theta=40, phi=30 )

Note the theta (rotation along the vertical axis) and phi (rotation along the horizontal axis) parameters. It is useful to play with these a bit, as different data sets will require different viewing angles. The r ("eyepoint distance") and d ("perspective strength") parameters provide further control of the view. Note also that box and axes parameters are FALSE: we will be drawing our own axes.


Drawing the Axes

In this plot, the X axis will be drawn at min.y and min.z (left side of Y, bottom of Z), Y at max.x and min.z (right side of X, bottom of Z), and Z at min.x and min.y (left side of X, left side of Y).

These parameters are passed to trans3d() to calculate the coordinates of a line at each axis, as described previously. The translated coordinates can be passed directly to lines().

lines(trans3d(x.axis, min.y, min.z, pmat) , col="black")
lines(trans3d(max.x, y.axis, min.z, pmat) , col="black")
lines(trans3d(min.x, min.y, z.axis, pmat) , col="black")


Drawing Tick Marks

Adding tick marks requires calculating the position of a second line, parallel to the axis, and using segments() to draw ticks that span the distance between the axis and the second line. The basic procedure is as follows:

tick.start <- trans3d(x.axis, min.y, min.z, pmat)
tick.end <- trans3d(x.axis, (min.y - 0.20), min.z, pmat)
segments(tick.start$x, tick.start$y, tick.end$x, tick.end$y)

Note the (min.y - 0.20) in the calculation of tick.end. This places the second line, parallel to the X axis, at the position -0.20 on the Y axis (i.e., into negative/unplotted space).

The tick marks on the Y and Z axes can be handled similarly:

tick.start <- trans3d(max.x, y.axis, min.z, pmat)
tick.end <- trans3d(max.x + 0.20, y.axis, min.z, pmat)
segments(tick.start$x, tick.start$y, tick.end$x, tick.end$y)

tick.start <- trans3d(min.x, min.y, z.axis, pmat)
tick.end <- trans3d(min.x, (min.y - 0.20), z.axis, pmat)
segments(tick.start$x, tick.start$y, tick.end$x, tick.end$y)


Adding Tick Mark Labels

The final step is to label the ticks on each axis. Once again, the procedure is to calculate the position of a line, parallel to the axis, at the position where the labels are to be displayed:

labels <- c('first', 'second', 'third', 'fourth', 'fifth', 'sixth', 'seventh', 'eighth', 'ninth', 'tenth')
label.pos <- trans3d(x.axis, (min.y - 0.25), min.z, pmat)
text(label.pos$x, label.pos$y, labels=labels, adj=c(0, NA), srt=270, cex=0.5)

The adj=c(0, NA) expression is used to left-justify the labels, the srt=270 expression is used to rotate the labels 270°, and the cex=0.5 expression is used to scale the label text to 75% of its original size.

The labels on the Y and Z axes are produced similarly:

labels <- c('alpha', 'beta', 'gamma', 'delta', 'epsilon', 'zeta', 'eta', 'theta', 'iota', 'kappa')
label.pos <- trans3d((max.x + 0.25), y.axis, min.z, pmat)
text(label.pos$x, label.pos$y, labels=labels, adj=c(0, NA), cex=0.5)

labels <- as.character(z.axis)
label.pos <- trans3d(min.x, (min.y - 0.5), z.axis, pmat)
text(label.pos$x, label.pos$y, labels=labels, adj=c(1, NA), cex=0.5)

Note that the Y and Z axis tick labels do not need to be rotated.


The Final Product