realurl

The inability of browsers to deal with shortened URLs led to the writing of a quick one-liner for verifying the final destination of a URL.

Add this line to ~/.bashrc :

alias realurl='ruby -r net/http -e "ARGV.map{ |s| while(r=Net::HTTP.get_response(URI(s)) and r.code[0] == \"3\"); s=r[\"location\"];end;puts s}"'

Then use the realurl alias to look up URLs:

bash# realurl http://goo.gl/KjdQYT
https://www.blackhat.com/us-16/briefings.html#weaponizing-data-science-for-social-engineering-automated-e2e-spear-phishing-on-twitter?Wolfesp18

The expanded Ruby code is as follows:

#!/usr/bin/env ruby
# expand a shortened URL by following all redirects
require 'net/http'

def expand_url(str)
  while(hdr=Net::HTTP.get_response(URI(str)) and hdr.code[0] == '3')
    str = hdr['location']
  end
  puts str
end

if __FILE__ == $0
  ARGV.each do |arg|
    puts expand_url arg
  end
end 

Sure, it would be nice to print the page Title attribute as well, but that means loading up Nokogiri and ends up being a bit of overkill.

UPDATE: I've been making pretty heavy use of a shell function to get the page title via wget:

pagetitle () 
{ 
    wget --quiet -O - "$1" | sed -n -e 's!.*.*!\1!p'
}

Big-M Notation

Today's node.js chaos seemed like it was only a simple lesson in Operations 101 (hint: always host the code for your application and the modules it depends on in the same place) until one looks at the actual function that everyone was linking to:

function leftpad (str, len, ch) {
  str = String(str);

  var i = -1;

  if (!ch && ch !== 0) ch = ' ';

  len = len - str.length;

  while (++i < len) {
    str = ch + str;
  }

  return str;
}

This is quite possibly the worst algorithm in the (admittedly short) history of padding strings. Unless JS is doing something special behind our backs, we're looking at something on the order of len string allocations here, each one slightly larger than the last.

OK, so JS more or less asks for problems like this because it refuses to provide anything remotely resembling a standard library. Still, this is a common enough problem: people using dynamic memory management as if it were free, with no awareness of the costs of memory allocation and subsequent garbage collection.

To help raise awareness, I propose a Big-M notation to supplement Big-O notation (and Big-theta and all that). Big-M, or M( ), determines how memory allocations in an algorithm will grow with an input n. As with the other Big-notations, lower-order terms are dropped. 

Using this notation, the above string padding algorithm is M(n), when it should be something constant like M(1).

firefox profiles for ad-hoc privilege separation

Just a quick write-up, since few people seem to actually be doing this.

Firefox supports profiles: each profile is a distinct configuration under ~/.mozilla/firefox, with its own extensions, themes, bookmarks, history, and cookies. This means that badly-behaved web pages in one profile will not be able to read the data from another profile (unless they actually own your system, which is beyond the scope of this simple technique). It also means you can have as many GMail accounts as you like without having to sign out and wipe cookis every time.

To create non-default profiles, open the profile manager from the command line:

bash# firefox --no-remote -P

Create as many profiles as you like. There can be one for each online persona, one for email, one for paying bills, one for pr0n, whatever. For this discussion, we will assume the profile test was created.


For each profile, do the following:

1. Create a desktop file for the profile in ~/.local/share/applications . Make sure the .desktop filename is unique.

  bash# cp /usr/share/applications/firefox.desktop ~/.local/share/applications/firefox-test.desktop

2. Add a distinct icon for the profile in ~/.local/share/icons . You can get different icons by doing a Google image search for firefox icon. There are plenty.

  bash# cp ~/downloads/firefox-green.png ~/.local/share/icons/firefox-test.png

3. Edit the .desktop file to ensure that the Name and Icon are unique, and that the Exec line references the new profile:

Name=Mozilla Firefox (testing)
Exec=firefox --no-remote -P test
Icon=firefox-test.png

That's all there is to it. A new icon for Firefox (testing) will appear in your favorite window manager. Add whatever extensions you deem fit for the purpose.

RIP inittab

Your getty configs are all in /lib/systemd/system/getty@.service now.

[Service]
# the VT is cleared by TTYVTDisallocate
ExecStart=-/sbin/agetty --noclear %I $TERM                                      
Type=idle
Restart=always
RestartSec=0
UtmpIdentifier=%I
TTYPath=/dev/%I
TTYReset=yes
TTYVHangup=yes
TTYVTDisallocate=yes
KillMode=process
IgnoreSIGPIPE=no
SendSIGHUP=yes

Control via /etc/systemd/logind.conf -- see man 5 logind.conf for more info.

xdg-open arcanery

In the midst of playing with tycat and terminology on a virtual console*, it became apparent that xdg-open wasn't doing its job in regards to opening PDF documents.

  bash# xdg-open /tmp/dw.pdf 
  Filename "file:///tmp/dw.pdf" does not exist or is not a regular file

A simple check of the default handler shows that everything is fine:
  bash# xdg-mime query default application/pdf
  evince.desktop
..but for fun, let's try to change it and see what happens.
  bash# xdg-mime default apvlv.desktop application/pdf
  bash# xdg-open /tmp/dw.pdf 
  Filename "file:///tmp/dw.pdf" does not exist or is not a regular file
Nothing.

Grepping for the error in /usr/bin/xdg-open comes up empty. It's odd that two unrelated programs (evince and apvlv) would have the incorrect error message (the file does exist and is regular), or that they would both fail to handle file:// URIs. How does xdg-open actually route these requests?

Turns out it depends on your window manager. If you are running Enlightenment, xdg-open performs the following:

open_enlightenment()
{
    enlightenment_open "$1"

    if [ $? -eq 0 ]; then
        exit_success
    else
        exit_failure_operation_failed
    fi
}

Replacing enlightenment_open "$1" with open_generic "$1" works. Can we invoke open_generic when enlightenment_open fails? Turns out we can't, because enlightenment_open returns 0 (success) even when the file isn't opened.

Fixing enlightenment_open might do the trick. The relevant code (enlightenment.git/ src/bin/e_open.c) is here:

        char **itr;

        int ret = EXIT_SUCCESS;

        for (itr = cmds; *itr != NULL; itr++)

          {

             /* Question: should we execute them in parallel? */

             int r = system(*itr);

             if (r < 0)

               fprintf(stderr, "ERROR: %s executing %s\n", strerror(errno),

                       *itr);

             free(*itr);

             if (r > 0) /* Question: should we stop the loop on first faiure? */

               ret = r;

          }

        free(cmds);

        return ret;

It turns out that the status of what enlightenment_open is calling is ignored. But what is it calling? Adding the line fprintf(stderr, "Trying %s\n", *itr); after the system() call uncovers what is actually happening:

bash# make src/bin/enlightenment_open

  CC       src/bin/src_bin_enlightenment_open-e_open.o

  CCLD     src/bin/enlightenment_open

bash# src/bin/enlightenment_open /tmp/dw.pdf 

Filename "file:///tmp/dw.pdf" does not exist or is not a regular file

TRYING evince-previewer 'file:///tmp/dw.pdf

Where the hell did evince-previewer come from? To make a long story short, xdg-open (and enlightenment_open, and all other *_open tools) uses /usr/bin/mimeopen as a backend. 

This utility operates on files, not mime-types:

bash# mimeopen -a /tmp/dw.pdf 

Please choose an application

1) Print Preview  (evince-previewer)

2) Document Viewer  (evince)

3) vprerex  (vprerex)

4) GNU Image Manipulation Program  (gimp)

use application #

Cancelled

OK, so that is where evince-previewer is coming from. A quick test shows that the actual bug is caused by evince-previewer not understanding file URIs:

bash# evince-previewer file:///tmp/dw.pdf

Filename "file:///tmp/dw.pdf" does not exist or is not a regular file

bash# evince file:///tmp/dw.pdf

(evince:21480): Gtk-WARNING **: gtk_widget_size_allocate(): attempt to allocate widget with width -82 and height 15

That Gtk warning accompanies evince loading and displaying the file just fine. 

Changing the handler for PDFs using mimeopen turns out to work just fine:

bash# mimeopen -d /tmp/dw.pdf 

Please choose a default application for files of type application/pdf

1) Print Preview  (evince-previewer)

2) Document Viewer  (evince)

3) vprerex  (vprerex)

4) GNU Image Manipulation Program  (gimp)

5) Other...

use application #2

Opening "/tmp/dw.pdf" with Document Viewer  (application/pdf)

(evince:21510): Gtk-WARNING **: gtk_widget_size_allocate(): attempt to allocate widget with width -82 and height 15

bash#  xdg-open /tmp/dw.pdf 

(evince:21550): Gtk-WARNING **: gtk_widget_size_allocate(): attempt to allocate widget with width -82 and height 15

xdg-open /tmp/a.pdf

(evince:21602): Gtk-WARNING **: gtk_widget_size_allocate(): attempt to allocate widget with width -82 and height 1

Of course, that doesn't mean that evince-previewer is the only bug. Since evince-previewer returns 1 (failure), enlightenment_open should be returning 1 instead of 0:

        int ret = EXIT_FAILURE;

        for (itr = cmds; *itr != NULL; itr++)

          {

             int r = system(*itr);

             if (r < 0)

               fprintf(stderr, "ERROR: %s executing %s\n", strerror(errno),

                       *itr);

             free(*itr);

             ret =  r;

          }

...and xdg-open should check the return value and fallback to open_generic:

    enlightenment_open "$1"

    if [ $? -eq 0 ]; then

        exit_success

    else

        open_generic "$1"

        if [ $? -eq 0 ]; then

            exit_success

        else 

            exit_failure_operation_failed

        fi

    fi

Let's not even get started on why xdg-open doesn't propagate changes back to mimeopen. Wait, why doesn't it?

UPDATE: looks like there's something deeply wrong with with the xdg-* utilities:

bash# xdg-settings get default-web-browser
xdg-settings: unknown desktop environment

The problem here is that xdg-settings performs its own check for desktop environment (modularity, people!) and, of course, implements this poorly. If the desktop environment is detected (as is Enlightenment), but is not KDE|Gnome|XFCE|MATE,  it returns "unknown desktop environment". If the environment is not detected, a generic handler is called. There are some very confused people working on this software.

The fix is to modify xdg-settings, changing the default case statement
*)
    exit_failure_operation_impossible "unknown desktop environment"
    ;;

to

*)
    dispatch_specific generic "$@"
    ;;

* For those interested: install tslib and rebuild EFL from source, then run the following on a virtual terminal:
ELM_ENGINE=fb /usr/local/bin/terminology -f=nexus/20

Yes, the -b option for a backgroud image will work, as will tytls, tyop, and tycat.

Dispatches from the X1 front

The first-gen Thinkpad X1: possibly the best laptop I've ever owned. Lasted three years, compared to the usual 12-18 months. It still runs, but the battery is down to about 40 minutes, and the (second!) keyboard is starting to lose keys.

The third-gen Thinkpad X1 Carbon: a little annoying at first: no SD slot, no h/w kill switch for the wireless, no spill-through keyboard (wtf guys, that was brilliant!). About 6 months in, the video glitches started: there is a short which corrupts video memory while using wifi. The fix, of course, is a replacement system board, which is great if you have a spare third-gen X1 Thinkpad Carbon lying around and can just up and change workstations like it's no big. Workaround: rfkill block wifi, or just get used to the strobe effect.

The latest: Thinkpad X1 Yoga. And let's just put it out there: this doesn't work out of the box like the previous two did. Lenovo and Intel seem to have backed off of their Linux commitment, though that's probably half their users these days (errybody else on the mobile, right?)

Here's the rundown on getting the X1 Yoga up and running with a Linux install.


Basic Linux

Debian 8.3.0 AMD64 netinst ISO if you were wise enough to buy the OneLink+ RJ45 ethernet adapter; otherwise, use the regular (non-netinst) ISO.

Use unetbootin to make a USB flash drive image for this. Go into the BIOS on the X1 Yoga, enable F12 for boot device selection (just because), turn off secure boot, maybe enable diagnostic boot as well. No need to allow legacy BIOS installs, Debian will work just fine with uEFI (unlike FreeBSD where you need to use the 10.3-Beta or 11 memstick). Go through the install like you usually do. Your ethernet (if you have it) will be detected (again, unlike in FreeBSD).

Sound, APCI (suspend, hibernate, and battery status) and video card should all be working out of the box.


Wireless

Upgrade your kernel to 4.4.1 from the ubuntu PPA:
  http://kernel.ubuntu.com/~kernel-ppa/mainline/v4.4.1-wily/
This is per notes about a typo in the iwlwifi driver at
 https://bbs.archlinux.org/viewtopic.php?id=206831
Looking at the kernel source, 4.2.7 has the fix as well, but might as well go with the latest stable release of the kernel if we're upgrading.

You'll also need to upgrade the firmware, or your Wifi will not be recognized. There is a debian package in backports for this:
 https://packages.debian.org/jessie-backports/firmware-iwlwifi


Skylake P-State Fix

If you rebooted excitedly into your new kernel without following the rest of these steps, you're likely looking at a blank screen. That's because of a skylake p-state bug reported here:
 https://www.reddit.com/r/thinkpad/comments/410haw/yoga_260_wont_boot_ubuntu_with_kernel_4244_but/

This is because the kernel needs a command line parameter to not totally crap itself when it encounters Skylake's poor implementation of pstate.

To fix this, change the cmdline in /etc/default/grub to:
  GRUB_CMDLINE_LINUX_DEFAULT='intel_pstate=no_hwp quiet'

Don't forget to run update-grub when you're done.

Hopefully this will go away in 4.5; according to a commenter, that's what the kernel guys are trying for.


i915 Xorg Video fix

Of course, the fancy new kernel hosed your X windows, didn't it?

Not sure why there's such a tight coupling there, but never mind. This one's an easy fix, per http://comments.gmane.org/gmane.linux.debian.user/508779 . Download (and install) a new version of the xserver-xorg-video-intel package from backports:

https://packages.debian.org/jessie-backports/xserver-xorg-video-intel


Touchscreen

The touchscreen is recognized and configurable in X Windows, but does not respond to touch or stylus events. A bug report is open at the linuxwacom project.

The touchscreen (+ stylus) works out of the box once the firmware update is installed (via windows, sadly,so don't pave the drive until you've updated firmware). Make sure to perform a hard reset (60 sec with a paperclip in the bottom of the laptop) after the upgrade.

To disable the touchpad when entering tablet mode (note: the keyboard will be disabled by  the BIOS), create the file /etc/acpi/events/thinkpad-tablet-touchpad-off with the following contents:

event=ibm/hotkey LEN0068:00 00000080 000060f0
action=/usr/local/bin/touchpad_disable.sh

You can verify the hotkey parameter by running acpi_listen and folding the display back to tablet position.

Next, create the file /usr/local/bin/touchpad_disable.sh with the following contents:

export XAUTHORITY=`ls -1 /home/*/.Xauthority | head -n 1`

export DISPLAY=":`ls -1 /tmp/.X11-unix/ | sed -e s/^X//g | head -n 1`"

tp_dev='SynPS/2 Synaptics TouchPad'

state=`xinput list-props "$tp_dev" | grep 'Device Enabled' | cut -d':' -f2 | tr -d '\t '`

if [ $state -eq 1 ]

then

  xinput disable "$tp_dev"

else

  xinput enable "$tp_dev"

fi

You will need to give root access to the X server session for this. The quickhack version is to add xhost + & to ~/.xsessionrc, but it's better to do something like +root@localhost..

The Acer BMA150 accelerometer is detected as a joystick device, so various /sys/bus/iio -based screen orientation scripts like thinkpad-yoga-scripts won't work. An alternative is to define a Window Manager screen edge event that invokes a script to cycle through screen orientation rotations:

#!/bin/sh
# script to rotate screen (counter-clockwise) based on current orientation

conn_line=`xrandr | grep ' connected'`
dev=`echo "$conn_line" | cut -d' ' -f 1`
orient=`echo "$conn_line" | cut -d' ' -f 5`

case $orient in
  '(normal' )
    xrandr --output $dev --rotate left
    ;;
  'left' )
    xrandr --output $dev --rotate inverted
    ;;
  'inverted' )
    xrandr --output $dev --rotate right
    ;;
  'right' )
    xrandr --output $dev --rotate normal
    ;;
  * )
    xrandr --output $dev --rotate normal
    ;;
esac

Some other approaches are discussed in this Manjaro Linux post.

Fingerprint reader

Is not supported by fprint. USB ID: 138a:0090 (Validity, no product string in usb-devices).


Camera

Works out of the box. Driver is uvcvideo. VLC mirrors it without any problems:

vlc v4l:// :v4l-dev="/dev/video0" :v4l-adev="/dev/audio"


FreeBSD

You need to use a 10.3 (BETA3 or more current) UEFI memstick image to do the install with, of course, the OneLink ethernet adapter connected. Presently, no support for wifi or Xorg support (perhaps in 11?). To boot, add an entry in /etc/grub.d/40_custom as follows:

menuentry "FreeBSD 10.3-BETA3" {
  insmod chain
  set root=(hd0,gpt10)
  chainloader (hd0,gpt10)/boot/loader.efi
}

...replacing gpt10 with the partition number where FreeBSD was installed. This starts counting from 1, so the example here is the tenth partition on the first hard drive (hd0).

Note that for update-grub to correctly recognize the FreeBSD partition (and therefore find loader.efi), you should provide an fstab entry to specify the UFS partition type:

/dev/sda10      /home/bsd       ufs    ro,noauto,ufstype=ufs2       0       0