Disabling startup (init.d) services in Ubuntu

Ubuntu never has made obvious what the "Ubuntu way" of removing services from System-V run levels is. The GUI tools in GNOME and KDE are incomplete, and a quick investigation of the run level directories shows that they are filled automatically -- so that symlinks added and removed manually might, in the future, get ignored.

Fortunately, the README in /etc/init.d ends with the following advice:

Use the update-rc.d command to create symbolic links in the /etc/rc?.d
as appropriate. See that man page for more details.

The man page lists the following forms for invoking upgrade-rc.d:                                                        


      update-rc.d [-n] [-f] B name  remove


       update-rc.d [-n] B name  defaults [NN | SS KK]


       update-rc.d  [-n]  name  start|stop  R  NN runlevel  [ runlevel ]... 


       update-rc.d [-n] B name  disable|enable [ S|2|3|4|5 ]

The following command will remove the service collectd from all run levels:


bash$ sudo update-rc.d collectd disable
update-rc.d: warning: collectd start runlevel arguments (none) do not match LSB Default-Start values (2 3 4 5)
update-rc.d: warning: collectd stop runlevel arguments (none) do not match LSB Default-Stop values (0 1 6)
 Disabling system startup links for /etc/init.d/collectd ...
 Removing any system startup links for /etc/init.d/collectd ...
   /etc/rc0.d/K95collectd
   /etc/rc1.d/K95collectd
   /etc/rc2.d/S95collectd
   /etc/rc3.d/S95collectd
   /etc/rc4.d/S95collectd
   /etc/rc5.d/S95collectd
   /etc/rc6.d/K95collectd
 Adding system startup for /etc/init.d/collectd ...
   /etc/rc0.d/K95collectd -> ../init.d/collectd
   /etc/rc1.d/K95collectd -> ../init.d/collectd
   /etc/rc6.d/K95collectd -> ../init.d/collectd
   /etc/rc2.d/K05collectd -> ../init.d/collectd
   /etc/rc3.d/K05collectd -> ../init.d/collectd
   /etc/rc4.d/K05collectd -> ../init.d/collectd
   /etc/rc5.d/K05collectd -> ../init.d/collectd

The command to remove a service from a specific runlevel should be the following:


sudo update-rc.d BASENAME disable `runlevel | cut -d ' ' -f 2`

...however a quick experiment shows that the runlevel argument is ignored:

bash$ sudo update-rc.d collectd disable 2

sudo update-rc.d collectd disable 2
update-rc.d: warning: collectd start runlevel arguments (none) do not match LSB Default-Start values (2 3 4 5)
update-rc.d: warning: collectd stop runlevel arguments (none) do not match LSB Default-Stop values (0 1 6)
 Disabling system startup links for /etc/init.d/collectd ...
 Removing any system startup links for /etc/init.d/collectd ...
   /etc/rc0.d/K95collectd
   /etc/rc1.d/K95collectd
   /etc/rc2.d/S95collectd
   /etc/rc3.d/S95collectd
   /etc/rc4.d/S95collectd
   /etc/rc5.d/S95collectd
   /etc/rc6.d/K95collectd
 Adding system startup for /etc/init.d/collectd ...
   /etc/rc0.d/K95collectd -> ../init.d/collectd
   /etc/rc1.d/K95collectd -> ../init.d/collectd
   /etc/rc6.d/K95collectd -> ../init.d/collectd
   /etc/rc2.d/K05collectd -> ../init.d/collectd
   /etc/rc3.d/K05collectd -> ../init.d/collectd
   /etc/rc4.d/K05collectd -> ../init.d/collectd

perf-backed disassembly

Since 2.6.31 or thereabouts, the Linux kernel has come with a built-in performance counter known as perf.

The common form of perf is well-known to be useful in gathering performance statistics on a running program:


bash$ perf stat -cv ./a.out 


cache-misses: 11313 2020574449 2020574449
cache-references: 62031796 2020574449 2020574449
branch-misses: 17909 2020574449 2020574449
branches: 606684832 2020574449 2020574449
instructions: 6324531571 2020574449 2020574449
cycles: 6408533747 2020574449 2020574449
page-faults: 304 2019963367 2019963367
CPU-migrations: 7 2019963367 2019963367
context-switches: 205 2019963367 2019963367
task-clock-msecs: 2019963367 2019963367 2019963367


 Performance counter stats for './a.out':

             11313 cache-misses             #      0.006 M/sec
          62031796 cache-references         #     30.709 M/sec
             17909 branch-misses            #      0.003 %    
         606684832 branches                 #    300.344 M/sec
        6324531571 instructions             #      0.987 IPC  
        6408533747 cycles                   #   3172.599 M/sec
               304 page-faults              #      0.000 M/sec
                 7 CPU-migrations           #      0.000 M/sec
               205 context-switches         #      0.000 M/sec
       2019.963367 task-clock-msecs         #      0.996 CPUs 


        2.027948307  seconds time elapsed

The  events to be recorded can be specified with the -e option in order to refine the output:

bash$ perf stat -e cpu-clock -e instructi

ons  

 Performance counter stats for './a.out':

       2026.748812 cpu-clock-msecs         

        6324293589 instructions             #      0.000 IPC  

        2.032519896  seconds time elapsed

A list of available events can be obtained via perf list:

bash$ perf list | head

List of pre-defined events (to be used in -e):

  cpu-cycles OR cycles                       [Hardware event]

  instructions                               [Hardware event]

  cache-references                           [Hardware event]

  cache-misses                               [Hardware event]

  branch-instructions OR branches            [Hardware event]

  branch-misses                              [Hardware event]

  bus-cycles                                 [Hardware event]

The perf toolchain also includes the utility perf top, which can be used to monitor a single process, or which can be used to monitor the kernel:

bash$ sudo perf top 2>/dev/null

-------------------------------------------------------------------------------

   PerfTop:       0 irqs/sec  kernel:-nan%  exact: -nan% [1000Hz cycles],  (all, 4 CPUs)

-------------------------------------------------------------------------------

             samples  pcnt function               DSO

             _______ _____ ______________________ __________________

               77.00 39.3% intel_idle             [kernel.kallsyms] 

               13.00  6.6% __pthread_mutex_unlock libpthread-2.13.so

               13.00  6.6% pthread_mutex_lock     libpthread-2.13.so

               12.00  6.1% __ticket_spin_lock     [kernel.kallsyms] 

                7.00  3.6% schedule               [kernel.kallsyms] 

                6.00  3.1% menu_select            [kernel.kallsyms] 

                6.00  3.1% fget_light             [kernel.kallsyms] 

                6.00  3.1% clear_page_c           [kernel.kallsyms] 

Where things start to get interesting, however, is with perf record. This utility is generally used along with perf report to record the performance counters of a process, and review them later.

This can be used, for example, to generate a call graph:

bash$  perf record -g -o /tmp/a.out.perf ./a.out

[ perf record: Woken up 1 times to write data ]

[ perf record: Captured and wrote 0.148 MB /tmp/a.out.perf (~6461 samples) ]

bash$ perf report -g -i /tmp/a.out.perf

# Events: 1K cycles

#

# Overhead        Command  Shared Object  Symbol

# ........  .............  .............  ......

#

    99.90%          a.out  a.out          [.] main

            |

            --- main

                __libc_start_main

     0.10%          a.out  [l2cap]        [k] 0xffffffff8103804a

            |

            --- 0xffffffff8105f438

                0xffffffff8105f675

...

Once perf data has been recorded, the perf annotate utility can be used to display a disassembly of the instructions that were executed:

bash$  perf annotate -i /tmp/a.out.perf |more

------------------------------------------------

 Percent |      Source code & Disassembly of a.out

------------------------------------------------

         :

         :

         :

         :      Disassembly of section .text:

         :

         :      0000000000400554

:

    0.00 :  400554:       55                      push   %rbp
    0.00 :  400555:       48 89 e5                mov    %rsp,%rbp
    0.00 :  400558:       48 81 ec 30 00 0c 00    sub    $0xc0030,%rsp
    0.00 :  40055f:       48 8d 85 d0 ff fb ff    lea    -0x40030(%rbp),%rax
    0.00 :  400566:       ba 00 00 04 00          mov    $0x40000,%edx
    0.00 :  40056b:       be 00 00 00 00          mov    $0x0,%esi
    0.00 :  400570:       48 89 c7                mov    %rax,%rdi
    0.00 :  400573:       e8 b0 fe ff ff          callq  400428 <memset@plt>
    0.00 :  400578:       c7 45 fc 00 00 00 04    movl   $0x4000000,-0x4(%rbp)

    ...

    4.21 :  4006a5:       8b 45 d0                mov    -0x30(%rbp),%eax
   15.54 :  4006a8:       83 c0 01                add    $0x1,%eax
    4.97 :  4006ab:       89 45 d0                mov    %eax,-0x30(%rbp)
    4.87 :  4006ae:       8b 45 d0                mov    -0x30(%rbp),%eax
   17.79 :  4006b1:       83 c0 01                add    $0x1,%eax
    4.36 :  4006b4:       89 45 d0                mov    %eax,-0x30(%rbp)
    4.72 :  4006b7:       48 83 45 f0 01          addq   $0x1,-0x10(%rbp)
    0.00 :  4006bc:       48 8b 45 f0             mov    -0x10(%rbp),%rax

    ...

As to be expected from Torvalds and company, the utilities include a number of options for generating parser-friendly output, limiting reporting to specified events and symbols, and so forth. Check the man pages for details.

Easy E(17)

Ubuntu finally ships with a working E17, but there is a lingering problem: most of the modules are gone!

To get a full-featured install of E17, replete with modules and UI enhancements (finally! the settings panel entries re available from the main menu!), it is best to install from SVN.

Building E17 has always been an ordeal. Fortunately, the script easy_e17.sh makes downloading, building, and installing E17 from SVN to be quite a simple affair, once all of the dependencies are met.


To begin with, download the easy_e17 script from http://omicron.homeip.net.

There is a bug in the script which must be fixed if packages are going to be built. According to this bug report, the line (around 36 or 37)

  packages_full="$efl_basic $bin_basic $e_modules_bin $e_modules_extra $efl_extra
$bin_extra"


should be

  packages_full="$efl_basic $bin_basic $e_modules_bin $efl_extra $bin_extra $e_modules_extra"


... in order to get all the internal dependencies straight. Failure to do this results in the following error:
e_mod_main.c:1527:32: error: ‘ethumb_client’ undeclared (first use in this function)


Next, install all necessary dependencies to build E17 and the modules:

bash$ sudo apt-get install autopoint libudev-dev libgcrypt11 libgcrypt11-dev
bash$ sudo apt-get install libasound2-dev libasound2 libxine-dev 
bash$ sudo apt-get install paman padevchooser paprefs pavucontrol pavumeter 
bash$ sudo apt-get install libiptcdata-dev libmpd-dev cython libxcb-shape0-dev

According to this guide, the full list of dependencies is as follows:

bash$ sudo apt-get install xterm make gcc bison flex subversion cvs automake1.10 autoconf autotools-dev autoconf-archive libtool gettext libpam0g-dev libfreetype6-dev libpng12-dev zlib1g-dev libjpeg62-dev libtiff4-dev libungif4-dev librsvg2-dev libx11-dev libxcursor-dev libxrender-dev libxrandr-dev libxfixes-dev libxdamage-dev libxcomposite-dev libxss-dev libxp-dev libxext-dev libxinerama-dev libxft-dev libxfont-dev libxi-dev libxv-dev libxkbfile-dev libxres-dev libxtst-dev libltdl7-dev libglu1-xorg-dev libglut3-dev xserver-xephyr libdbus-1-dev liblua5.1-0-dev libasound2-dev libudev-dev autopoint  libxml2-dev

Chances are, however, that most of these are already installed.


The E17 build makes some assumptions about the location of the gcrypt libraries, so they must be symlinked:

bash$ sudo ln -s /usr/lib/x86_64-linux-gnu/libgcrypt.a /lib/x86_64-linux-gnu
bash$ sudo ln -s /usr/lib/x86_64-linux-gnu/libgcrypt.la /lib/x86_64-linux-gnu
bash$ sudo ln -s /usr/lib/x86_64-linux-gnu/libgcrypt.so /lib/x86_64-linux-gnu

...other wise linker errors such as the following will appear:
../../src/lib/.libs/libeet.so: undefined reference to `gcry_cipher_setiv'
../../src/lib/.libs/libeet.so: undefined reference to `gcry_cipher_setkey'

The Xine development headers (libxine-dev) must be installed in order for emotion to compile; otherwise an error like this will appear:
configure: error: Xine, Gstreamer or VLC backends must be selected to build Emotion
Naturally, Gstreamer or VLC headers could be installed instead.


The asound2-dev and the pulseaudio tools are required to get the mixer module working. Without them, the error
No ALSA mixer found!

will appear in the mixer module settings. Note that pulseaudio requires that the current user be added to the audio group; this can be done with the following command:

  sudo adduser $USERNAME audio

Once all this is done, it is a simple matter to install E17 with the following command:

./easy_e17.sh --instpath=/usr/local/e17 --srcpath=/home/$USER/src/e17 -i

This downloads the E17 source code to ~/src/e17, and sets the install directory to /usr/local/e17. There should be a nice success message if everything goes well.

Next, build all of the modules:

./easy_e17.sh --instpath=/usr/local/e17 --srcpath=/home/$USER/src/e17 --packagelist=full -i


Upgrading is just as straightforward:

./easy_e17.sh --instpath=/usr/local/e17 --srcpath=/home/$USER/src/e17 --packagelist=full -u


Final notes:

  * The composite module causes problems: when using the Software engine, it causes typing delays that make terminals near unusable; when using the OpenGL engine, windows do not update until there is a focus change (BIG problem). It should be disabled.

  * The shelf containing the systray module must be given the stacking order Above Everything; otherwise, the applications in the E17 systray will not receive mouse clicks (VERY BAD).

  * Determining the fonts used by various modules is easier when the source is available. The font classes used in the tclock module, for example, are in E-MODULES-EXTRA/tclock/tclock.edc. The time is displayed in text_class: "module_large", and the date is displayed in text_class: "module_small". The Settings->Look->Fonts dialog can be used to determine the font used for these classes by enabling (via the checkbox at bottom-right) the font class Modules::Small or Modules::Large, and setting the font (e.g. Sans/Regular/12 pixels) to be used for each.

UPDATE : Recent builds now fail when using --packagelist=full. Best stick with --packagelist=half if the full build fails.

Static DNS servers in /etc/resolv.conf

It's an old problem: NetworkManager overwrites /etc/resolv.conf with each connection, forcing one to use the local ISP's crappy DNS servers instead of one's tried-and-true public DNS servers.

The fixes for this are legion: chattr resolv.conf, disable dhclient, replace NetworkManager.

It turns out that dhclient can be configured to include specified DNS servers (up to twp before a warning is displayed) in the resolv.conf.

Simply add the following line to /etc/dhcp/dhclient.conf :

prepend domain-name-servers 4.2.2.1, 4.2.2.4;                

Obviously, replace 4.2.2.1 and 4.2.2.4 with whatever name servers are desired. The semicolon, by the way, is extremely important -- don't leave it out.

Ubuntu, you're fired.

Attempted to upgrade a workstation (dual-xeon, sata3 ssd for /; sata 3 hdd for /usr, /var, /opt, /usr/local; sata2 mirrored raid (via dmraid) for /home) from 10.04 to 11.04 via do-release-upgrade (in update-manager-core) last night. Why do that? Because Ubuntu only updates software packages for its newest release, so if you need, say, Python 2.7 in order to work on a project, you're SOL unless you upgrade.

Everything went well until the final reboot, which left the machine at:


error: the symbol 'grub_xputs' not found.

Verified that the grub software is all present:

grub rescue> ls (hd0,1)/boot
... lots of modules and such...

Everything seemed to be there, so attempted to boot manually, under the assumption it was all due to misconfiguration:

grub rescue> set root=(hd0,1)
grub rescue> set kernel=/boot/vmlinuz-2.6.32-33-generic
grub rescue> set initrd=/boot/initrd.img-2.6.32-33-generic
grub rescue> boot
error: no loaded kernel

Second attempt, loading the grub modules manually (per Ubuntu Grub2 Docs and this thread):

grub rescue> set prefix=(hd0,1)/boot/grub
grub rescue> insmod normal
error: the symbol 'grub_xputs' not found.
grub rescue> insmod linuxgrub
error: the symbol 'grub_xputs' not found.

For some reason, the symbols in the grub2 bootstrap code do not match the ones expected by the modules. One of them is the wrong version. Looks to be a reported bug.

The solution, per Ubuntu Grub2 Docs and this fix, is to boot a LiveCD (which must be the same version as the newly-installed-but-broken Ubuntu) and run grub-install. This turns out to be a huge delay, as the upgrade took place without a LiveCD, so an ISO must be downloaded. It is also a huge pain when one is out of blank CDs, and has to press a usbstick into service (accomplished per Ubuntu Install Docs).

It also doesn't work. The new error on reboot is

error: file not found

grub rescue> insmod linux

ELF header smaller than expected

It turns out that 10.04 upgrades to 10.10, NOT 11.04, so the LiveCD contained the wrong version of grub!

Fortunately, there is a guide to purging and reinstalling grub once booted from any LiveCD.

The steps are (details such as hdd devices and mount points are specific to this workstation):

bash$ sudo bash

bash# mount /dev/sdk1 /mnt

bash# mount /dev/sdf3 /usr

bash# mount /dev/sdf4 /var

bash# mount --bind /dev /mnt/dev

bash# mount --bind /dev/pts /mnt/dev/pts

bash# mount --bind /proc /mnt/proc

bash# mount --bind /sys /mnt/sys

bash# chroot /mnt

bash-chroot# apt-get update

bash-chroot# apt-get purge grub grub-common grub-pc

bash-chroot# apt-get install grub-common grub-pc

bash-chroot# #NOTE: Be sure to select /dev/sdk for Grub2 install location

bash-chroot# exit

bash# umount /mnt/dev/pts

bash# umount /mnt/dev 

bash# umount /dev/sys

bash# umount /mnt/proc

bash# umount /mnt/var

bash# umount /mnt/usr

bash# umount /mnt

bash# reboot

That does the trick.

But you know what would really be nice? A way to upgrade the OS, including the kernel and all packages, WITHOUT TOUCHING THE GODDAMN BOOT-LOADER. Seriously, it's an upgrade, the bootloader already works -- don't mess with it!

Pre-defined preprocessor definitions (GCC)

The predef project is extremely convenient for looking up architecture-dependent, os-dependent, compiler-dependent, or standard C/C++ compiler macros, but it is not available when working offline.

To get around this, the preprocessor definitions defined automatically by GCC can be viewed with this command line:

bash$ gcc -dM < /dev/null

#define __DBL_MIN_EXP__ (-1021)
#define __UINT_LEAST16_MAX__ 65535
#define __FLT_MIN__ 1.17549435082228750797
...


This will output the #define commands as they would be executed by the preprocessor. To get a list of the non-expanded macros, use -dN :

bash$ gcc -dN < /dev/null
# 1 ""
# 1 ""
#define __STDC__
#define __STDC_HOSTED__
#define __GNUC__
#define __GNUC_MINOR__
...

This is a bit more noisy, as undefined macros (e.g. "# 1") are included, but may be more suitable for parsing.

/usr/bin/say linux

Anyone who has spent any time fiddling with OS X on the command line will have discovered /usr/bin/say, and incorporated it into a few shell scripts to provide background notifications ("build complete", etc).

While Linux doe snot provide this command out of the box, there is a package called festival which provides text-to-speech capability:

apt-get install festival festival-freebsoft-utils

The usage is quite straightforward:

echo "this is only a test" | festival --tts

A simple alias can be used to replace /usr/bin/say on the command line:

alias say='festival -tts'

A more script-friendly solution would be to create a small wrapper script that passes /usr/bin/say arguments (filenames or STDIN) to festival:


#!/usr/bin/env ruby                                                             


def say(str)
  `echo '#{str}' | festival --tts`
end


if __FILE__ == $0
  say(ARGF.read)
end