How cloud-init systemd service is activated on Ubuntu 16.04 (xenial)

Recently I've trace how cloud-init is being run. First, I try to find service file in /etc/system/system or /lib/systemd/system. Although I can locate the unit files, I can't see any systemd target that will bring up these cloud-init services. I did't see how is activated by systemd.

Finally, I found that cloud-init project utilize the systemd generator to generate the unit files dynamically. The dynamic unit files is located at /run/system/generator.early/ With this unit file, when the is activated by systemd, the will also be activated. Then the systemd will further activate related cloud-init service in /etc/systemd/system/

Below is the dynamic unit file.

$ ll /run/systemd/generator.early/
lrwxrwxrwx 1 root root 37 Jul 14 07:52 /run/systemd/generator.early/ -> /lib/systemd/system/

If I want to disable cloud-init, there are several ways to do it.

  1. Mask the default cloud-init systemd generator.
    ln -sf /dev/null /etc/systemd/system-generator/cloud-init-generator
  2. Add kernel parameter cloud-init=disabled or touch /etc/cloud/cloud-init.disabled. This kernel parameter will be processed by cloud-init systemd generator. See


use snapper to Travel back in time and compare (The ultimate snapshot tool for Linux)

Snapper is a great tool for making snapshot on linux. You can download it on ArchLinux/Debian/OpenSUSE/Ubuntu

Personally I use btrfs as underlying file system. Here are some basic command to snapshot my data.

# install snapper
sudo apt install -y snapper

# Suppose /dev/sda3 is formatted as btrfs file system.
# mount the file system
sudo mount /dev/sda3 /mnt

# create a btrfs subvolme "data"
sudo btrfs subvolume create /mnt/data
# unmount /dev/sda3
sudo umount /mnt

# mount only the subvolume "data", instead of whole file system
sudo mount -o subvol=data /dev/sda3 /data

# create the snapper config for "/data" folder. The config name could be anything, here I choose "data"
sudo snapper -c data create-config /data

# By default, the snapper will make a snapshot per hour, just like Mac OS's Time machine.
# Check the snapshot list
# sudo snapper -c data list 
Type   | #    | Pre # | Date                            | User | Cleanup  | Description | Userdata
single | 1 |       | Tue 16 Aug 2016 09:17:02 AM CST | root | timeline | timeline    |         
single | 2 |       | Tue 16 Aug 2016 10:17:02 AM CST | root | timeline | timeline    |         
single | 3 |       | Tue 16 Aug 2016 11:17:03 AM CST | root | timeline | timeline    |         
single | 4 |       | Tue 16 Aug 2016 12:17:03 PM CST | root | timeline | timeline    |      

# In fact, you can find the underlying snapshot in "data" subvolume
sudo mount /dev/sda3 /mnt
ls -al /mnt/data/.snapshots/
drwxr-x--- 1 root root 230 Aug 16 12:17 ./
drwxr-xr-x 1 root root 340 Aug 15 10:47 ../
drwxr-xr-x 1 root root  32 Aug 16 09:17 1/
drwxr-xr-x 1 root root  32 Aug 16 10:17 2/
drwxr-xr-x 1 root root  32 Aug 16 11:17 3/
drwxr-xr-x 1 root root  32 Aug 16 12:17 4/

If your root file system is also a btrfs subvolume. That would be even better. You can snapshot your whole system!

sudo snapper -c root create-config /

LXD container Getting Started

Below I give some simple command to run a ubuntu:16.04 lxd container. LXD container is fast, very efficient, very low-footprint virtual machine. I recommend people who didn't need low-level system control (such as disk, loopback device) give it a try. I think this kind of lightweight virtual machine would benefit people who study machine learning or similar scientific computing.

# check current status of lxd. If you install lxd the first time, it should containing no virtual machines.
lxc list

# if my-ubuntu virtual machine not exist yet.
lxc launch ubuntu:16.04 my-ubuntu
# if my-ubuntu virtual machine already exist.
lxc start my-ubuntu

# check status of my-ubuntu virtual machine
lxc list

# login to root account
lxc exec first -- /bin/bash
# or login to ubuntu account
lxc exec first -- /bin/su - ubuntu

# stop the virtual machine
lxc stop my-ubuntu

# you can even copy virtual machine
lxc copy my-ubuntu my-ubuntu2
# or rename it
lxc mv my-ubuntu stanley-ubuntu
lxc mv stanley-ubuntu my-ubuntu

# delete my-ubuntu virtual machine
lxc delete my-ubuntu

# check my-ubuntu is deleted
lxc list

Varies images is provided by Canonical in

Note that lxd has some limitaions. Such as you can't

  • setup loopback device
  • mount fuse file system
  • have bridged network.
  • load/remove kernel module ... (But if you choose to run a priviledged lxd virtual machine, some limitations might not exist anymore. But I haven't dig into it currently.)

Build Debian/Ubuntu LiveCD

Note that this article introduces how to build Debian/Ubuntu LiveCD iso image from scratch. With this method, you have great flexibility to customize the LiveCD.

However, if you only want to customize the LiveCD a little bit. Such as adding some packages for installation, or changing the preseed configuration, you can refer to DebianCustomCD, Simple-CDD and LiveCDCustomization

Debian Jessie

# within Debian jessie
git clone git://
cd live-build
git checkout debian/4.0.5-1
dpkg-buildpackage -b -uc -us
cd ..
sudo dpkg -i live-build_4.0.5-1_all.deb

sudo apt-get install -y live-images
cp -r /usr/share/live/images/standard .
cd standard/
lb config --source false # optional, would be faster if not build source
sudo lb build
# here we got live-image-amd64.hybrid.iso, about 417MB
# refer to for more information

Ubuntu Xenial 16.04

Automatically build with ubuntu-defaults-builder

sudo apt install -y ubuntu-defaults-builder
sudo apt install -y syslinux-utils
ubuntu-defaults-template myconfig
cd myconfig/
cd ../
ubuntu-defaults-image --package myconfig_0.1_all.deb --components main,restricted,universe,multiverse

# here we got binary.hybrid.iso and livecd.ubuntu.iso

Manually build with live-build

# First, set up the build tools and workspace.
# The scripts require that you work in /build

sudo apt-get install -y genisoimage syslinux-utils # tools for generate ISO image
sudo apt-get install -y memtest86+ syslinux syslinux-themes-ubuntu-xenial gfxboot-theme-ubuntu

sudo -i
apt-get install -y livecd-rootfs
mkdir -p /build/chroot
cd /build
cp -a /usr/share/livecd-rootfs/live-build/auto .

# All the hard work is done with live-build (lb command)
# and we have to configure it with environment variables

export SUITE=xenial
export ARCH=amd64
export PROJECT=ubuntu
export MIRROR=
export BINARYFORMAT=iso-hybrid
export LB_SYSLINUX_THEME=ubuntu-xenial

# Now we can have live-build set up the workspace

lb config --initramfs-compression=gzip 

# And finally, start the build

lb build

[2] Ubuntu live filesystem builds moved to live-build
[3] LocalizedCDImageTools - Ubuntu Wiki
[4] ISO with l10 preloaded for LoCo + UEFI
[5] UbuntuStudio/Seeds - Ubuntu Wiki
[6] Building large-scale Ubuntu derivatives using seeds - Google Docs

[7] ReleaseTeam/CDImageSetup - Ubuntu Wiki

PuDB Python Debugging Tool. A full-screen, console-based Python debugger

I highly recommened pudb, which is a full-screen, console-based visual debugger for Python. The user-interface is nicely designed for python developers.

Here is an example debugging a simple python script:
Introduction to the PuDB Python Debugging Tool

Also, another great tutorial is from Jordi Gutiérrez Hermoso.
Montreal, QC, September 14, 2015 - Jordi Gutiérrez Hermoso presents PuDB, a full-screen, console-based visual debugger for Python.
An interesting comment from the speaker:

Everyone should use a debugger

But If pdb is the only debugger you've every seen, I won't blame you not using a debugger

because the debugger is ugly, not because the source code is ugly

To install pudb on Debian/Ubuntu, you can use apt-get to install the package

# for python 2

sudo apt-get -y python-pudb
# for python 3

sudo apt-get -y python3-pudb

# pudb is even better when debugging with ipython

# so that's install ipython

sudo apt-get install ipython

How to prevent Linux from waking up due to USB devices

最近換了一台工作機,但我發現在 Ubuntu 14.04.3 上,有時候會休眠失敗。
在失敗的時候,機器有 suspend 成功,但是過2秒後機器就會自動醒來。查看了 /var/log/kern.log, /var/log/syslog, /var/log/pm-suspend.log 沒有太異常的訊息。後來上網查了一下,這可能是有 BIOS 有問題或 device 不正常運作。

在 /proc/acpi/wakeup 中,列出了數種 wakeup event, 以下面的表來說,代表了機器開放了 EHC1, EHC2, PWRB, LID0 這幾種 wakeup event

RP06      S4    *disabled
PXSX      S4    *disabled
RP07      S4    *disabled
PXSX      S4    *disabled
EHC1      S0    *enabled   pci:0000:00:1d.0
EHC2      S4    *enabled  pci:0000:00:1a.0
PWRB      S4    *enabled   platform:PNP0C0C:00
LID0      S4    *enabled   platform:PNP0C0D:00

試著開關 wakeup trigger 來找出問題

# ignore XHC device wakeup event

echo "XHC" > /proc/acpi/wakeup
# try suspend, still failed

# ignore EHC1 device wakeup event

echo "EHC1" > /proc/acpi/wakeup
# try suspend, still failed

# ignore EHC1 device wakeup event

echo "EHC2" > /proc/acpi/wakeup
# try suspend. Bingo!! It suspend successfully!

發現只要關閉 EHC2 wakeup trigger, 機器就能正常 suspend。可以放罝一個 upstart script,讓每次開機時,都自動關閉 EHC2 wakeup trigger

# /etc/init/disable-EHC2.conf
start on started dbus
stop on stopping dbus

   sudo -u root sh -c "echo 'EHC2' > /proc/acpi/wakeup"
end script


在 NAS 上架設 ubuntu chroot 環境

今天發現可以透過 Canonical Ltd. 的 linuxcontainer image 快速的在 NAS 上建立一個 chroot 環境。我的機器 cpu 是 armhf ,下面示範建立 Ubuntu 15.04 Vivid 的 chroot 環境。

步驟1:下載 Image,解開後進入 rootfs,並修改 nameserver

mkdir rootfs
tar xvf rootfs.tar.xz -C rootfs
cd rootfs
echo "nameserver" > etc/resolv.conf

步驟2:mount 需要的 sysfs

mount -t proc proc proc/
mount -t sysfs sys sys/
mount -o bind /dev dev/
mount -t devpts pts dev/pts/


chroot . su -l

使用 UPMOST DVB193 在 Ubuntu 14.04 上看數位電視

手上有一支 UPMOST DVB193 數位電視棒,最近測試了它是不是能在 Ubuntu 14.04 下運作。原本以為需要自己編譯 driver,不過把數位電視棒插入筆電, lsusb 後發現其實 driver 已被 Kernel 收錄成為 built-in driver。 Driver 的訊息如下

filename:       /lib/modules/3.19.0-28-generic/kernel/drivers/media/tuners/it913x.ko
license:        GPL
author:         Antti Palosaari <>
description:    ITE IT913X silicon tuner driver
srcversion:     3A631200871E9FCB4913859
alias:          i2c:it913x
intree:         Y
vermagic:       3.19.0-28-generic SMP mod_unload modversions 
signer:         Magrathea: Glacier signing key
sig_key:        29:4D:C0:12:70:6F:48:B7:CD:DF:63:74:E2:D7:9F:E1:B0:60:92:69
sig_hashalgo:   sha512

所以接下來只要處理好看數位電視的軟體就可以。以 apt-get 安裝好 vlc ,再用 vlc 開啟 channels.conf





sudo apt-get install -y dvb-apps
sudo apt-get install vlc
scan /usr/share/dvb/dvb-t/tw-All >channels.conf

這個 channels.conf ,就可以用 vlc 打開收看。


基於 upstream git-tree 產生 debian package patch

最近在工作上遇到一個 bug,在某些狀況下執行 lshw (02.16-2ubuntu1.2) 會 segmentation fault。找了 lshw 最新的 code 編譯執行後發現沒有這個問題。於是就開始了 git bisect。最後找到是 d048d300b5daeb44887a7fc06ddeb120119cac8a 這筆修改在 src/core/ 上面的 commit 解決了這個問題

順著這筆 commit,找到了 lshw project 的 bug report lshw segfaults, with some 16GB USB-3 sticks from Patriot 。也了解是 USB3.0 stick plugged 時會出現問題。在手上的機器上試了一下,果然是這樣沒錯。原本沒有頭緒的 random segmentation fault,現在則是有了 root cause 的 issue。

既然找到了 Solution,就開始要加 patch 進 Ubuntu source 了。為了將 patch 加到 Ubuntu 14.04 (Trusty) 目前的 lshw (02.16-2ubuntu1.2) 裡,我們還必須加上 src/core/ 的前一個 commit b79f299319f61bc80e8d38e61631cfee7521a729

# clone upstream

git clone lshw-lyonel
# clone ubuntu source

bzr branch lp:ubuntu/trusty-proposed/lshw

# make patch from upstream git tree

cd lshw-lyonel/
git format-patch b79f299^..b79f299
git format-patch d048d30^..d048d30
cd ..

# import patches to ubuntu source

cd lshw/
dquilt push -a # apply existing patches

dquilt import ../lshw-lyonel/0001-use-a-different-approach-for-scanning-SCSI-generic-d.patch
dquilt import ../lshw-lyonel/0001-presumably-fix-653.patch

# edit changelog

dch -v 02.16-2ubuntu1.3 -D trusty

# local commit


這樣就完成了,最後這一包 source 就可以拿來做 debian package 的打包,並且開始跑 Ubuntu package propose 的流程。


  1. how to add upstream git patches to an existing debian package
  2. Chapter 3. Modifying the source
  3. How To Survive With Many Patches or Introduction to Quilt