Ubuntu: Install Ubuntu Server to a flash drive, permanently



Question:

First thing, I wanted to make sure, there is an understanding what this post is NOT about:

What I am looking to do is to have a home NAS (file) server that boots from a USB stick or CF card. It is irrelevant for the point of this post how the flash drive is connected (USB, IDE, some other way), what is relevant: I want to have an entire operating system installation boot and run from flash drive. The reasons I want specifically this type of installation is: I want to be able to easily make a backup copy of operating system (image the flash), I want to make sure I can swap operating system instantly and boot it (by replacing flash drive), and I want drives on a file servers be replaceable separately from OS.

I am aware that it is possible to simply format flash drive as ext2 and install on it. What I am looking for is recommendations for creating more robust solution. Here are a few items I am looking to do:

  • configure root partition as YAFF2, or LogFS, or as some other file system designed specifically for flash drives.
  • reconfigure the base OS to minimize writes (ramfs in /tmp and such)

Please submit your recommendations. I wonder if Ubuntu is even applicable for such scenario really...

I want to clarify my goal a little bit. The regular SSD drives feature firmwares that will keep the drive alive for a long time by using tricks, such as: constantly reallocating writes to new sectors. USB sticks have no such firmware, so using regular file system could kill them fast. I am looking if it is possible to emulate SSD firmware by using file system designed for flash. In a sense, I am trying to make it so that USB drive could run a system in a normal fashion, as if it was an actual SSD.


Solution:1

Use LVM. In this case you can use snapshots - instant clones of your system, so you can easily copy entire system or just migrate it between flash drives*.

And if you'd like to have sort of "Wear-leveling", you can use aufs for root file system on usb flash. So you don't have to use YAFF2 etc, ext4 will be fine.

*But /boot you will always need to copy manually + you will need to modify grub/disk's UUID. You will need rescue CD in case to resize root volume to the size of USB Flash. And in case you will broke grub read this manual.

I wrote the manual for that.


Solution:2

If you happen to have Ubuntu at HDD with LVM (Ubuntu Server utilizes LVM by default).

This scripts was tested in Ubuntu Oneiric 11.10 x86_64 3.0.0-16-server with next applications:

LVM version:     2.02.66(2) (2010-05-20)  Library version: 1.02.48 (2010-05-20)  Driver version:  4.20.0  grub-install (GRUB) 1.99-12ubuntu5  aufs - advanced multi layered unification filesystem. version 2.1-standalone.tree-3.0-rcN-20110620  

Step1. Shrink root FS to the size of your FlashDrive or smaller. Skip this step if your root FS size the same or smaller then the FlashDrive.

Boot with Resc CD mount your root partition. Run next commands:

#make the kernel realize the LVM volumes are there  lvm vgchange -a y    #Before shrinking the file system, it is good practice to check it is all in order.  # Change /dev/ubuntu/root with your root lvm partition  e2fsck -f /dev/ubuntu/root    #After the check, resize2fs can be used to shrink the file system on the LVM volume  # Change /dev/ubuntu/root with your root lvm partition. Replace 6G with your FlashDrive capacity-500Mb  resize2fs -f -p /dev/ubuntu/root 6G  #The e2fsck check command could be run again to make sure the, now significantly smaller, root file system is OK.  # Change /dev/ubuntu/root with your root lvm partition. Replace 6G with your FlashDrive capacity-500Mb  resize2fs -f -p /dev/ubuntu/root 6G    #The lvreduce command reduces the size of LVM volumes.  # Change /dev/ubuntu/root with your root lvm partition. Replace 6G with your FlashDrive capacity-500Mb  lvm lvreduce -L6G /dev/ubuntu/root    #you resided root partition !  reboot  

Step 2. Preparing the FlashDrive: for migrating

take a look at your old root UUID

blkid    #it will show something like:  #/dev/sda1: UUID="6b5085da-5b7c-4ca5-a907-1e527c87a478" TYPE="ext2"  #/dev/sda2: UUID="iZjCPX-AwGS-kEmg-zXPc-Z7ne-Xd3N-gXoPQd" TYPE="LVM2_member"    #We will need /dev/sda1's UUID later. Lets remember it (6b5085da-5b7c-4ca5-a907-1e527c87a478) for the future.    # Replace /dev/sdg with your FlashDrive. All data will be last!  fdisk /dev/sdg  make first partition (sdg1) about 500Mb as ext2/ext4  make second partition (sdg2) for example 6GB as LVM partition.    #replace ubuntu with the same LVM VolumeGroup for your root fs  #Extending VG with FlashDrive  vgextend ubuntu /dev/sdg2  modprobe dm-mirror  #Moving root partition to flash drive online :)  #Replace /dev/sdg2 with your Flash Drive's second partition  #Replace /dev/sda5 with your root drive  pvmove --verbose /dev/sda5 /dev/sdg2  #removing old root drive from VolumeGroup, so we can remove HDD from PC after manipulations. But not now.  vgreduce ubuntu /dev/sda5  #Creating boot partition.  #Replace /dev/sdg1 with your Flash Drive's first partition  mkfs -t ext4 -v /dev/sdg1  #Mounting first FlashDrive partition.  mkdir /mnt/boot  #Replace /dev/sdg1 with your Flash Drive's first partition  mount /dev/sdg1 /mnt/boot    #we will need some tools  apt-get install dump  #coping boot drive from HDD to FlashDrive first partition  #Replace /dev/sda1 (source) and /mnt/boot (destination) with your environments  (dump -0f - /dev/sda1) | ( cd /mnt/boot; restore -rf - )  #And let's not forget to restore boot sector (MBR):  # Replace /dev/sdg with FlashDrive disk path  grub-install /dev/sdg    #change UUID for /boot with the one we remember at the beginning of the Step 2 (6b5085da-5b7c-4ca5-a907-1e527c87a478).  vi /etc/fstab    #now we need to write changes to grub.  update-grub    #Now you can power off your PC.  poweroff  #Now you can replace old root FS's HDD.  #Do not forget configure your PC's BIOS to boot from your FlashDrive.  

A) If your grub loaded but it do not able to load OS at first reed this article. try ls command to figure out what grub seeing.

grub>ls  # Replace (hd0,msdos1) regard to previous output  grub>ls (hd0,msdos1)    # Replace (hd0,msdos1)/grub regard to previous output  grub>ls (hd0,msdos1)/grub    grub>configfile (hd0,msdos1)/grub/grub.cfg  

After successful booting the changes not permanent, so you have to update grub

update-grub  

B) If you do not sees grub at all, then use Rescue CD to boot.

#and install grub  #Lets figure out new name of your FlashDrive  blkid  # Replace /dev/sda with new name of your FlashDrive which system shows you  grub-install /dev/sda  #Reboot without Rescue CD and go to point A)  reboot  

Step 3. AUFS for root file system on usb flash drive.

apt-get update   apt-get dist-upgrade   apt-get install aufs-tools      # Aufs as root with apparmor. Enter dhclient3 to complain mode  # Network interface without it wount start/  apt-get install apparmor-utils  aa-complain dhclient3  # Or disable apparmor  #/etc/init.d/apparmor stop  #update-rc.d -f apparmor remove  #aptitude remove apparmor apparmor-utils    echo aufs >> /etc/initramfs-tools/modules  

Now create /etc/initramfs-tools/scripts/init-bottom/__rootaufs file and place next script:

#!/bin/sh  #  Copyright 2008 Nicholas A. Schembri State College PA USA  #  #  This program is free software: you can redistribute it and/or modify  #  it under the terms of the GNU General Public License as published by  #  the Free Software Foundation, either version 3 of the License, or  #  (at your option) any later version.  #  #  This program is distributed in the hope that it will be useful,  #  but WITHOUT ANY WARRANTY; without even the implied warranty of  #  MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the  #  GNU General Public License for more details.  #  #    You should have received a copy of the GNU General Public License  #    along with this program.  If not, see  #    <http://www.gnu.org/licenses/>.    # https://help.ubuntu.com/community/aufsRootFileSystemOnUsbFlash#rootaufs_Script    # Thank you Voyage Linux for the idea, http://voyage.hk/ Great job on release 0.5  #  # Tested with 8.04.1  # tested with 9.10  # tested with debian live 6.0.1  #  # ****************************************************************************************  #   #                                 Change log  #  # 2008.08.01  Added debugging comments in "drop to a shell" section. grub option aufs=tmpfs-debug will stop the init script.  #             reviewed *********** fix fstab on tmpfs ******************  #             rootaufs failed when system was booted with /dev/xxx and fstab had uuid= info.  #             BlaYO pointed out the best and simplest solution was to use grep -v. Grep replaces a sed one liner.  #             Add the comment block to fstab  #               # 2009.12.07 Corrected issue caused by Apparmor.  #            Name changed to __rootaufs.  #   # 2011.08.19 Changed if condition to avoid issue (sh: bad number) when $aufsdebug is not set.  #            Now checks exists apparmor before delete.  #  # 2011.08.20 For work correctly with Debian Live 6.0.1 (http://live.debian.net/) two lines must be removed from rc.local modification part:  #             'mount -f  /ro'  #             'echo aufs-tmpfs /rw tmpfs rw 0 0 >>/etc/mtab'  #            case $1 in  prereqs)      exit 0      ;;  esac    export aufs    for x in $(cat /proc/cmdline); do       case $x in       root=*)          ROOTNAME=${x#root=}          ;;      aufs=*)          aufs=${x#aufs=}          case $aufs in          tmpfs-debug)              aufs=tmpfs              aufsdebug=1              ;;          esac              ;;      esac  done      if [ "$aufs" != "tmpfs" ]; then      #not set in boot loader       #I'm not loved. good bye      exit 0  fi          # This is a simple overview of the steps needed to use aufs on the root file system and see the /rw and /ro  branches.  # initramfs init-botton script   # move the root file system to aufs/unionfs readonly /ro  # root is mounted on ${rootmnt}  # create tmpfs on /rw  # create a aufs using /ro and /rw  # put some files on the tmpfs to fix mtab and fstab   # move aufs to rootmnt to finish the init process.  # No changes to the root file system are made by this script.  #  #  Why!  #  This will allow you to use a usb flash drive and control what is written to the drive.  #  no need to rebuild the squashfs file just to add a program.   #  boot to single user mode.  The system works the way you expect. boot aufs=tmpfs and no changes are written to the flash.  #  run ubuntu on an eeePC .      # Install   # Install ubuntu 8.04 Hardy. Hardy has aufs installed by default  # apt-get update  # apt-get dist-upgrade  # apt-get install aufs-tools  # echo aufs >> /etc/initramfs-tools/modules  # put this file in /etc/initramfs-tools/scripts/init-bottom/rootaufs  # chmod 0755 rootaufs  # # clean up menu.lst   # update-grub  # update-initramfs -u  # vi /boot/grub/menu.lst  # add aufs=tmpfs to the default entry.   # do not add this line to single user mode.  # boot to single user mode in order to install software.   # note: if your home account is on the root file system, your files are in ram and not saved.  #       echo   echo "       root-aufs:  Setting up aufs on ${rootmnt} as root file system "  echo     modprobe -q --use-blacklist aufs  if [ $? -ne 0 ]; then      echo    root-aufs error:      Failed to load aufs.ko      exit 0  fi    #make the mount points on the init root file system  mkdir /aufs  mkdir /rw  mkdir /ro    # mount the temp file system and move real root out of the way  mount -t tmpfs aufs-tmpfs /rw  mount --move ${rootmnt} /ro   if [ $? -ne 0 ]; then      echo    root-aufs error:    ${rootmnt}  failed to move to /ro      exit 0  fi      mount -t aufs -o dirs=/rw:/ro=ro aufs /aufs  if [ $? -ne 0 ]; then      echo    root-aufs error:      Failed to mount /aufs files system      exit 0  fi      #test for mount points on aufs file system  [  -d /aufs/ro ] || mkdir /aufs/ro  [  -d /aufs/rw ] || mkdir /aufs/rw    # the real root file system is hidden on /ro of the init file system.  move it to /ro   mount --move /ro /aufs/ro  if [ $? -ne 0 ]; then      echo    root-aufs error:      Failed to move /ro /aufs/ro       exit 0  fi    # tmpfs file system is hidden on /rw  mount --move /rw /aufs/rw  if [ $? -ne 0 ]; then      echo    root-aufs error:      Failed to move /rw /aufs/rw       exit 0  fi        #*********** fix fstab on tmpfs ******************  # test for /dev/sdx   # this is not on the real file system.  This is created on the tmpfs each time the system boots.  # The init process will try to mount the root filesystem listed in fstab. / and swap must be removed.    # the root file system must be mounted on /ro not on /    if [ "0$aufsdebug" -eq 1 ]; then      echo  "   root-aufs debug:    Remove the root file system and swap from fstab "      echo       echo       echo  "         ROOTNAME $ROOTNAME "      echo  "         resume   $resume   "      echo       echo  '     BlaYO pointed out that grep can be used to quickly remove '      echo  '      the root file system from fstab. '      echo       echo  '     Thank you BlaYO for the debug info.'      echo    fi  # old code  # I'm sure that sed can do this in one step but I want to correct on the rootname  not matching the root in fstab.  #cat /aufs/ro/etc/fstab|sed -e s/$ROOTNAME/\#$ROOTNAME/ -e s/$resume/\#$resume/ >/aufs/etc/fstab      #Add the comment block to fstab  cat <<EOF >/aufs/etc/fstab  #  #   RootAufs has mounted the root file system in ram  #  #  This fstab is in ram and the real fstab can be found /ro/etc/fstab  #  the root file system ' / ' has been removed.  #  All Swap files have been removed.   #    EOF    #remove root and swap from fstab  cat /aufs/ro/etc/fstab|grep -v ' / ' | grep -v swap >>/aufs/etc/fstab    if [ $? -ne 0 ]; then      echo    root-aufs error:      Failed to create /aufs/etc/fstab       #exit 0  fi          # add the read only file system to fstab  #ROOTTYPE=$(/lib/udev/vol_id -t ${ROOT})  ROOTTYPE=$(cat /proc/mounts|grep ${ROOT}|cut -d' ' -f3)  ROOTOPTIONS=$(cat /proc/mounts|grep ${ROOT}|cut -d' ' -f4)  echo ${ROOT} /ro $ROOTTYPE $ROOTOPTIONS 0 0 >>/aufs/etc/fstab      # S22mount on debian systems is not mounting  /ro correctly after boot  # add to rc.local to correct what you see from df  #replace last case of exit with #exit  cat /aufs/ro/etc/rc.local|sed 's/\(.*\)exit/\1\#exit/' >/aufs/etc/rc.local    echo mount -f  /ro >>/aufs/etc/rc.local     # add back the root file system. mtab seems to be created by one of the init proceses.   echo "echo aufs / aufs rw,xino=/rw/.aufs.xino,br:/rw=rw:/ro=ro 0 0 >>/etc/mtab" >>/aufs/etc/rc.local  echo "echo aufs-tmpfs /rw tmpfs rw 0 0 >>/etc/mtab" >>/aufs/etc/rc.local   echo exit 0 >>/aufs/etc/rc.local     #  Copyright 2008 Joaquin I. Bogado Garcia  #fix para apparmor, se desactiva y listo ( From the lethe project. )  [ -e /scripts/init-bottom/_apparmor ] && rm /scripts/init-bottom/_apparmor  [ -e /aufs/etc/init.d/apparmor ] && rm /aufs/etc/init.d/apparmor      #build remountrw  echo \#!/bin/sh >/aufs/bin/remountrw  echo mount -o remount,rw ${ROOT} >>/aufs/bin/remountrw  chmod 0700 /aufs/bin/remountrw    #build remountro  echo \#!/bin/sh >/aufs/bin/remountro  echo mount -o remount,ro ${ROOT} >>/aufs/bin/remountro  chmod 0700 /aufs/bin/remountro    # This should drop to a shell. (rewrite)  if [ "0$aufsdebug" -eq 1 ]; then      echo      echo "   root-aufs debug:    mount --move /aufs ${rootmnt} "      echo       echo '   root-aufs debug:   init will stop here.   '      echo        exit 0  fi    mount --move /aufs ${rootmnt}    exit 0  

After we need to change permissions for this file:

chmod 0755 /etc/initramfs-tools/scripts/init-bottom/rootaufs  

Step 4. Changing GRUB2.

cat /etc/grub.d/10_linux|sed  s/'linux_entry "${OS}" "${version}" false'/'linux_entry "RO: ${OS}" "${version}" false'/ |sed  s/'"${GRUB_CMDLINE_LINUX} ${GRUB_CMDLINE_EXTRA} ${GRUB_CMDLINE_LINUX_DEFAULT}"'/'"${GRUB_CMDLINE_LINUX} ${GRUB_CMDLINE_EXTRA} ${GRUB_CMDLINE_LINUX_DEFAULT} aufs=tmpfs"'/ >/etc/grub.d/06_custom    sed -i.orig -e \  "/{GRUB_DISABLE_RECOVERY}\" != \"xtrue\" ]; then/s/^.*$/    \  if [ 0 ]; then/" \  /etc/grub.d/06_custom      sed -i.orig -e \  "/in_submenu=:/s/^.*$/    \  /" \  /etc/grub.d/06_custom    sed -i.orig -e \  "/\"Previous Linux versions/s/^.*$/    \  echo ''/" \  /etc/grub.d/06_custom    rm /etc/grub.d/06_custom.orig    chmod 755 /etc/grub.d/06_custom    update-grub  update-initramfs -u  


Solution:3

I want to ask the same question and do exactly the same thing too. Does this answer help?

How do I optimize the OS for SSDs?

these links also looks very useful although i can not vouch for the accuracy of the information

http://bernaerts.dyndns.org/linux/53-debian-server-compact-flash

http://www.chrisnewland.com/using-linux-tmpfs-to-reduce-disk-writes-for-power-saving-and-longer-cf-lifespan-186

I guess once you have done those things you will need to monitor to see what your specific applications are writing to the boot flash drive. Is there any easy way to see what is writing to a given drive?


Note:If u also have question or solution just comment us below or mail us on toontricks1994@gmail.com
Previous
Next Post »