Friday, March 2, 2018

Manually creating Linux USB flash drive installations

Rev 3/4/18 (see Recent Revisions at end)


In the interest of becoming less dependent upon USB-installers to create the simple flash-drive installations which I prefer for running my "secure" PC (a Zbox without any internal storage, internet connection, or wireless capability, running a nonpersistent flash drive installation of Linux - data is stored on small, easily-concealable encrypted flash drives), I decided to try to create one manually, and found that it's easy to do in some cases, but that in general it requires a lot of knowledge about the subtleties of the boot process and about configuring bootloaders. In cases where it works, however, changing installations might be as easy as deleting the existing version of Linux from the flash drive and replacing it with another.

The basic procedure is to create a FAT32 partition on a MBR/FAT-formatted flash drive (I use the Disks and/or GParted utility for this), then install the Syslinux bootloader-chain (mbr.bin, ldlinux.sys, and ldlinux.c32 - commands to do this are included below, but there are many approaches to accomplishing the same result), extract an ISO and copy the extracted files to the flash drive (or copy the files from a bootable DVD), and create a text file named syslinux.cfg (contents listed below) to tell the bootloader what to do (if it's different than the default behavior), and place it on the flash drive along with everything else.

I did this, using a Debian DVD which I purchased from OSDisc.com as the source material. The DVD contains some "symbolic links," which are files whose icons include "shortcut"-arrows, which cannot be copied to a FAT partition. So, what you do in such cases is to right-click on them, identify the item to which they link/point, and copy that item to the flash drive. In one case, the symbolic link pointed to the directory in which it was located, i.e. the "." directory. (".." is the parent directory.) So, I just disregarded the link. Some other symbolic links pointed to a certain folder, so I copied that folder.

To do this with an ISO-file, the file has to be extracted first, by right-clicking on it, etc. The extraction-process creates a folder named after the ISO, extracts the ISO's contents, and places them in the folder. The contents (known as the "image"), not the folder, would be copied to the flash drive.

The configuration file is a text file named syslinux.cfg. To avoid having to learn how to write these configuration files, which requires considerable education, I decided to try one which I found on an installation which I made with Rufus. The contents of this configuration file are as follows:

DEFAULT loadconfig

LABEL loadconfig
  CONFIG /isolinux/isolinux.cfg
  APPEND /isolinux/

This essentially tells the bootloader to use the isolinux.cfg file in the isolinux folder (included on DVDs and in ISO's) as its configuration file, and to append certain elements in the isolinux directory to the kernel command line, to add options (which I presume are those contained in the boot-menu).


> The general Linux boot-process using the Syslinux bootloader

The boot-process begins with a few instructions which are built into the PC's firmware and executed whenever the PC is started, which loads mbr.bin from the flash drive's first sector (which isn't part of the file system), and then runs mbr.bin, which passes control to ldlinux.sys, which does some things and passes control to ldlinux.c32, which reads the configuration file and acts accordingly, typically by executing a menu-program to give the user various boot-options. These pass control to the OS-kernel, which creates an "initial ram disk" (initrd), which is a temporary file system to contain such things as drivers for hardware required for the boot-process, and starts the core the of OS, which does a lot of stuff to set up shop, and passes control to user-interface. The kernel has enough intelligence to find what it needs if you just put everything from the DVD or extracted ISO in the root directory of the flash drive.


> Detailed bootloader installation process

To install the Syslinux bootloader-chain onto a flash drive formatted as described above, enter the following commands. (Don't include the quotation-marks which I use to identify the beginning and end of each command. The commands can be copied to a "scratchpad" text-file, edited as necessary to include the specifics, then copied and pasted into the terminal with Ctrl-Shift-V.

A) "sudo syslinux -i /dev/sdX1" (X is typically b, c, or d, depending on the PC's drive-configuration at the time. The specific letter can be found by using the Disks or Gparted utility included with many types of Linux, although typically not in those with the KDE interface (there are exceptions). This installs ldlinux.sys and ldlinux.c32 in the flash drive's first partition (which in this case is the only one).

B) "cd" to the directory containing the mbr.bin file. This directory is typically /usr/lib/syslinux/mbr, and if so so the command would be "cd /usr/lib/syslinux/mbr" (without quotes).

C) Enter "sudo dd conv=notrunc bs=440 count=1 if=mbr.bin of=/dev/sdX" (without quotes). (This puts mbr.bin, which is the primary bootloader, into the master boot record of the flash drive, where the BIOS will find it when the PC is booted.)

D) Enter "sudo parted /dev/sdX set 1 boot on" (without quotes) (this sets the partition's boot-flag to make it bootable, which can also be done with GParted)

E) Simply copy the contents of a bootable Linux DVD or an extracted ISO-file (as mentioned previously) to the flash drive partition/directory where ldlinux.sys and ldlinux.c32 are located.

F) Likewise, copy the aforementioned syslinux.cfg file to the flash-drive partition/directory where ldlinux.sys etc. are located.

That's what I did, basically, and it worked, but it doesn't work in all cases. But if you find that it works for certain types of Linux, you might be able to change the installation on a flash drive by simply deleting one version of Linux and replacing it with another.


Notes

Recent Revisions

Rev 3/3/18 - Pretty much rewrote the entire thing to improve its clarity.

Rev 3/4/18 - Revised the first paragraph to reflect additional experience with the system described in this article.