Terry Gray
Original: 14 Jul 2003
Revised: 04 Jul 2004 (what else would one do on Independence day?)
Introduction
This document is inspired by and derived from an
article at www.knoppix.net
When I started this project, there were some bugs in that guide (as there
may be in this one.) Specifically:
This revision (July 2004) reflects changes introduced in recent versions of Knoppix, as well as providing additional information on configuration and customization options. For reference, my original version is here
This is guide is intended to explain how to put a Knoppix distribution onto a hard drive or a Compact Flash IDE drive without actually installing it. (The knoppix.net HowTo page uses the term "HD Based" to contrast this scenario with an "HD install"). A drive with such a "non-installed" OS is used just like a "live CD", with the drive serving in lieu of the CDROM. This allows the CDROM drive to be used for other things, and is typically faster and quieter than using the CDROM. To reiterate: this is NOT a normal Linux install. The Knoppix kernel will still execute from RAMDISK, just as it does when running from CDROM.
Motivation
One might wonder why this is an intersting thing to do. Why not just do a normal hard drive installation? Several reasons: First, one of the beauties of Knoppix is that it represents a complex unit that can be upgraded enmasse, with all local customization kept completely separate (not intermixed with the "installed" files). Second, normally-installed operating systems really like to write to their disk drives a lot. If you are a silent computing freak, this can be annoying, since the drives rarely spin down and sit quietly, waiting until you want to do something with them. So running Knoppix-style out of RAM Disk is an ideal solution for such things as small file/print servers, media servers, and maybe even basic desktop computers. Third, if you want to use a Compact Flash IDE drive to hold the OS bits (i.e. a CF card plugged into a CF-IDE adapter), thereby eliminating all the rotating machinery needed to keep the OS functioning, this RAMdisk-based mode of operation is again ideal, because it avoids the stress of swap files or log files that wear out CF cards (since they have limited life for writing, as compared to reading data.) Finally, (in contrast to just booting from a live-CD) there is an opportunity to adjust the default boot options to your liking, so you don't have to type in your desired kernel boot options every time you start Knoppix. (Examples include "home", "myconf", and "dma".)
Don't the "tohd" and "fromhd" codes make this guide obsolete?
Knoppix 3.3 introduced cheatcodes "tohd" and "fromhd" (and "toram") to load Knoppix onto a hard drive, and subsequently execute from the saved image. On first glance, I thought these might obviate the need for my procedures to install Knoppix on a CF or conventional hard drive, but it turns out that while using the "tohd" option does put the needed Knoppix files on the drive or CF card, it does not configure the drive for booting Knoppix. So these procedures are still relevant.
IDE Compact Flash vs. USB Compact Flash
The procedures herein describe how to boot from a compact flash card plugged into an IDE drive connector via a CF-IDE adapter card. In this context, the CF card behaves "just like" an IDE hard drive. Now that 1GB USB flash drives are starting to come down in price, would that be a preferred option? And would these procedures work for external USB drives? The answer to the first question is "maybe", and the answer to the second is "I don't know".
The advantage of a USB flash drive is convenience: you don't need to acquire a CF-IDE adapter card, and if you want to move the drive to a different system, you don't have to "crack the case"... assuming your CF system has a case :)
On the other hand, at this writing, 1GB USB flash drives cost more than twice as much as basic vanilla CF cards, which are rapidly approaching the $100 price point. Another advantage of the CF-IDE approach described here is that virtually any computer can boot from a CF card in a CF-IDE adapter board, because it behaves just like a hard drive. In contrast, to boot from a USB flash drive, the motherboard's BIOS must be fairly modern and explicitly provide for a "USB HDD" boot option. Will these procedures work for USB flash drives? I don't know. But I am curious, so let me know if you try it. Finally, the read performance of a CF-IDE arrangement ought to be better than that of a USB2 flash drive, but that is conjecture on my part.
Latest Revision
In Knoppix 3.4, the boot loading method changed from SYSLINUX to
ISOLINUX, so the steps in my earlier version of this document will no
longer work. Also, Knoppix 3.4 includes both Linux 2.4 and 2.6 kernels, so
some path names in the boot loader configuration (lilo) have changed. The
procedures outlined in this revision have been tested with Knoppix version
3.4 of 5/17/04.
There are several steps to the this installation process:
What we'll need:
Finding a CF-IDE Adapter
Originally I implemented this approach on an existing hard drive, but with the decline of prices for 1GB CF cards, I decided to push further toward truly solid-state/no-moving-part systems. To do that, I also needed to obtain a CF-IDE adapter. Google revealed several sources:
I used PriceWatch to find a source for well-priced 1GB Toshiba Compact Flash cards, and ended up with a Toshiba card which appears to work fine. Ironically, Klaus Knopper released a *DVD* version of Knoppix at LinuxTag the same week I started this project, so 1GB is now a bit limiting... but I think I'll pretend I didn't know that for awhile.
Preparing the Staging System
We can use any Linux system to accomplish the setup, but I usually just boot a Knoppix live-CD in the target system. We'll need a staging drive to store and access the original Knoppix ISO file, as well as the target drive for the new installation.
The target and staging drives can be one and the same drive if big enough (at least 1.5GB free), but if you want to use a 1GB flash drive for the target, you'll need a separate HD (or space on an existing drive) for staging (in particular, to hold the Knoppix ISO image file.)
We need a source of Knoppix files. I'm not sure whether you can mount the CDROM to copy from when Knoppix is executing from it. To avoid that issue, we'll assume that a Knoppix ISO file can be made available, just as when the file has been been downloaded from one of the Knoppix mirror sites. The version to be installed need not be the same version as that running, but keeping them the same avoids potential anomolies between versions of software used during installation, notably lilo.
You can use the new/target disk to store it if it has at least 1.5GB available, or you can use any other disk as a place to save the Knoppix ISO file. For this example, we'll save the ISO file on a separate staging drive as /mnt/hdb1/newknoppix.iso
For achieving the goal of a system running off of Compact Flash, we first need a system with a CDROM, a hard drive for staging, and a CF adapter to hold the CF card (plugged into an IDE drive connector).
The device name for the target drive depends on how it is attached
to the computer, and whether it is configured via hardware jumper on the
drive (or CF adapter card) as a master or slave:
MASTER on PRIMARY IDE == /dev/hda
SLAVE on PRIMARY IDE == /dev/hdb
MASTER on SECONDARY IDE == /dev/hdc
SLAVE on SECONDARY IDE == /dev/hdd
I recommend making the CF card the "HDA" drive. The following instructions assume that the target drive is /dev/hda and the staging drive is /dev/hdb
Remember that you can use the target drive for staging if it's big enough (not the case if the target is a 1GB compact flash card), but if using the same drive for both target and staging, change "hdb1" to "hda1" in the command scripts below.
Once the drives are properly attached, boot the staging system. If using the Knoppix live-CD, be sure to configure the PC BIOS to boot first from CDROM, then from hard drive. Then boot using your Knoppix CDROM or equivalent.
We need a root shell. Either select one from the KNOPPIX menu under the KDE start button, or start a regular terminal window and enter the command: sudo bash
If you are starting with a fresh hard drive or CF IDE drive, or want to make a fresh start with an existing drive, you will need to format it and make Knoppix aware of the new configuration. We will use a tool such as "qtparted" or "cfdisk" to partition and format the CF disk.
First step is to decide the destination/target drive layout, which in turn depends whether or not you want to use the space not needed for the system image files for something else, e.g. saving configurations or a persistent home directory --two important features of Knoppix.
If you want to use the same target drive for saving Knoppix configs or for the Knoppix "persistent home", and/or saving configuration data, I suggest making separate partitions for each purpose. Knoppix will take exclusive control of the partition it boots from, so saving configs to it or using it for your home directory will not work. Ultimately I ended up dividing the 1GB CF card/drive into three partitions: one for the OS, one for Knoppix config file storage, and one for my "persistent home" directory.
Partitions on 1GB CF card: hda1: 700MB (690 needed) hda2: 260MB (for persistent HOME) hda3: 15MB (for MYCONF configuration data)
Alternatively, you can format the CF drive as one single bootable partition and use a separate USB flash drive or two for saving configs and/or persistent home. This wastes the unused space on the 1GB CF card, but permits you to easily carry your home or config files to different systems.
I chose to have a separate dedicated partition for the persistent home feature of Knoppix, rather than use the facility to create a loopback file for HOME that shares an existing partition. This makes it easier for me to deal with expanding the HOME partition if needed later, or to access HOME files in other contexts than the running Knoppix system. On the other hand, if you use the "knoppix.img" loopback file to save the HOME files in, you can use a single partition to share both it and the saved configuration files, knoppix.sh and configs.tbz. Having one fewer partition to think about is a plus, and you can also simplify the boot parameters a bit, by using "scan" as the argument for both HOME and MYCONF, but for the moment I'm keeping the configs and HOME on separate partitions.
NB: There is a bug in the knoppix-autoconfig script (in ver 3.4) such that the HOME=scan boot parameter will fail to detect the home directory if the persistent home setup script was told to use an entire partition rather than a loopback image file. Hence, the HOME partition must be explicitly specified in the boot parameter in this case.
NB: Another gotcha may occur if you use separate partitions for HOME and MYCONF and use the "MYCONF=scan" boot parameter. If you use "scan", don't put a file with the same name Knoppix uses for configs (knoppix.sh or configs.tbz) in the HOME partition, unless you reverse the hda2 and hda3 partition assignments and make HOME be hda3. This is because the knoppix-autoconfig script searches the lower numbered partitions first. If it happens to find a config file in hda2 before getting to looking for one on hda3, it will use it --which (if it were an actual knoppix config file pair) would be OK except that after unpacking and restoring the config files, the script unmounts the partition it found the config files on. This is not good if that partition is also serving as the persistent HOME.
To recap: I set HOME to be an entire partition rather than a loopback file, so that it's easier to manage from outside the running system, or deal with running out of space/expansion... BUT that means I can't easily use the same partition for both HOME and MYCONF because MYCONF processing results in unmounting HOME partition. I could probably compensate by adding an additional "mount --bind" command (which maps the persistent HOME to the /home/knoppix path) somewhere near end of boot sequence, but complexity level is already higher than I'd like. So I choose to use three separate partitions on the CF disk, and because I felt "hda2" was the most comfortable partition number for HOME, I need to either explicitly specify that partition in the boot parameter or make sure I don't accidentally put files named knoppix.sh or configs.tbz in HOME.
To implement the three partitions listed above, I used the "qtparted" program, found under the "System" menu in Knoppix. It is pretty intuitive. I used ext2 filesystem format for the HOME and MYCONF partitions (because that's what Knoppix uses), but for the main boot partition (hda1) I chose to format as "fat16", just to make sure the bootloader wouldn't have any trouble finding the boot files.
After writing the partition table and boot record, it is saftest to reboot the system before trying to do anything with that drive.
We need to get a booloader installed on the target disk in order to boot an OS from that disk. In Linux land, "lilo" and "grub" are the common hard drive bootloaders. Many swear by Grub, but I have not used it yet and Lilo is included in Knoppix, so that's what we'll use here. Lilo is not intended to boot from ISO images, and it is not what Knoppix uses to boot from CDROM. Originally, Knoppix used a boot tool called "Syslinux", which emulated a floppy boot disk. Beginning with Knoppix 3.4, a related boot tool called "Isolinux" is used, which was designed specifically for booting ISO CDROM images. ISOLINUX avoids the floppy emulation "boot.img" method used previously (which among other things, limits the size of the kernel). Note that Lilo (or Grub) would be an appropriate bootloader for either an actual hard drive, or a Compact Flash card emulating a hard drive. (As noted previously, I'm not sure whether they would work with a USB flash drive.)
Since we're going to use the lilo bootloader, which is different than the Syslinux or Isolinux loaders used on the Knoppix CD, we need to create a lilo.conf file. For now we'll create it on our staging drive, /dev/hdb1
Note that Lilo really wants you to boot from /dev/hda, so don't try to make /dev/hdc be the target, like I originally did. When Lilo gets ready to write the boot block to /dev/hdc, it will refuse.
With the advent of Knoppix 3.4, the lilo.conf file needed to change to reflect new path names, and (due to a change in the newer version of lilo) the "boot" path argument needed to be fully qualified.
By convention, two critical files in the boot process were often called "vmlinuz" and "miniroot.gz". Beginning with Knoppix 3.4, two Linux kernels were included, and the conventional names were modifed as follows:
Let's now create the lilo.conf file on your staging drive. There is a template Lilo config file available on Knoppix in /etc/lilo.conf, but quite a few changes would be needed (unless it was updated in 3.4 --I didn't look.) Alternatively, you can copy and paste the following lines using your favorite Knoppix text editor, e.g. vim /mnt/hdb1/lilo.conf
Here is the lilo.conf file that I am using:
#================================================================== # /etc/lilo.conf - See: `lilo(8)' and `lilo.conf(5)', # --------------- `install-mbr(8)', `/usr/share/doc/lilo/', # and `/usr/share/doc/mbr/'. # +---------------------------------------------------------------+ # | Don't forget to run `lilo' after you make changes to this | # | conffile, `/boot/bootmess.txt', or install a new kernel. | # +---------------------------------------------------------------+ #install=/boot/boot.msg backup=/dev/null #don't bother to save old MBR state map=/boot/map #where lilo puts its map prompt #prompt for which version to boot timeout=150 #but only wait 15 seconds menu-title=" Terry's Knoppix Box " lba32 #enable large disk support vga=791 #assume 1024x768 screen boot=/dev/hda #put boot loader in MBR of /dev/hda root=/dev/hda1 #set root to be /dev/hda1 # This segment defines my default, and boots the 2.4 kernel # It specifies the HOME and MYCONF locations, and enables DMA image=/boot/isolinux/linux24 #file containing image to boot label=Knoppix2.4 initrd=/boot/isolinux/minirt24.gz #initial ram disk contents read-only append="ramdisk_size=100000 init=/etc/init lang=us apm=power-off hda=scsi hdb=scsi hdc=scsi hdd=scsi hde=scsi hdf=scsi hdg=scsi hdh=scsi nomce quiet BOOT_IMAGE=/KNOPPIX/KNOPPIX home=/mnt/hda2 myconf=/mnt/hda3 dma" # This option boots the 2.6 kernel image=/boot/isolinux/linux26 #file containing image to boot label=Knoppix2.6 initrd=/boot/isolinux/minirt26.gz #initial ram disk contents read-only append="ramdisk_size=100000 init=/etc/init lang=us apm=power-off hda=scsi hdb=scsi hdc=scsi hdd=scsi hde=scsi hdf=scsi hdg=scsi hdh=scsi nomce quiet BOOT_IMAGE=/KNOPPIX/KNOPPIX home=/mnt/hda2 myconf=/mnt/hda3 dma" # This option uses HOME=SCAN and MYCONF=SCAN instead of specifying # their locations image=/boot/isolinux/linux24 #file containing image to boot label=Knoppix24-SCAN initrd=/boot/isolinux/minirt24.gz #initial ram disk contents read-only append="ramdisk_size=100000 init=/etc/init lang=us apm=power-off hda=scsi hdb=scsi hdc=scsi hdd=scsi hde=scsi hdf=scsi hdg=scsi hdh=scsi nomce quiet BOOT_IMAGE=/KNOPPIX/KNOPPIX home=scan myconf=scan dma" # This option does not try to find configs or a persistent home image=/boot/isolinux/linux24 #file containing image to boot label=Knoppix-NOSCAN initrd=/boot/isolinux/minirt24.gz #initial ram disk contents read-only append="ramdisk_size=100000 init=/etc/init lang=us apm=power-off hda=scsi hdb=scsi hdc=scsi hdd=scsi hde=scsi hdf=scsi hdg=scsi hdh=scsi nomce quiet BOOT_IMAGE=/KNOPPIX/KNOPPIX dma" #==================================================================
Notes
Notes from the previous version of this document, based on Knoppix 3.3:
The instructions below may lend themselves to becoming a script for installing new Knoppix releases as they become available, but I have not yet tested that idea.
Notes
Annotated Command Script
#==================================================== # Running KNOPPIX from HD or CF Without Installing it # Terry Gray http://staff.washington.edu/gray # 12 Jul 2003... Revised July 2004 # Inspired by http://www.knoppix.net/docs/index.php/HdBasedHowTo # but the original didn't work for me... # Current Knoppix as of this writing: 3.4 of 17 May 04 # Assumes target drive is /dev/hda and is already formatted # Assumes source ISO file is in /mnt/hdb1/newknoppix.iso # Assumes the desired lilo.conf is in /mnt/hdb1/lilo.conf # Mount the new target drive # Assumes Knoppix recognized it, and it is by now formatted mount /dev/hda1 # We need to mount the ISO file using the loopback driver. # Start by creating a mount point directory; we'll call it "staging" mkdir /mnt/staging mount -t iso9660 -o loop,ro /mnt/hdb1/newknoppix.iso /mnt/staging # Now copy the contents of the mounted ISO to the target drive cp -af /mnt/staging/* /mnt/hda1 # We'd only need the "f" flag for later updates to blow away the old files. # copy the lilo config file cp /mnt/hdb1/lilo.conf /mnt/hda1/boot # lilo writes the necessary boot sector info to the target drive, # but it also needs to write a map file somewhere... # The following command creates a sym link from the read-only knoppix # /boot directory to the read-write /mnt/hda1/boot directory: ln -sf /mnt/hda1/boot / # Now run lilo: lilo -C /mnt/hda1/boot/lilo.conf # you could also say # lilo -C /boot/lilo.conf # which is equivalent due to the preceding symbolic link command. # lilo should say "Added Knoppix" and maybe also complain about a # few other things, but "fatal" is the result you need to avoid. # Now we're ready to reboot: reboot # Be sure to take the CDROM out when you get the Knoppix prompt to do so, # and make sure your BIOS is configured to boot from a hard drive. # If all goes well, you'll reboot from the new target drive. #====================================================
NB: This section may more properly belong in Chapter 5: A Nearly Noisless Knoppix Server, and I have added it there, but for the time being decided to leave it here too for convenience.
In this section I'll note a few things I've learned about persistent homes, saving configurations, and local/personal mods to the boot sequence. A key for me was to understand the sequence of events in the knoppix-autoconfig script. Here is a rough outline of relevant processing in the autoconfig script:
One of my goals for this project was to find a way to add my own additions to the bootup process without having to change files on the boot partition every time I want to make a change. Originally I wrote a simple script to append a command to the standard knoppix.sh script in the MYCONF partition, but I would need to remember to do this every time I ran the Knoppix SaveConfig script.
In a www.knoppix.net forum posting, I found:
"To start a service on boot add a script to /etc/init.d and do:
sudo update-rc.d
Alas, the above didn't work for me... the Knoppix "save config" script evidentally doesn't save the resulting /etc/ links from update-rc.d so that method only applies to a "conventionally installed" Linux system.
Another note on the knoppix.net forum observed that inserting a modified "knoppix.sh" earlier in the search path that the knoppix config script uses might do the trick, but if you then want to invoke the real knoppix.sh then you have to be mindful that it assumes the config.tbz file is in the same place as the knoppix.sh and that directory path is passed as an argument.
I got close to success with this strategy, by putting a knoppix.sh file containing my local additions into /mnt/hda1 --but knoppix-autoconfig complained about something, and while examining the autoconfig script, I noticed that there is already provision for executing an "extra" knoppix.sh, if it is located in /cdrom/knoppix/knoppix.sh so I just needed to move my "local script invoker" routine from /mnt/hda1/knoppix.sh to /mnt/hda1/knoppix/knoppix.sh (Note that /mnt/hda1 becomes linked to /cdrom during booting.)
My local-script-invoker in /mnt/hda1/knoppix/knoppix.sh currently consists of:
#!/bin/sh echo "------------------------------------------" echo In /cdrom/knoppix/knoppix.sh... Trying /etc/init.d/knoppix-local.sh /etc/init.d/knoppix-local.sh echo "------------------------------------------" echo Now trying /home/knoppix/knoppix-local.sh /home/knoppix/knoppix-local.sh echo "------------------------------------------"
This redirection script gives me two places where I can add local customizations without having to modify the boot partition (which is hard to do while running from that partition). Only one choice is needed; I wasn't sure which would work better (the relevant filesystem has to be successfully mounted by the time the autoconfig script gets to this point). However, both locations seem to work fine, and it is more convenient to directly manipulate a file in the persistent home than to edit a file in /etc/init.d and have to remember to re-save the Knoppix config. So I may delete the first part and just use the invocation of /home/knoppix/knoppix-local.sh
This approach allows me to easily customize the script in my persistent home directory partition at /mnt/hda2. In contrast, if I want to change the modified script I put in /mnt/hda1/knoppix , aka /cdrom/knoppix , I need to actually boot from cdrom, not from /dev/hda1 , because the boot partition is not writeable while knoppix is running.
Customization Summary
I make three classes of personal customizations:
Finally, I'll note that the
key to setting your persistent home as an entire partition is the
following command done by the
knoppix-autoconfig script:
mount --bind /mnt/hda2 /home/knoppix
The --bind construct makes the two points in the namespace to
appear/behave identically, which is a very convenient mechanism... but sub
directories below the --bind mount point are not necessarily
equally visible from either root path; their visibility depends on the
paths used to create and view them, so be careful!
I hope this information is useful to others, and I would welcome feedback. (Follow the TEG HOME link below for contact info.)
This is one of a series of articles describing my silent computing
adventures. This
link will take you to the beginning of the story.