Flytech box Linux router training manual

June 24th 2002 Ver 1.3

1. Introduction

The Flytech boxes [1] that we use for our routers are small PCs marketed as Point Of Sale terminals (POS). Because they are fully functional PCs and small in size they are very useful for many other tasks, not to mention that they cost much less than Cisco routers.

Because we run the Linux OS on them we have much more flexibility on how they are set up and configured. The trade off is that you do have to plan out carefully exactly what you want them to do. In addition to routing you can run other services on them such as a web server to display the status of the network etc...

The Particular model of Flytech that we are using have to PCI slots or one PCI and one ISA. Since our links out of Thimphu use the existing microwave network to get close to our sites we have an E1 card in one slot to connect to free E1 slices of this network. The other PCI slot has a four port 10/100 card in it. The motherboard itself has a built in 10/100 port. This configuration gives you five 10/100 ports and one E1. You can also get multiport E1 cards from SBE if you want to connect to several at one site. You could also put two four port 10/100 cards in these boxes for a total of nine ports.

You can buy them with various speed CPUs and amounts of RAM. Some example costs are about $500 for a 500MHz 64MB, 30Gig HD, and $815 for a 850MHz 512MB 40Gig HD. The four port cards are about $340 and the single port E1 cards are about $900.

2. Installation

The hardware part of the installation is fairly straight forward. Just find a shelf to put the box on and connect a monitor and keyboard. You should power it with a UPS or inverter since the network must stay up even when there is a power failure. The power supply auto detects 120 or 240 Volt AC.

Next connect the network cables to free ports on the back. Since the microwave network uses 75 ohm coax to provide the E1 connections and the E1 card uses UTP cable we need a little Balun to convert between them. Mainly at this stage you just need to keep track of which ports you plug the cables into. Ideally the port on the motherboard would be eth0 and the four on the quad port card numbered from left to right become eth1, eht2, eth3, and eth4. The E1 port is hdlc0.

3. Configuration

3.1 Introduction to networking

Well I was looking for a quick introduction to networking and routing, and found a couple, [2], and [3] , but they weren't quite what I was looking for. So for now this will have to do. OK there are other protocols than IP but our network is an IP network like the Internet so that is what we will cover here.

First of all every device that you can talk to has an IP address, which is like a telephone number. Thats how you call the device. An IP address is four bytes long, that means it has 32 bits total. IP addresses are written with the byte values separated by dots. For example: 202.144.156.240, or 10.10.5.128

Next there is the netmask it tells you who is on the same network with you. It's like your local calling area. You can call anyone on your local network directly but if you want to talk to someone on another LAN somewhere else then you will have to go through a gateway. This is like an operator assisted call. After you pass your packet to the gateway it figures out how to deliver it. All a netmask  does is tell you what portion of the IP address contains the host address on the local network. The rest of it is just an address for local network itself, and as long as that part of the address is the same as the local network address then no gateway is needed to deliver it because all machines on the LAN are listening for packets sent to them. Here are some example netmasks: 255.255.255.0,  255.255.0.0, 255.255.255.240, 255.255.255.255

If you display these netmasks as binary numbers then you will see that the left hand side is all ones  and some number of bits on the right are all zeros. The first is a common class C subnet which means that the last byte contains the host portion of the address. So you can have hosts numbered from 1 to 254 (addresses 0 and all ones are special and can't be used for hosts). The next one is a class B which has a 16 bit host portion so it can have about 64,000 hosts on the same network. Next we have a small subnet that ends in .240 this gives you four bits of host address space so you could have hosts numbered from 1 to 14, remember to skip the first and last values. Here is a little trick. If you take 2 to the power of the number of bits you want and subtract that from 256 it will give you the netmask you need. Lets try it:

If we want a 5 bit subnet then 2^5 is 32. So 256 - 32 is 224, so our full netmask would be 255.255.255.224. Now don't forget we can't use all 32 addresses in our 5 bit subnet. The usable space is always 2^N-2 so for this case it would be 2^5-2 which is 23-2 or 30. So if you wanted to have say, 31 machines on a subnet you would have to go up to a 6 bit netmask which would give you 2^6-2 = 62 possible addresses.

The two mysterious addresses that you can't use are the ones where the host portion are all zeros and all ones. The first is the network address and the other is the broadcast address. The broadcast address is used when you need to ask all devices something. So actually any device will respond to packets sent specifically to their address and also to packets sent to the broadcast address.

Last but not least you sometimes need to specify a gateway to handle traffic outside of your LAN. The gateway address must always be for some machine in your LAN so it has a local address. The way gateways work is that they another interface on another LAN or even several interfaces and they route packets between them. Its like a telephone exchange routing the long distance calls onto the microwave link.

So when you set up a network interface you need to specify several things, an address, netmask, and probably a broadcast address though some configuration software will calculate the broadcast address for you. Then you may also want to set a gateway for the LAN this interface is attached to.

3.2 Basic commands

There are some simple commands to set and change these values for an interface. First each interface has a name such as eth0, eth1, hdlc0, etc... You can list all the active interfaces by using the command ifconfig. Here is an example:
# ifconfig 
eth0 Link encap:Ethernet HWaddr 00:07:50:CA:C3:C6
inet addr:202.144.156.230 Bcast:202.144.156.255 Mask:255.255.255.0
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:3830 errors:13 dropped:0 overruns:0 frame:13
TX packets:1779 errors:1 dropped:0 overruns:0 carrier:187
collisions:572 txqueuelen:100
RX bytes:1550803 (1.4 MiB) TX bytes:216743 (211.6 KiB)
Interrupt:3 Base address:0x100

lo Link encap:Local Loopback
inet addr:127.0.0.1 Mask:255.0.0.0
UP LOOPBACK RUNNING MTU:16436 Metric:1
RX packets:2822 errors:0 dropped:0 overruns:0 frame:0
TX packets:2822 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:0
RX bytes:281570 (274.9 KiB) TX bytes:281570 (274.9 KiB)
#

This shows two interfaces, eth0 and lo. Eth0 is the one real connection to the outside world and lo is known as the loopback interface it is a way for applications and other processes on the computer to talk to themselves in the same way applications on two different computers would. The loopback interface is convenient and necessary so you will always see it listed. Under the listing for eth0 you can see that it's address is 202.144.156.230, and its netmask is a class C. There are also some simple statistics tracking the numbers of errors and total packets sent.

So lets say you wanted to set this interface up for another network you could type a command like:

ifconfig eth0 10.10.10.130 netmask 255.255.255.0 broadcast 10.10.10.255

This would set the new address, netmask and broadcast for eth0 to 10.10.10.130, 255.255.255.0, and 10.10.10.255 respectively.
You can also check to see if an interface is physically there and has the correct driver by typing something like:
ifconfig eth0 up
If you don't get any errors then it's ready to be assigned an address etc, like the example above. You could also now type ifconfig to list all the interfaces again. If you want to temporarily turn off an interface then you can type something like this:
ifconfig eth0 down
But don't forget if you're talking to the machine over the network then you can't turn off the interface that your talking to it on. Also, if you re re-boot the machine the interface will most likely come back up if that is how the boot scripts are configured.
For the complete documentation on ifconfig you can type "man ifconfig" at a prompt.

Another handy command is route it lets you add among other things a gateway. Here is an example:
route add default gw 202.144.156.254
This sets the default gateway to 202.144.156.254. Route can add many other kinds of routes and a full discussion of routing is beyond the scope of this document. There are some other introductions to networking listed in the appendix. For a complete description of route type "man route" at a prompt.

another handy command is netstat. "netstat -rn" will list your routing table. Example:

# netstat -rn

Kernel IP routing table
Destination Gateway Genmask Flags MSS Window irtt Iface
202.144.156.0 0.0.0.0 255.255.255.0 U 40 0 0 eth0
0.0.0.0 202.144.156.254 0.0.0.0 UG 40 0 0 eth0
#


This is a very simple table with two entries. The first says that the class C network can be reached through eth0, and the second says that any other destination you should go though the gateway 202.144.156.254 which is also reachable through eth0.

3.3 Config files

Now when a computer boots up one of the last things it does is read some config files that tell it who it is and what it's address is etc... If you make some changes with ifconfig or route and reboot the computer they will all be forgotten when it reads these files and sets itself up with the old information. So to make any permanent changes you need to edit the actual config file that controls these settings. In Debian the file that sets up the network interfaces is /etc/network/interfaces. Here is an example file:

# /etc/network/interfaces -- configuration file for ifup(8), ifdown(8)
# The loopback interface
# automatically added when upgrading
auto lo
iface lo inet loopback

# The first network card - this entry was created during the Debian installation
# (network, broadcast and gateway are optional)
# automatically added when upgrading
iface eth0 inet static
address 12.33.21.201
netmask 255.255.255.0
network 12.33.21.0
broadcast 12.33.21.255
gateway 12.33.21.254

iface eth1 inet static
address 198.0.0.254
netmask 255.255.255.0
network 198.0.0.0


The paragraph for eth0 sets the address, netmask, network, broadcast, and gateway. The paragraph for eth1 lets the system calculate the network, and broadcast addresses. After you have made your changes you could type a command like "ifup eth0" to bring up a interface. To take down an interface the command is ifdown eth0, or eth1 etc... Sometimes you want to edit /etc/network/interfaces and then update the values for a particular interface. You need to be careful because most of the time you will be talking on the interface you want to change. So a command like this works pretty well:

ifdown hdlc0; ifup hdlc0
This is how you write several commands on one line, they are separated by semicolons ";".  The commands are executed one after the other when you press enter. So you connection to this machine goes down and then comes back up with the new information in the interfaces file. Often after you make a change like this you also need to update the interface on your local machine before it can talk to it's partner again. Of course a simple way to update everything on the machine is to just reboot it.

4. Maintenance

There is really not much maintenance that needs to be done on a simple router like the Flytech boxes unless you want to make changes to the network. All the routes are static right now because it is a small pilot project but you could run routd or BGP or almost any routing protocol if you felt it was necessary.

4.1 Adding new interfaces

Since these routers have five 10/100 interfaces on them you could plug in additional networks and route between them. Say for example that you wanted to install another radio in Dochula on a different subnet like 10.4.0.0 going off in another direction. Then you would find an unused interface on the Flytech box there and add config information for it in the interfaces file. Lets say that we'll use eth3. It might look something like this:

# /etc/network/interfaces -- configuration file for ifup(8), ifdown(8)
# The loopback interface
# automatically added when upgrading
auto lo
iface lo inet loopback

# The first network card - this entry was created during the Debian installation
# (network, broadcast and gateway are optional)
# automatically added when upgrading
iface eth0 inet static
address 12.33.21.201
netmask 255.255.255.0
network 12.33.21.0
broadcast 12.33.21.255
gateway 12.33.21.254

iface eth3 inet static
address 10.4.0.0.1
netmask 255.255.255.0
network 10.4.0.0
broadcast 10.4.0.255
Sometimes you add a subnet out in the field which is served by a radio repeater. The clients for that repeater need a gateway in their network so that they can talk to the rest of the system or the world. The Radios themselves don't need to use routes to figure out which way to send packets. They do something called bridging which is done only using MAC addresses (the documentation for the radios refer to these as "node" addresses). Any bridging is invisible to the IP address world. You can have any number of IP subnets served by a  group of radios and the packets will be delivered correctly. Because of this it turns out that it is convenient to assign the radios and Flytech boxes in an area with there own subnet separate from all the client subnets. But then what about the gateway for a group of clients? You can't make it their parent repeater because it already has an IP address in a different subnet, and the same goes for the router interface on the Flytech box for the same reason. What to do? ;-) ... Well you could connect another interface to a hub that had the original two cables on it and configure that interface to be the chosen gateway address for the client's subnet. But you would still need a new interface for each new subnet and since there are only five on the Flytech boxes you would quickly run out. luckily there is something called "Virtual Interfaces" to solve this problem. It's a way make one interface act like many, all with different address and even different subnets. The way you specify a virtual interface is to first find the real interface that you want to add a new address and/or subnet to. Lets say that we want to add a gateway for the subnet 10.5.1.0 to eth2. Then the new virtual interface could be called eth2:0 or eth2:1, or eth2:5. or eth2:99, it's up to you. Here is how the paragraph of the interfaces file might look:
iface eth2:5 inet static
address 10.5.1.254
netmask 255.255.255.0
network 10.5.1.0
broadcast 10.5.1.255
So we now have a new virtual interface on eth2 that acts as a gateway for the example  client's subnet. Because the radios do bridging they will soon learn of the new interface automatically and deliver packets to the new gateway as needed. Then the router will look in it's routing table to direct them to the proper destination.

By the way, each time you add an interface weather it is real or virtual, a route is automatically added to the routing table to direct packets for the new subnet to that interface.

4.2 Adding routes

Sometimes you need to let other machines know how to get to particular subnets. For example, in Thimphu there are two Flytech boxes. One talks to the Limukha area and the other talks to Gelephu. In Limukha all the client subnets are in the range 10.2.y.x, and in Gelephu they are 10.3.y.x where "y" is some small number from 0 to 3 or so. But since there are no interfaces on these Flytech boxes that are set up for these subnets there are no routes to tell them where to send packets to for all these clients. When this happens a router will use the default route (If it has one) to some other gateway. This is like saying "I have no clue where this packet goes, please deliver it for me." But in this case some of the client packets need to go to the box on the other end of the E1 line and the other clients are reached via the other Flytech box in Thimphu.

So we need to add some additional routes that could look like this:
route add -net 10.2.0.0 netmask 255.255.255.0 gw 10.1.0.2
route add -net 10.2.1.0 netmask 255.255.255.0 gw 10.1.0.2
route add -net 10.2.2.0 netmask 255.255.255.0 gw 10.1.0.2

These direct packets for the three class C subnets 10.2.0.0, 10.2.1.0, and 10.2.3.0 to the box whose IP address is 10.1.0.2 which is the router on the other end of the E1 in this case. But we still need to deliver packets for the other set of clients in Gelephu, which are handled on the other box. So we add some more routes like this:
route add -net 10.3.0.0 netmask 255.255.255.0 gw 10.0.0.254
route add -net 10.3.1.0 netmask 255.255.255.0 gw 10.0.0.254
route add -net 10.3.2.0 netmask 255.255.255.0 gw 10.0.0.254

And these routes send these subnets to the box whose IP address in 10.0.0.254. Now you may wonder how this box knows which interface to use to get to these gateways. Well remember any gateway you list must already be on a subnet that is attached to a real or virtual interface on this box, and whenever you configure an interface a route is automatically added to the table to direct packets destined for that subnet to the correct interface. If you make a mistake and the gateway is not directly reachable then you will get a "Network is unreachable" error message.

Here is a little trick to making your routing tables easer to read and maintain. notice that the Flytech boxes in Thimphu don't really need to know exactly what class Cs are being used in Limukha and Gelephu, they just need to know that all packets in the 10.2.y.x range go to Limukha and all the ones in the 10.3.y.x range go to Gelephu. We don't even have to worry about the fact that there are network addresses and broadcast addresses at the start and end of each range, all we need to do is ship all that stuff upstream to the router that really handles it. So we can replace those six routes with just two like this:
route add -net 10.2.0.0 netmask 255.255.0.0 gw 10.1.0.2
route add -net 10.3.0.0 netmask 255.255.0.0 gw 10.0.0.254
Notice that we changed the netmask to be a whole class B this includes all packets in a y.x range. Then each time you add a new client subnet on the boxes for Limukha or Gelephu you don't need to do a corresponding update in Thimphu because it is already handled. Also the fewer routes in your routing table the faster a box can move packets. Though these tables are so small anyway it doesn't matter much. :-)

5. Operation

There is not much that needs to be done to the routes for day to day operation. However there are some precautions you need to take to keep them running smoothly. First as you know most computers are not set up to handle random power failures. All routers should be on UPSs or inverters if they are powered off of the telco battery bank in a switching center. This means that you can't just turn them off by throwing the MCB with out asking them to shutdown first. If you do this it is like playing Russian roulette with your file system. Most of the time the errors caused by a loss of power can be automatically corrected... but not always. Sometimes the machine will not comeback up and require manual intervention. Usually this means typing in the root password and running fsck on the corrupted file system and typing "Y" for each major repair that it needs to do. This also usually means that some date is lost. If you are lucky it was just temporary data in a scratch file somewhere. Once in a while though you are not so lucky and a critical file is damaged. If this happens the system will no longer run even after the file system is repaired. Now you have to reload the who system from scratch. This will probably take a whole day or more depending on how far you have to drive. No phone calls can be made during this time and lots of people will be upset. So you need to educate the maintenance people not to go throwing MCBs on the Flytechs even if they think they have a good reason to. By the way, this is exactly the same situation you have with the servers at Druknet.

There are enhancements that could be made to the Flytechs to partially or completely remedy this situation. For example they could be converted to use journaling file systems. this means they would roll forward or backward any partially written updates more like a data base does. Also you could install a smaller Linux system that could all fit in RAM, the disk would just be used at boot time to initialize the RAM disk and then all updates would be done there. Of course if you make any configuration changes you also need to copy them back to the disk or they will be lost on the next boot. If you are worried that the hard drives might eventually fail you can put the system on a 128MB or 256MB Compact Flash (CF) card and boot off of that since the Flytechs do support this option.

5.1 Monitoring the E1 interface

There is a handy utility called sbectl that lets you view the state of the hdlc interface or display stats on it. type sbectl -h for help.

6. Backup

The basic idea here is to save a known working system to a CD or a hard disk on another computer and later if you need to, reformat your drive and write it back to disk. To do this you need a Debian rescue disk, but not just any Debian rescue disk. You need one specially configured for the Flytech box. This is because the Flytech boxes don't have a normal floppy drive. Instead they have a LS-120 drive which can also read floppy disks, but Linux usually expects floppies to be read from device /dev/fd0 and the LS120 is an IDE device so it has a name like /dev/hda or /dev/hdb. This causes lots of problems unless you make or have disks that are set up correctly, then you don't even notice the difference. What you should do then is make master copies of these rescue disks and two or three sets for emergency use. Then you can always make new ones from the master which is never used for anything else.

Backing up is pretty easy first we need to do a df to see how big the system is and where we might find some extra disk space.
#df
Filesystem           1k-blocks      Used Available Use% Mounted on
/dev/hdc4             29459052    485388  27477212   2% /
/dev/hdc3              9614148   5949876   3175896  66% /aux

/dev/hdc1                 7746      3375      3971  46% /boot

So what we want to back up is / (root) and /boot. /Aux is just extra storage and is not part of the OS though you might want to back it up  separately if you value it's contents. We see that the OS is about 500MB which fits nicely on a CD. Actually when compressed it may only take up about 300MB so you could fit two on one CD. To get the cleanest backups of a system it should not be in use, there are always a few files being updated like logs and temporary scratch files etc. These are fairly unimportant and you really rather not have to take the routers down to back them up anyway so I would suggest you do live backups. So it goes something like this. Lets say that /aux has enough free space so we can put the image there.
cd /aux
tar czlf flyback1.tgz / /boot
So we cd to /aux and use the tar command to make a compressed archive of all the important directories there.
Then you can ftp this file off to some other system to store or burn onto a CD. Now you may not want to copy 300MB of data each time you change an interface on the boxes. So since nearly all the config files are in /etc you could just copy that directory and back it up like this:
cd /aux
tar czf flyetc1.tgz /etc
Then after you load the latest full backup you load the latest /etc backup and your done.

In June 2002 I automated the backup system a bit. Now every week full system images are made along with MD5 checksums on each machine. Then these are copied to most of the other machines. The exception is the two remote machines don't get each others copy because I didn't want to take up too much bandwidth on the E1 links. All the backups on each machine are in /aux/backups. Older copies are also kept up to about a month. Now if you need to do a restore you can copy the appropriate image from almost any working Flytech box to your laptop. You can find the backup scripts in the appendix.

To restore It's a little more complicated because your starting from scratch. Lets assume that your disk is not physically damaged but is corrupted beyond repair. You may not need to repartition it but if you do then it's best to keep the partition numbers unchanged, ie. If root was partition one before it needs to stay partition one. See the appendix for the originals [8.3]. But basically make a small 5MB partition at the start, then slightly less than half of the disk, then a 256MB swap partition, and last, whatever is left. But it's quite likely that you can skip this step because partition tables rarely get corrupted.

Before we get started you will need a laptop with a CD drive and a network interface running a web server. This is where the backup images will come from. They will either be on a CD or the hard drive and should be findable by the web server.

OK to start, take the special LS120 rescue disk and boot off of it. You might need to check that the BIOS boot sequence is LS120,C. I think it's possible to also create a set of 1.44 MB floppies but it's less convenient. When you get to the "Debian GNU/Linux 2.2!"  screen and the boot prompt at the bottom, you can just hit enter. Then after a little while you  get to another informational screen: Then follow these steps:

1) Press enter to get to the main menu.
2) Press Enter twice to configure your keyboard for the US standard.
3) If you don't need to partition your drive again then it's safe to activate your swap partition so you would press enter
    about four times to do this and return to the menu.
4) You will probably want to initialize one or all of the Linux partitions. Select this item if it's not at the top and it will let you chose
    which partitions to initialize. You must select the one that will be the root partition first. Answer no when it asks if you want to retain             kernel 2.0 compatibility, and press enter for the rest. It will mount the first one as root. Then go on to the rest, the very small one should be     /boot and the other large one should be /aux. See the appendix for the current partition layouts of the Flytechs.
5) Now it should ask you to install kernel and modules. So select that item by pressing enter. Select IDE floppy for the installation                         medium, and then the line that says LS-120. Put the disk in the drive and press enter until it starts installing and returns you to the menu.         Note that if your floppy is write protected it will fail to mount, so you will have to make it writable and try again. This is a bug in the             Debian script.
6) Configure the device driver modules. Since all the necessary modules are already compiled into the kernel you can just press enter         here until you're back at the main menu.
7) Configure the network. It will ask you for a host name which will be over written with the backup image later so you can just press         enter for the default. Since you probably will not have a DHCP server handy you should answer no at the next question and do a             manual configuration instead. Next type in a temporary IP address. Then press enter for the default Class C netmask. Thats all you         really need, no gateway or nameservers are necessary so answer those questions as you like.
8) Install the base system. Here is where we get the full backup. Select network as the installation medium, this is the last item on the menu. It will tell you what file(s) it wants to download so press enter. Then type Control-U to erase the first line on the next page so you can type in the IP address of the laptop a "/" then the rest of the URL to the file which must be renamed "base2_2.tgz". You could also make a symlink of that name to the actual filename. For example if base2_2.tgz is in the DocumentRoot of the web server then you could use a URL like this: http://202.144.156.39/ Then hit tab until "OK" is highlighted and press enter. It should start downloading the compressed tar archive.
9) Remove the floppy if you haven't done so already
10) Select Make Linux Bootable Directly from hard disk. You now have two choices, Install Lilo in the MBR (which should work) or install Lilo in the root partitions boot sector. Hit enter for the first choice. If you get no complaints then you can reboot
11) Reboot the machine, You will notice that it continues the tail end of the Debian configuration. This is a remnant of the rescue disks install
        process. Don't bother to answer any questions, just press ALT-F2 to switch to the next virtual console and log in as root.
12) cd to / and rm the file @longlink
13) cd to /etc and restore the correct inittab file like this:
        cd /etc; mv inittab.real inittab
14) Reboot again, and your done!

Now you may want to copy some of the more recent config file images onto the system.

7. Other services

On the Gelephu box there is also an Apache web server used just to display the radio statistics. It's config files are in /etc/apache. These stats are collected by Cricket which is similar to MRTG. Cricket's config files are in /home/cricket/cricket-config and the online documentation is in /home/cricket/cricket/doc. To view the radio pages goto:

http://10.0.0.254/~cricket/

8. Appendix

8.1 Backup scripts

The stage zero scripts are all about the same just the file name is different. They rotate the images to older names, make the tar archive and the MD5 checksum. They are run by  cron every Sunday at 4:00 AM. Here is the one from bt1:
#/bin/sh
# Stage zero of the Flytech backup image copying script
# Clif Cox June 2002

cd /aux/backups
mv -f bt1arc.1.tgz bt1arc.2.tgz; mv -f bt1arc.1.sum bt1arc.2.sum
mv -f bt1arc.0.tgz bt1arc.1.tgz; mv -f bt1arc.0.sum bt1arc.1.sum
mv -f bt1arc.tgz bt1arc.0.tgz; mv -f bt1arc.sum bt1arc.0.sum

mv -f bt2arc.1.tgz bt2arc.2.tgz; mv -f bt2arc.1.sum bt2arc.2.sum
mv -f bt2arc.0.tgz bt2arc.1.tgz; mv -f bt2arc.0.sum bt2arc.1.sum
mv -f bt2arc.tgz bt2arc.0.tgz; mv -f bt2arc.sum bt2arc.0.sum

mv -f bt3arc.1.tgz bt3arc.2.tgz; mv -f bt3arc.1.sum bt3arc.2.sum
mv -f bt3arc.0.tgz bt3arc.1.tgz; mv -f bt3arc.0.sum bt3arc.1.sum
mv -f bt3arc.tgz bt3arc.0.tgz; mv -f bt3arc.sum bt3arc.0.sum

mv -f bt4arc.1.tgz bt4arc.2.tgz; mv -f bt4arc.1.sum bt4arc.2.sum
mv -f bt4arc.0.tgz bt4arc.1.tgz; mv -f bt4arc.0.sum bt4arc.1.sum
mv -f bt4arc.tgz bt4arc.0.tgz; mv -f bt4arc.sum bt4arc.0.sum

tar czlf bt1arc.tgz / /boot
md5sum bt1arc.tgz > bt1arc.sum

And here is the one from bt2:
#/bin/sh
# Stage zero of the Flytech backup image copying script
# Clif Cox June 2002

cd /aux/backups
mv -f bt1arc.1.tgz bt1arc.2.tgz; mv -f bt1arc.1.sum bt1arc.2.sum
mv -f bt1arc.0.tgz bt1arc.1.tgz; mv -f bt1arc.0.sum bt1arc.1.sum
mv -f bt1arc.tgz bt1arc.0.tgz; mv -f bt1arc.sum bt1arc.0.sum

mv -f bt2arc.1.tgz bt2arc.2.tgz; mv -f bt2arc.1.sum bt2arc.2.sum
mv -f bt2arc.0.tgz bt2arc.1.tgz; mv -f bt2arc.0.sum bt2arc.1.sum
mv -f bt2arc.tgz bt2arc.0.tgz; mv -f bt2arc.sum bt2arc.0.sum

mv -f bt3arc.1.tgz bt3arc.2.tgz; mv -f bt3arc.1.sum bt3arc.2.sum
mv -f bt3arc.0.tgz bt3arc.1.tgz; mv -f bt3arc.0.sum bt3arc.1.sum
mv -f bt3arc.tgz bt3arc.0.tgz; mv -f bt3arc.sum bt3arc.0.sum

mv -f bt4arc.1.tgz bt4arc.2.tgz; mv -f bt4arc.1.sum bt4arc.2.sum
mv -f bt4arc.0.tgz bt4arc.1.tgz; mv -f bt4arc.0.sum bt4arc.1.sum
mv -f bt4arc.tgz bt4arc.0.tgz; mv -f bt4arc.sum bt4arc.0.sum

tar czlf bt2arc.tgz / /boot
md5sum bt2arc.tgz > bt2arc.sum


The stage one scripts are run about 15 minutes later. They use the ssh copy program "scp" to transfer the files from neighboring machines. We run the stage one and two scripts as user flyback instead of root because scp needs to login to another machine without using a password. It does this by using a cryptographically strong key that is shared by all machines. This is probably secure enough for a non-privileged user. But if we had used root instead then anyone who accidently found a root shell on one machine could get root on any other without typing the password, probably not what we want...  Here is the stage one script from bt3:
#/bin/sh
# Stage one of the Flytech backup image copying script
# Clif Cox June 2002

cd /aux/backups
scp -pBq "bt4:/aux/backups/bt4arc.*" .
scp -pBq "bt1:/aux/backups/bt1arc.*" .


The stage two script is run about 45 minutes after stage one. It copies a image from its neighbor that was transfered during stage one. Stage two script from bt3:
#/bin/sh
# Stage two of the Flytech backup image copying script
# Clif Cox June 2002

cd /aux/backups
scp -pBq "bt1:/aux/backups/bt2arc.*" .


8.2 Making a rescue floppy

As was mentioned in section 6 the Standard issue Debian rescue floppies have to be modified a bit to work with the Flytech boxes. I describe how to make new ones from scratch here, and yes it is a hassle. You relay don't want to do this if you can avoid it. So please take good care of your master rescue disks and only use copies of them to reload your machines if you need to. If something happens to your master disks you can get a copy of the image [4] Which can be copied to an LS-120 disk like so:

1) Uncompress the image
        bunzip2 flyrescue.bin.bz2
2) Put an LS-120 disk in the drive, we'll assume that it is device /dev/hdb
3) Copy the image to the LS-120
        dd if=flyrescue.bin of=/dev/hdb bs=1M
4) Reboot the machine to test the new rescue floppy

That said, this procedure involves getting some images from the www.debian.org site and building a 2.2.19 version Linux kernel from the source. Some of the steps involved are outside the scope of this document. Try to enlist the help of a  Linux person.

So the files we will need from the debian site are: the 2.9MB rescue.bin [7], or a tar file of its contents [8],  and drivers.tgz [9].
From www.kernel.org you need the source to Linux kernel version 2.2.19. The extra utilities that you will need are mkdosfs to format an LS-120, and syslinux to install the boot block on the new rescue disk. Of course you will also need some LS-120 floppy disks which are probably available by mail order or from Bangkok.

First we need to build a new kernel, so un-tar the source in a convenient location and configure it with at least these drivers linked in:
initrd, ramdisk, loop, msdos, fat, elf, ext2fs, procfs, IDE disk support, IDE floppy support, and in the Ethernet 10/100 area:
CONFIG_EEXPRESS_PRO, CONFIG_DEC_ELCP, CONFIG_EEXPRESS_PRO100.

These ethernet drivers should cover the possible chip sets on the Flytech mother boards but I haven't been able to test all of them. If the ethernet port doesn't come up then you might try loading different drivers from that menu item on the rescue disk. For the standard 2.2.19 modules that come with the debian rescue disk to load correctly it's important to compile your kernel with the "Set version information on all symbols for modules" turned on. Of course there are many other kernel options that must be set correctly but the defaults should work fine. It might be good to start with the kernel config file I used [6]. A little more documentation on this part is in the readme.txt file on the rescue disk itself.

If you are using LS-120 disks:

1) Put an LS-120 disk in the Flytechs floppy drive.
2) Determine the device name of the LS-120 drive. You could use the following command:
        find /proc/ide -type f -exec egrep -l "LS.?120" {} \;
        You should get one line of output that has an hda, hdb, hdc, etc in it. This is the device name.
3) From a root shell format it using the mkdosfs command. Here we are assuming that the LS-120 is device hdb, don't forget the /dev/ :-)
        mkdosfs -v -c -I -n Flytech_Rescue /dev/hdb
        The "-c" option checks for bad blocks and takes a long time for a LS-120 disk but it's probably a good idea to double check your rescue             disks integrity.
5) mount the new floppy with the command:
        mount -t msdos /dev/hdb /floppy
6) Download or find the 2.9MB rescue floppy image on the debian site or get a copy here [7]. Alternatively get the tar file of its contents [8] .
7) If you have the rescue.bin image then mount it using a loop mount like this:
        mount -t msdos rescue.bin /mnt/ -o ro,loop
8) Cd into your new floppy disk
        cd /floppy
9) Copy the files from the rescue image here like this:
        cp -r /mnt/* .
10) Or if you have the tar file instead then do this:
        tar xzf /some/path/to/rescue-2.9.tgz
11) Now copy your new kernel to the floppy:
        cp /some/path/to/your/linux/arch/i386/boot/bzImage linux
12) Then we need to run the rdev.sh command like it says in readme.txt. If you don't do this the console will lock up with some obscure error             about an insmod unix.o failing over and over again.
        ./rdev.sh
13) OK now we make a place for the kernel image and drivers to go
        mkdir images-1.44
14) And copy these files there:
        cp drivers.tgz rescue.bin images-1.44
        Yes it seems funny but you need a copy of rescue.bin inside of rescue.bin! There is probably a more elegant way to do this but there is                 plenty of space on the LS-120.
15) Now we just unmount the floppy and run syslinux, and we can reboot to test it.
        cd; umount /floppy
        syslinux /dev/hdb

By the way there are some references to /dev/fd0 (the usual floppy drive) in the config file syslinux.cfg These won't be used for the restore procedure that we outline in section 6 but if you'd like to use these for more general purposes then you should change all the /dev/fd0 and /dev/fd1 to /dev/hdb etc...

8.3 Flytech Partition table info

If in the unlikely event that the partition table of a Flytech box becomes corrupted then you will find this section helpful in recreating it. Note that the Backups of the file systems expect the root partition to have a particular number and so on. So it is important that when you recreate it that the numbering of the partitions is exactly the same as listed here, though the sizes could be a little different if you like. By the way as of June 23rd BT1 became a spare router and now has a copy of bt3's image on it.

BT1 partition table:

Disk /dev/hda: 255 heads, 63 sectors, 5005 cylinders
Units = cylinders of 16065 * 512 bytes

Device Boot Start End Blocks Id System
/dev/hda1 * 1 1 8001 83 Linux
/dev/hda2 2 1217 9767520 83 Linux
/dev/hda3 1218 1279 498015 82 Linux swap
/dev/hda4 1280 5005 29929095 5 Extended

/dev/hda5 1280 5005 29929063+ 83 Linux

BT2 partition table:

Disk /dev/hda: 255 heads, 63 sectors, 2491 cylinders
Units = cylinders of 16065 * 512 bytes

Device Boot Start End Blocks Id System
/dev/hda1 1 1 8001 83 Linux
/dev/hda2 2 1225 9831780 83 Linux
/dev/hda3 1226 1258 265072+ 82 Linux swap
/dev/hda4 1259 2491 9904072+ 83 Linux

BT3 partition table:

Disk /dev/hda: 255 heads, 63 sectors, 5005 cylinders
Units = cylinders of 16065 * 512 bytes

Device Boot Start End Blocks Id System
/dev/hda1 * 1 1 8001 83 Linux
/dev/hda2 2 1217 9767520 83 Linux
/dev/hda3 1218 1279 498015 82 Linux swap
/dev/hda4 1280 5005 29929095 5 Extended
/dev/hda5 1280 5005 29929063+ 83 Linux

BT4 partition table:

Disk /dev/hda: 255 heads, 63 sectors, 5005 cylinders
Units = cylinders of 16065 * 512 bytes

Device Boot Start End Blocks Id System
/dev/hda1 * 1 1 8001 83 Linux
/dev/hda2 2 1217 9767520 83 Linux
/dev/hda3 1218 1279 498015 82 Linux swap
/dev/hda4 1280 5005 29929095 5 Extended
/dev/hda5 1280 5005 29929063+ 83 Linux

9. References

[1] Flytech Group International: http://www.flytech.com/
[2] Linux Headquarters: Network Configuration Using the Command Line: http://www.linuxheadquarters.com/howto/networking/networkconfig.shtml
[3] Introduction to Networking: http://www.thelinuxreview.com/howto/intro_to_networking/book1.htm

10.1 Supporting files for building rescue disks

[4] The compressed rescue floppy image: http://www.bhutan-notes.com/clif/flyrescue/flyrescue.bin.bz2
[5] The a Linux kernel with the IDE floppy driver built in: http://www.bhutan-notes.com/clif/flyrescue/linux
[6] The config file for the above kernel: http://www.bhutan-notes.com/clif/flyrescue/kernel-config.txt
[7] The Original Debian rescue image: http://www.bhutan-notes.com/clif/flyrescue/rescue.bin
[8] A tar file of the contents of the rescue image: http://www.bhutan-notes.com/clif/flyrescue/rescue-2.9.tgz
[9] Additional drivers that are probably not needed: http://www.bhutan-notes.com/clif/flyrescue/drivers.tgz