The recent introduction of Linux into the embedded sector has been one of the most exciting changes in the last few years. Based on the open-source model, it offers new possibilities to embedded engineers traditionally used to commercial operating systems. At our university, EUSS, we train students to develop embedded products. Until last year, our lectures and lab sessions were based on a commercial real-time operating system. Now we use Linux, and this fact has opened our minds and given us possibilities for a future evolution.

In this article we start with a description of our teaching and the objectives that triggered this change. We continue with the subject curricula and the embedded distribution created for the lab sessions. Finally, conclusions from the first year are presented.

In 1995 we were assigned to teach a course titled "Industrial Computing". The course is taught every year and is divided in two halves. In the first part, we work with 8-bit systems, and we study 32-bit systems in the second half. For the 32-bit section, a proprietary operating system VxWorks was initially selected. The reason for that decision was that Wind River Systems admitted our school into its university program. This basically implies zero software cost, and they offered us a complete solution for our hardware: the evaluation board that Intel had delivered for the i386EX (EV386EX).

We have to admit that the start-up of the software was not very difficult, and in a reasonable amount of time, the system was available in our lab (14 seats). Problem arose when we discovered that the development environment for Windows NT was not very stable. In addition, in order to use new versions, we had to subscribe to their maintenance service, which was not included in the university program. The maintenance plan, however, was not affordable for us. Consequently, we started to search for alternative solutions.

Then, two years ago, we landed in the Linux world. Since that time, we have followed a rigorous self-training program that has allowed us to develop a tuned embedded distribution for our hardware.

The change from the proprietary operating system to Linux has had the most main implications in the lab sessions, where an embedded Linux distribution needed to be generated for our target machines. In designing the course, we have taken into account the students' backgrounds, which generally is limited to courses in basic programming, digital systems and 8-bit microcontroller assembler programming. The objectives of our course are: 1) basic skills in embedded systems programming with operating systems; 2) knowledge of the main software aspects of an embedded system; 3) ability to development an environment based on a host and a target; 4) familiarity with the Linux API; and 5) comfortability with desktop Linux systems. The class schedule is one theory hour and two lab hours a week. The lab sessions align with the week's theory lesson.

Overall, the topic has been divided into four sections. The first section introduces the Linux operating system and desktop Linux systems. This often is a very new environment for the students, because they have used only Windows systems until this point. Therefore, practical shell exercises are stressed. The second section covers the bulk of the Linux API. It is devoted to process and thread creation, interprocess communication (mainly pipes) and mutual exclusion problems, where the mutexes are introduced. Tutorial examples are given to students in the lab sessions; they have to analyze a solved problem and make some improvements. The learning process here is very guided.

In the third section, the input/output system is introduced. The students design an application that controls the lab target (see the next session). Here, students follow the typical design process of analysis, implementation and debugging. Finally, the fourth section covers communication by sockets through TCP/IP, and students have to add a server thread that accepts remote connections to telecontrol the system. The client part is initially implemented as a line-command application. The students end the course by implementing this client in a CGI program executed from a web page served by the web server introduced in their own embedded Linux distribution.

The target used in the lab sessions emulates an automatic quality system. It is based on a conveyor belt that transports hypothetical pieces. Every piece passes through an optical barrier that takes information about its size; in fact, the optical barrier measures the time that a piece blacks it. If we know the velocity measurement, we can obtain its size. If the piece size is judged unacceptable, it can be thrown out by a solenoid (an electro-mechanical device). Additionally, a LCD and a keyboard with user interfaces are used.

The embedded system is designed for the evaluation board that Intel delivered for its i386EX (EV386EX). This board includes the microprocessor at 50MHz, one SIMM connector that supports up to 16MB of DRAM, 0.5 MB of Flash (loaded through a DOS application via serial port), two serial ports and a PC/104. In the PC/104 connector two boards are connected. The first one is a compatible NE2000 Ethernet board and the second one is a customized I/O board that was developed in 1995, at the time of the original system development. This last customized board includes all the logical control for the target devices. Technologically, this board is based on an EPLD by Altera. Pentium 200MHz machines with 32MB of RAM are used as host systems to both develop software and monitor the targets.

The obvious memory constraints force us to use the Etherboot application to boot the system from the Net. Although it is possible to fit a kernel in the Flash, we opted to load only the software that will not change in the near future. The update process is time consuming; we have to put every one of the 14 boards in update mode with a special jumper and download the new software image with the DOS application.

We started this process by downloading the Etherboot source codes from the home page and compile them for the Ethernet board. The outcome was the binary image that we had to load in the boot EPROM of the Ethernet board. At that point, a big problem appeared: the EV386EX was delivered without a BIOS and the binary image file generated was really a BIOS extension (a mechanism that allows code to be added to the basic BIOS core). Looking at the software, we found that it came with five BIOS demos with limited capabilities. These BIOS demos can be downloaded and tested with a demo embedded DOS version, too. We tested whether any one of the delivered BIOS demos supported the mentioned mechanism; only one was able to recognize and execute the Etherboot image.

In order to trace the booting process, we had to redirect the standard output to a serial port. Fortunately the Etherboot system incorporates this option--this redirection is needed in the majority of embedded Linux distributions dedicated to systems without screen. The BIOS comes with this type of redirection too.

One of the most curious questions of the PC architecture is how it resolves the A20 problem, which is a compatibility issue found in the emulation of the real mode when the microprocessor boots. In the real-mode segmented addressing, the largest possible segment_offset (0FFFFH:0FFFFH) results in a physical address of 10FFEFH. In the 8086, the most significant bit (bit 20) is lost because the processor only has address line support for A0-A19, obtaining a direction towards 0FFEFH. As a result, a device is implemented. This device is connected to port 92, which grounds the A20 and emulates the 8086. The 386EX has a special hardware, and the #define IBM_L40 line in the source code must be uncommented in order to activate it.

In the netboot application two options are available to mount the root filesystem: NFS or initial RAM disk. We decided to use the last one. The target has 16MB of memory, which is enough to contain the root filesystem and the kernel. The mknbi tool was used to build the tagged image, but in order to make it work correctly, the #define IBM_L40 line had to be uncommented in the kernel sources. Moreover, it was necessary to force the mknbi tool to assume that the memory is 16MB. This is essential because we use a limited BIOS that does not support the 15 BIOS interrupt, used to obtain the memory RAM.

At this point it is important to realise that all these conclusions were reached after hours of tracing the booting process via a serial line. The fact that the system is not 100% PC-compatible was the biggest handicap. The Linux kernel was built from the source for version 2.2.19, and a minimal configuration was used, redirection of the console to a serial port, change of the already mentioned BIOS int15 and support for properly sized initial RAM disk. As far as the BIOS int15 is concerned, we had to force its references into the small amount of memory.

A suitable image of the RAM disk was built by using a loopback device. We used Busybox and Tinylogin to get a minimal system. We completed it with the little Boa web server, the bash shell (we decided to use the bash instead of one of the Busybox shells to have a full functional shell) and a stripped version of the GDB to have debugging capabilities. All these elements were put into a 7MB RAM disk. The console device is a symlink to ttyS0, and the inittab generates the consoles to serial channels ttyS0 and ttyS1. The initial scripts were reduced to minimal expression.

The lab target has some devices directly accessible with an I/O map. A kernel module was developed to contain all the needed drivers for devices accessed from user space. They include drivers for the keyboard, display, solenoid, encoder (to acquire the conveyor speed), optical barrier (to measure the piece size) and motor actuator (to control the motor power). The implementation is very simple and is based on access to the I/O map with the encoder and the optical barrier. A blocked read is also done with a wait queue, which is unblocked with an interrupt handler.

In order to allow development, a lab target simulator was developed. The simulator emulates the hardware behavior and uses a new kernel module to communicate with user space. From the students' point of view, it doesn't matter whether the application is executed on the lab target or on the host; the correct module routes the access to either the hardware or the simulator. The visualization part of the simulator is implemented with OpenGL, and the core uses C++.

Finally, the host system was based on limited hardware: Pentium 200MHz machines with 32MB RAM. This fact forced us to base the host on a Red Hat 6.2 standard installation with limited services and a Blackbox window manager instead of KDE or GNOME. With this configuration we can open simultaneously the KDevelop IDE (we use the 1.3 version, the most recent usable one with Red Hat 6.2) and DDD debugger.

At the time of this writing, we are about the finish the first Linux course at EUSS. The general conclusion is that the change has been a positive one. Students have received it with enthusiasm, surely motivated by the increase presence of Linux in the news and the proliferation of Linux magazines. This fact gives them the impression they are learning a technology important for their professional success. The students' progress has been very satisfactory, and the learning objectives seem to have been met. Regarding the embedded Linux creation for the lab target, it has been stable enough to support the lab free access, which submits the system to a higher stress load than anybody could have imagined. We are now planning to migrate all the system to the 2.4 kernel and update the root filesystem components. As far as the host is concerned, next year the machines will be updated and the system surely will be based on a newer version of Red Hat.

Dr. Andreu Moreno has been a lecturer and an embedded systems developer for eight years. He landed in the Linux world two years ago. He now combines teaching at EUSS with research in the fault-tolerant field and in embedded systems development for the industrial sector.

Agusté Fontquerni has been a lecturer for five years and has worked in the industrial sector. He is interested in fields of microprocessors, Internet protocols, network administration and embedded systems. He teaches at EUSS and develops software for embedded Linux systems.