Paul Boddie's Free Software-related blog
Paul's activities and perspectives around Free Software
Using ext2 Filesystems with L4Re
Previously, I described my initial investigations into libext2fs and the development of programs to access and populate ext2/3/4 filesystems. With a program written and now successfully using libext2fs in my normal GNU/Linux environment, the next step appeared to be the task of getting this library to work within the L4Re system. The following steps were envisaged:
- Figuring out the code that would be needed, this hopefully being supportable within L4Re.
- Introducing the software as a package within L4Re.
- Discovering the configuration required to build the code for L4Re.
- Actually generating a library file.
- Testing the library with a program.
This process is not properly completed in that I do not yet have a good way of integrating with the L4Re configuration and using its details to configure the libext2fs code. I felt somewhat lazy with regard to reconciling the use of autotools with the rather different approach taken to build L4Re, which is somewhat reminiscent of things like Buildroot and OpenWrt in certain respects.
So, instead, I built the Debian package from source in my normal environment, grabbed the config.h file that was produced, and proceeded to use it with a vastly simplified Makefile arrangement, also in my normal environment, until I was comfortable with building the library. Indeed, this exercise of simplified building also let me consider which portions of the libext2fs distribution would really be needed for my purposes. I did not really fancy having to struggle to build files that would ultimately be superfluous.
Still, as I noted, this work isn’t finished. However, it is useful to document what I have done so far so that I can subsequently describe other, more definitive, work.
Making a Package
With a library that seemed to work with the archiving program, written to populate filesystems for eventual deployment, I then set about formulating this simplified library distribution as a package within L4Re. This involves a few things:
- Structuring the files so that the build system may process them.
- Persuading the build system to install things in places for other packages to find.
- Formulating the appropriate definitions to build the source files (and thus producing the right compiler and linker invocations).
The Package Structure
Currently, I have the following arrangement inside the pkg/libext2fs directory:
include include/libblkid include/libe2p include/libet include/libext2fs include/libsupport include/libuuid lib lib/libblkid lib/libe2p lib/libet lib/libext2fs lib/libsupport lib/libuuid
To follow L4Re conventions, public header files have been moved into the include hierarchy. This breaks assumptions in the code, with header files being referenced without a prefix (like “ext2fs”, “et”, “e2p”, and so on) in some places, but being referenced with such a prefix in others. The original build system for the code gets away with this by using the “ext2fs” and other prefixes as the directory names containing the code for the different libraries. It then indicates the parent “lib” directory of these directories as the place to start looking for headers.
But I thought it worthwhile to try and map out the header usage and distinguish between public and private headers. At the very least, it helps me to establish the relationships between the different components involved. And I may end up splitting the different components into their own packages, requiring some formalisation of their interactions.
Meanwhile, I defined a Control file to indicate what the package provides:
provides: libblkid libe2p libet libext2fs libsupport libuuid
This appears to be used in dependency resolution, causing the package to be built if another package requires one of the named entities in its own Control file.
Header File Locations
In each include subdirectory (such as include/libext2fs) is a Makefile indicating a couple of things, the following being used for libext2fs:
PKGNAME = libext2fs CONTRIB_HEADERS = 1
The effect of this is to install the headers into a include/contrib/libext2fs directory in the build output.
In the corresponding lib subdirectory (which is lib/libext2fs), the following seems to be needed:
CONTRIB_INCDIR = libext2fs
Hopefully, with this, other packages can depend on libext2fs and have the headers made available to it by an include statement like this:
#include <ext2fs/ext2fs.h>
(The ext2fs prefix is provided by a directory inside include/libext2fs.)
Otherwise, headers may end up being put in a special “l4” hierarchy, and then code would need changing to look something like this:
#include <l4/ext2fs/ext2fs.h>
So, avoiding this and having the original naming seems to be the benefit of the “contrib” settings, as far as I can tell.
Defining Build Files
The Makefile in each specific lib subdirectory employs the usual L4Re build system definitions:
TARGET = libext2fs.a libext2fs.so PC_FILENAME = libext2fs
The latter of these is used to identify the build products so that the appropriate compiler and linker options can be retrieved by the build system when this library is required by another. Here, PC is short for “package config” but the notion of “package” is different from that otherwise used in this article: it just refers to the specific library being built in this case.
An important aspect related to “package config” involves the requirements or dependencies of this library. These are specified as follows for libext2fs:
REQUIRES_LIBS = libet libe2p
We saw these things in the Control file. By indicating these other libraries, the compiler and linker options to find and use these other libraries will be brought in when something else requires libext2fs. This should help to prevent build failures caused by missing headers or libraries, and it should also permit more concise declarations of requirements by allowing those declarations to omit libet and libe2p in this case.
Meanwhile, the actual source files are listed using a SRC_C definition, and the PRIVATE_INCDIR definition lists the different paths to be used to search for header files within this package. Moving the header files around complicates this latter definition substantially.
There are other complications with libext2fs, notably the building of a tool that generates a file to be used when building the library itself. I will try and return to this matter at some point and figure out a way of doing this within the build system. Such generation of binaries for use in build processes can be problematic, particularly if there is some kind of assumption that the build system is the same as the target system, but such assumptions are probably not being made here.
Building the Library
Fortunately, the build system mostly takes care of everything else, and a command like this should see the package being built and libraries produced:
make O=mybuild S=pkg/libext2fs
The “S” option is a real time saver, and I wish I had made more use of it before. Use of the “V” option can be helpful in debugging command options, since the normal output is abridged:
make O=mybuild S=pkg/libext2fs V=1
I will admit that since certain header files are not provided by L4Re, a degree of editing of the config.h file was required. Things like HAVE_LINUX_FD_H, indicating the availability of Linux-specific headers, needed to be removed.
Testing the Library
An appropriate program for testing the library is really not much different from one used in a GNU/Linux environment. Indeed, I just took some code from my existing program that lists a directory inside a filesystem image. Since L4Re should provide enough of a POSIX-like environment to support such unambitious programs, practically no changes were needed and no special header files were included.
A suitable Makefile is needed, of course, but the examples package in L4Re provides plenty of guidance. The most important part is this, however:
REQUIRES_LIBS = libext2fs
A Control file requiring libext2fs is actually not necessary for an example in the examples hierarchy, it would seem, but such a file would otherwise be advisible. The above library requirements pull in the necessary compiler and linker flags from the “package config” universe. (It also means that the libext2fs headers are augmented by the libe2p and libet headers, as defined in the required libraries for libext2fs itself.)
As always, deploying requires a suitable configuration description and a list of modules to be deployed. The former looks like this:
local L4 = require("L4");
local l = L4.default_loader;
l:startv({
log = { "ext2fstest", "g" },
},
"rom/ex_ext2fstest", "rom/ext2fstest.fs", "/");
The interesting part is right at the end: a program called ex_ext2fstest is run with two arguments: the name of a file containing a filesystem image, and the directory inside that image that we want the program to show us. Here, we will be using the built-in “rom” filesystem in L4Re to serve up the data that we will be decoding with libext2fs in the program. In effect, we use one filesystem to bootstrap access to another!
Since the “rom” filesystem is merely a way of exposing modules as files, the filesystem image therefore needs to be made available as a module in the module list provided in the conf/modules.list file, the appropriate section starting off like this:
entry ext2fstest roottask moe rom/ext2fstest.cfg module ext2fstest.cfg module ext2fstest.fs module l4re module ned module ex_ext2fstest # plus lots of library modules
All these experiments are being conducted with L4Re running on the UX configuration of Fiasco.OC, meaning that the system runs on top of GNU/Linux: a sort of “user mode L4”. Running the set of modules for the above test is a matter of running something like this:
make O=mybuild ux E=ext2fstest
This produces a lot of output and then some “logged” output for the test program:
ext2fste| Opened rom/ext2fstest.fs. ext2fste| / ext2fste| drwxr-xr-x- 0 0 1024 . ext2fste| drwxr-xr-x- 0 0 1024 .. ext2fste| drwx------- 0 0 12288 lost+found ext2fste| -rw-r--r--- 1000 1000 11449 e2access.c ext2fste| -rw-r--r--- 1000 1000 1768 file.c ext2fste| -rw-r--r--- 1000 1000 1221 format.c ext2fste| -rw-r--r--- 1000 1000 6504 image.c ext2fste| -rw-r--r--- 1000 1000 1510 path.c
It really isn’t much to look at, but this indicates that we have managed to access an ext2 filesystem within L4Re using a program that calls the libext2fs library functions. If nothing else, the possibility of porting a library to L4Re and using it has been demonstrated.
But we want to do more than that, of course. The next step is to provide access to an ext2 filesystem via a general interface that hides the specific nature of the filesystem, one that separates the work into a different program from those wanting to access files. To do so involves integrating this effort into my existing filesystem framework, then attempting to re-use a generic file-accessing program to obtain its data from ext2-resident files. Such activities will probably form the basis of the next article on this topic.