Running MySQL with <16MB memory in an ARM processor

In many small embedded systems it doesn’t make much sense to have a powerful Relational Data Base Management System (RDBMS) such as MySQL because there is only one application that access the database or one that writes and one/many that reads. In these cases, the best option is SQLite that is may be the widest used embedded database.

However, small systems are getting more complex and even when there are only few MB of memory and few MHz (in my case 64MB and 210MHz respectively), there could be several applications accessing the database simultaneously. For example, several C programs writting and several PHP script reading and writting (through a built-in web server such as Busybox’s httpd or lighttpd).
In these cases SQLite starts to have issues handling the concurrence even if you configure it for using Write-Ahead-Log (WAL) instead of the default rollback journal.

I was curious about the minimum requirements I would need to run MySQL in an small embedded system just to see if it was technically feasible. I gave it a try and I managed to run it in an ARM Atmel AT91SAM9260 with only 64MB.
In fact, MySQL was running exactly in 13.58MB:

htop_mysql

The first thing was obviously building MySQL for my ARM processor. I didn’t find any easy method, but this procedure worked for me. It’s based on this guide written by nonoo.

The most imporant thing when cross-building MySQL is that you have to build first a native MySQL and then cross-build MySQL for your target. This is necessary because MySQL, for building some pieces of itself, it uses binaries generated on-the-fly.

First, let’s export common environment variables that make our life easier:

Requirements

libncurses
Package: libncurses 5.7

Configure, build and install the only requirement:

MySQL
Package: MySQL 5.5.37

MySQL

Download MySQL and copy it these two directories:

Compile the host version. In my case I have an AMD 64 bits processor. Choose the one that fits your host:

Now we can start to cross-build MySQL:

Configuring

Create the CMake toolchain file mysql-src-arm/build/mysql_at91sam9260.cmake:

Create and run the script mysql-src-arm/BUILD/doconfig:

Prepare to build

Create the script mysql-src-arm/BUILD/domake:

The above script copies a couple of files before building. THis is needed everytime because MySQL will try to run the auto-generated binaries for ARM, but we need to execute them in our host (x86). Not doing this gives these errors:

Cross-building

Copy the following ncurses include files generated when building ncurses inside the client/ directory, otherwise the building process won’t find them:

and edit the file mysql.cc by modifying how the above files are included:

Run domake. If you’re lucky it will finish successfully at the first try.

If you have the following error

copy all executables from mysql-src-x86/BUILD/extra to mysql-src-arm/BUILD/extra and mysql-src-arm/extra.
Run domake again.

If you still have problems and get this error:

Copy mysql-src-x86/BUILD/scripts/comp_sql to mysql-src-arm/scripts/comp_sql.
Run domake again.

At this point you should have built it successfully.

Installing

It will install in the usr/local/ directory inside the $ROOTFS.

Add the mysql user and group in your target board. If you have more users than root, you have to adapt it by changing the user and group id:

Edit /etc/passwd

Edit /etc/group

and create the required directory

Now you can run MySQL’s install script:

Now that you have the MySQL server binary in your rootfs you can execute it, but if you have less than 128MB, it will refuse to start.

You need to configure it properly (my.conf) to use as less memory as possible. I made several tests and the best configuration in which I managed to make it work is the following:

Some of the parameters have hard-coded minimums and MySQL will refuse to start if you set a smaller number or simply ignore it and start with it’s default.

Notice that some of these options reduced significantly its functionality. For example, I was not interested in a TCP/IP server, everything runs inside the embedded device, so I set the skip-networking option.

Play around with the different options to fit your needs.

With this configuration I was able to run MySQL and to have several clients connected writing and reading simultaneously without any memory issues. Even the processor behave nicely when there were only few simultaneous writes and reads.

Have fun!

Installing the wireless USB adapter Edimax EW-7811Un in an embedded device

Recently, I needed to install and configure a Wireless 802.11/b/g/n USB Adapter in an ARM-based device that runs a 2.6.35 Linux. The adapter was an Edimax EW-7811Un.

There are several wireless USB adapters that run under Linux, you can have a look at the official site here for supported devices and their chipsets. However, this list is far from complete and out-dated.
Of course, the chipsets listed in this site are the ones inside the mainline kernel, but several vendors provide drivers that for one reason or another are not part of the mainline kernel and we have to compile them out-of-tree.

This was my case because I’m using an old 2.6.35 kernel and the 8192cu.ko driver was not part of the mainline kernel. It was included in the 2.6.39 under the name rtl8192cu.ko.

The Makefile provided is for compiling for an i386 based processor, so I created this small patch for compiling for an AT91 ARM processor. You could do the same by just changing the CONFIG_PLATFORM_ARM_AT91 by the one that defines your processor.

As you can see from the above patch, I’m using the kernel provided by buildroot as the sources for building the driver, but you can change the KSRC variable as you need.

Now you can build the driver with the ARCH and CROSS_COMPILE environment variables according to your platform and toolchain. In my case, I have an ARM processor (an AT91SAM9 from STMicroelectronics).

For testing quickly the driver in your embedded device you can copy it to the rootfs and insmod’it:

I’ve been using this driver for several months and it is quite stable using different WPA encryption methods with b/g wireless networks.

That’s it!

Building GNU libmatheval for ARM

Overview

GNU libmatheval is a library used for parsing and evaluating symbolic expressions input as text.

A simple and common use of this library is when you have a C program and you have mathematical expressions. For example,

x * y > x^2

contains two variables whose values can vary over time since they could be analog inputs and therefore you need to evaluate it often and execute different actions according to the resulting comparison.

This library can be quite useful in an embedded system that provides some control functionality, but it has the disadvantage that some of its requirements are not straight forward to cross-build and have significant memory and storage footprints.

I managed to build it and run it successfully on an Atmel AT91SAM9260 at 210MHz with 64MB RAM and a rootfs with 128MB NAND flash, so it should be sufficient for many embedded systems, but if you have a slower processor or less memory (it’s still common to find industrial embedded devices with significantly fewer processor horsepower and/or memory) you could have performance issues.
For this article, I’m using the more powerful Atmel Cortex-A5 ATSAMA5D35 with 512MB RAM.

Requirements

I used buildroot for building the toolchain and the rootfs for the evaluation board. Unfortunately, buildroot does not include libmatheval nor some of it’s requirements since they are not typically used in embedded systems.

Building a toolchain and a rootfs with buildroot is a straight-forward task that is well documented. By using it, you can easily build some of libmatheval’s requirements:

  • libtool
  • libgmp
  • libffi
  • pthreads
  • flex

The procedure that I’m describing here is for building libmatheval and the requirements that are not included in buildroot.

First, let’s export common environment variables that make our life easier:

Now we can start building the requirements:

libuninstring
Package: libunistring 0.9.3

This library is used for advanced text processing. The mathematical expressions we use need advanced text analysis.

Apply the following patch. The commented code should not be a problem since it’s for glibc < 2.4 and most embedded systems use uclibc or newlib and a newer GCC:

Build and install in $ROOTFS

bdw-gc
Package: gc-7.2e

This is a C garbage collector for C malloc or C++ new. As with libunistring, this library is needed for the following requirement (Guile).

Configure, build and install in $ROOTFS:

guile
Package: guile 1.8.8

Guile is a library, interpreter and compiler of Scheme, a symbolic programming language. It’s an ideal option for evaluating mathematical expressions, but it’s not the best companion for small embedded systems due to its memory and storage footprints. For example, the size of the shared lib libguile.so.17.4.0 in Guile 1.8.8 is ~3.1MB and ~790KB when stripped.

When building it, you must take care that the Guile version that you want to install in your embedded device must be equal or less than the version you have installed in your host since the cross-building process uses the native Guile. In my case I had guile 2.0.5 installed in my host, but I used guile 1.8.8 since I had several problems while trying to build 2.0.x versions.

First, apply this patch that removes the use of csqrt() and calculates it manually. I think this is a problem with my uclibc.

Configure by specifying where to find ffi and bdw-gc:

Edit the file libguile/Makefile.
Remove the flag -Werror to variable CFLAGS, otherwise it will complain that several variables are set but not used [-Werror=unused-but-set-variable].

Build and install:

libmatheval
Package: libmatheval 1.1.11

Now we can finally build libmatheval!

Edit the file Makefile.in and remove the directory tests from the building process:

Configure, build and install:

For compiling a program tha uses libmatheval you just need to add the linking flag -lmatheval and, of course, include the directories where it was installed, typically /your/rootfs/usr/include and /your/rootfs/usr/lib.

References: