Tuesday, May 8, 2018

OpenWRT on a Ubiquity EdgeRouterX - Part I

We have been using these Ubiquity EdgeRouter X devices for a while now in here (about a year). They're incredible devices; can't say enough about them.

We were curious to see if you could run OpenWRT on these.

The first thing we had to do is to get some intelligence on the device.

We could have cracked open the case and studied the circuit board, but we didn't do that. By using search engines, we were able to ascertain that it was a 32 bit MIPS processor (MIPS Technologies); MIPS32 1004K. They make a couple of renditions of these chips (i.e. MIPS321004Kc or MIPS321003Kf), the c standing for "coherent" and the f standing for "floating point".

After logging onto the device and looking in /proc/cpuinfo (issuing the command "show hardware cpu" also shows this information), we were able to ascertain that the board (system type?) is MediaTek MT6721 ver:1 eco:3. The processor was confirmed as MIPS 1004Kc V2.15.

So the next step was to figure out how to download and compile OpenWRT on this chipset. This can get hairy and complicated, depending on how much support for the chip there is out in the communities.

As it turns out, if you download OpenWRT on a Linux machine, there is some built-in cross compilation support for MIPS32 right within OpenWRT itself. Apparently the gnu compiler gcc supports the MIPS32 target architecture also, which is nice [ NOTE: validate this ].

If there wasn't, you would need to try to find a toolchain (which has compiler, linker and so forth), and [try to] build OpenWRT from scratch, and this would also include building any and all libraries that the code referenced - a potentially huge job. Just doing the Makefiles could take a week or two.

So to compile it, you need to download the OpenWRT source code (I created a directory called /opt/crosscompile and untarred to that, which creates a openwrt directory) and from the top level directory run "make config", or, in my case, I ran "make menuconfig". Running menuconfig pulls up a curses-based menu that allows you to configure your build - including the options to include in your OpenWRT and target platform specifics for a cross-compile.

Build Options
Originally when I ran this, I went to select the Target System and chose "MIPS Malta CoreLV board (qemu)". There was also an option called "MIPS pistachio". I was confused as to which was the right one. And - YOU BETTER CHOOSE THE RIGHT ONE! If you don't, you can risk bricking (ruining) the device when you try to run the code.

It turns out, that neither is the right one. The correct choice for "Target System" was further down below, called "MediaTek Ralink MIPS" (other than the Mediatek and MIPS, there wouldn't be any easy way to figure this out).

The Subtarget is MT6721, which now starts to make some sense based on the output of what showed up in /proc/cpuinfo on the Ubiquity device. But this subtarget would never have shown up as an option if you didn't get the proper Target System right to begin with.

If you select the proper Target and Subtarget, you get an option for "Ubiquity EdgeRouter X" as the Target Profile.

And finally, for "Target Image", you need to make sure that "ramdisk" is selected because this generates a "img" file that is tarred into a tarball that you will eventually boot into the Ubiquity image loader to boot the OpenWRT image you compile.

I made no changes to the Global Build Settings. Initially, I did select "Build the OpenWRT Image Builder" and "Build the OpenWRT SDK" options in the make menuconfig menu listing, but I had to back these out because the compile was spewing out warnings about library dependencies.

Libraries
Running the build without fully reading the README is almost always a mistake, and it bit me here as well. There is a script called "feeds" that will update and install the dependent libraries. This needs to be run.

This script failed when I ran it, and this COULD be because OpenWRT had an issue with their Github site that day (the mirrors).

I went on the web and did some checking, and the packages I needed in the build seemed to be on https://github.com/XXXXX as opposed to https://openwrt.git.org which is what the scripts were all primed with. So what I did was to fix these urls one at a time, and iteratively kick off the build until it launched with no dependent library warnings.

Build
After the build completed (which took a while - about 2 hours - in a 2 core VirtualBox virtual machine running CentOS7 Linux), I had to figure out where the resultant images were written. They are actually written to the /openwrt/bin/targets folder. The full path is:

/opt/crosscompile/openwrt/bin/targets/ramips/mt7621/openwrt-ramips-mt7621-ubnt-erx-initramfs-factory.tar.

There is also a bin file in this directory as well, but it is the tar you need to load into the device.

Transfer
The next step is secure copy (scp) the tar file into a temporary directory on the Ubiquity EdgeRouterX device. We elected to put it into /tmp.

Then - once copied over, you can log into the UbiquityEdgeRouterX via ssh or through the web console, and go into the temporary directory and type:
"add system image openwrt-ramips-mt7621-ubnt-erx-initramfs-factory.tar"

If this add system command worked you should get a confirmation (i.e. image added). It will place the image on a list of images stored on the device, and mark that image as bootable (apparently it works much like a LIFO queue, where the last image added is the bootable one).

It is now time to reboot the device, by typing "reboot". This is the moment of truth.

But...there is an important step!

Change Ethernet Cable to eth1 on the device!!!!
On the Ubiquity device, eth0 is the management interface, and eth1 is actually the WAN link (eth2-4 are local links).

On OpenWRT, it uses eth0 as a WAN link, and eth1 is the management link.

If you attempt to reboot the device and your cable is [ still ] plugged into the wrong NIC, you will not be able to log in and access the device under OpenWRT.

So switching the cable from eth0 to eth1 is the next step...


Image Re-load
If there any major issues with the image I think adding the image would have flagged that. But the real proof is when you reboot the device.

In our case, the image booted with a "OpenWRT" ASCII MOTD banner, letting you know it booted.

Explore

From here, you can dance around the OpenWRT file systems to your hearts content, as this review stops here.

The only thing I will mention is that I did install the graphical menu for OpenWRT, called Luci.


I will log into that web interface to see what it looks like as I explore OpenWRT.

No comments:

Fixing Clustering and Disk Issues on an N+1 Morpheus CMP Cluster

I had performed an upgrade on Morpheus which I thought was fairly successful. I had some issues doing this upgrade on CentOS 7 because it wa...