Hello,
First of all, I’d like to present myself. My name is Luís, I’m an MsC in Network and System Engineering from the Faculty of Science of University of Porto, Portugal.
The work provided here, is related with my master thesis, which resided on the implementation of mesh networks (namely open80211s) in a current (at the time) CyanogenMod distribution.
Originally, Mesh networks were implemented in CM12, but since the delivery of my thesis I’ve been hard at work to provide the means to enable mesh in CM 13.
I’ve accomplished so, and hence, I’m sharing with the community.
I’ve had good guides throughout my implementations, and hopefully I wont forget to acknowledge anyone who helped me with this implementation.
If you wish to skip the implementation steps, you can download an unnoficial version of the ROM I’ve developed from here. Bare in mind that the device must be rooted, the bootloader must be unlocked, and you must be running TWRP as a recovery partition. In this case I’ve used a piece of software (for windows) called Nexus Root Toolkit (Official Page)
So, let’s start.
Preparing the environment
$ sudo add-apt-repository ppa:openjdk-r/ppa
$ sudo apt-get update
$ sudo apt-get install bison build-essential curl flex git gnupg gperf libesd0-dev liblz4-tool libncurses5-dev libsdl1.2-dev libwxgtk3.0-dev libxml2 libxml2-utils lzop maven pngcrush schedtool squashfs-tools xsltproc zip zlib1g-dev lib32readline6-dev openjdk-7-jdk g++-multilib gcc-multilib lib32ncurses5-dev lib32z1-dev
——————————————————————————————————
Create work folders:
$ mkdir -p ~/bin
$ mkdir -p ~/android/system
——————————————————————————————————
Installing repo:
$ curl https://storage.googleapis.com/git-repo-downloads/repo > ~/bin/repo
$ chmod a+x ~/bin/repo
——————————————————————————————————
Setting up git
luis@static:~/android/system$ git config –global user.email “luis@static.pt”
luis@static:~/android/system$ git config –global user.name “Luis”
——————————————————————————————————
Synching the repo (source code):
$ cd ~/android/system/
$ repo init -u https://github.com/CyanogenMod/android.git -b cm-13.0
Start the source code download:
$ repo sync
[optionally you can run this with “-j X”, where X must be replaced by the number of threads your computer can run at once. If you own a dual core processor, usually you can launch 4 threads, if you own a quad-core, you can usually launch 8 threads. But first check your computer’s CPU on-line, and confirm the number of threads.)
In my case, I used -j 8 to tell the repo command that I have 8 available threads. If you dont know how many threads you can launch, just run “repo sync”
This will hopefully speed up the process of downloading and placing the code in the correct folder hierarchy. This will actually take a large ammount of time, so its time for you to go grab a beer and watch a tv show, as your workstation will be unavailable and heavily overloaded in the next 30min – 2 hours depending on your hardware and internet connection.
——————————————————————————————————-
Load the CM scripts available in the source code and fetch some remote device-specific configurations and files in order to compile accordingly.
$ cd ~/android/system/
$ source build/envsetup.sh
$ breakfast mako
note: Mako is the codename for the Nexus 4 device. If you are implementing in a different device, please make sure you identify its proper code name.
You will then need to grab some proprietary files from the nexus itself. This are drivers and proprietary software available for the device to work properly, such as hardware frequencies, firmware, etc… In this tutorial, we are assuming that your device is listed as supported by the cyanogenmod community. So you should download and install a stable release for your device.
To grab the proprietary files you will need ADB
Make sure your device has “developer options” enabled under settings and allow Android Debugging Bridge. Also, make sure that your device is running on root mode. This is necessary to allow access from ADB to folders that are protected from the common user to access.
To install ADB:
$ sudo apt-get install android-tools-adb
then
$ cd device/lge/mako
$ ./extract-files.sh
And then we can finally make our first standard-source build.
$ croot (go back to the root directory)
$ brunch mako
As it is the first build, it will compile all the source code for the first time. Since CM is an operating system, this will compile thousands of files, and place them accordingly in the “out” directory alongside with the “.zip” flasheable file.
I had a problem with the gello app. At the time of this build the Gello repository was missing a signed SSL key, and hence failing at compiling properly.If you don’t encounter issues while compiling your first ROM build, you can discard the next step. This was a temporary problem related with the SSL key and a keyring software included in the CM source code.
I’ve fixed that issue by commenting the gello app commands in device/lge/mako/device.mk
# Gello
# PRODUCT_PACKAGES += \
# Gello
Since gello is a browser, there is no problem in disabling its build in the compilation process. We can install it via the google playstore or via a standalone apk file.
With that commented, the compilation process finished successfully.
Now its time to flash the device and check if everything if working.
To do so, we connect our device to the computer and run
luis@static:~/android/system/device/lge/mako$ croot
luis@static:~/android/system$ cd $OUT
luis@static:~/android/system/out/target/product/mako$ adb push cm-13.0-20161121-UNOFFICIAL-mako.zip /sdcard/cm_mesh.zip
what we did was upload the compiled file with the name “cm_mesh.zip” to our sdcard. Please remeber that the target zip file name is directly related with the day it was compiled, so yours will vary.
after that, we do
$ adb reboot recovery
The phone will now boot in recovery mode. Once there (and assuming you rooted your phone and it is now resourcing to TWRP as the recovery system)
do “wipe” – and swipe to proceed
back to the main screen
install – select cm_mesh.zip – swipe to install
this may take a few minutes.. MAKE SURE THE DEVICE HAS AT LEAST 20% BATTERY
Once finished “wipe cache/dalvik”
and “reboot system”
If everything went right, the device will now take a few minutes to load and will run your fresh CM rom.
You could also try shutting down the device and start it with the “volume key up + start button”.
——————————————————————————————————
We have now estabelished that our device is in fact running with our source code and that it will work properly if we do everything correctly.
Now lets configure the kernel
$ make -C kernel/lge/mako/ O=$CM_ROOT/out/target/product/$CM_BUILD/obj/KERNEL_OBJ INSTALL_MOD_PATH=../../system ARCH=arm CROSS_COMPILE=”$CM_ROOT/prebuilts/gcc/linux-x86/arm/arm-eabi-4.8/bin/arm-eabi-” menuconfig
In this case, we will need to define:
Enable Loadeable Module Suppor (and all sub-entrys)
Cryptographic API -> CCM Support [M] –> Press “m” in the menuconfig to enable loading as a module.
Networking Support > Wireless >
- cfg80211 – wireless configuration API [M]
- Common Routines for IEEE 802.11 Networking Stack (mac80211)
- Generic IEEE 802.11 Networking Stack (mac80211) [M]
Device Drivers > Stating Drivers > Disable Qualcomm Atheros Prima WLAN module
This command will output a configuration file in the $CM_ROOT/out/target/product/$CM BUILD/obj directory of our development environment. Reason beeing that if the file was not given a different path, the compilation process of our kernel would state that it is not clean (the new file was not in its proper place).
We need to copy it to the proper file location. Since we are compiling a code-name “mako” device, we must copy the .config to the “cyanogen_mako_defconfig” file.
luis@static:~/android/system$ cp out/target/product/mako/obj/KERNEL_OBJ/.config kernel/lge/mako/arch/arm/configs/cyanogen_mako_defconfig
Now that we configured the kernel, its time to give the OS the new driver and the means to load it.
To do so, we must compile the new driver and add it to our distribution. Android kernels are statically linked, meaning that it would be very hard to natively add a driver to it. The next best idea is to add the driver as an external module and load it onboot.
To accomplish that, we must match the kernel module directly onto the serialized kernel we’re building. By serialized, we mean the exact specific kernel version running.
We could compile the CM source, accomplishing a flashable ROM, flash it and then compile the kernel module. We could then upload the module to the device and manually add it via “insmod” or “modprobe”.
We wish to deliver a ROM that is mesh ready right from the start. So, the idea behind it would be to integrate the kernel module creation within the ROM creation itself.
We must assure that the module is compiled after the kernel compilation process as it needs to match the exact version available in the device.
Lets add it to the cellphone
To do so, we need to compile it. wcn is available in the linux kernel as of version 3.16. Our kernel version is 3.4.. to make it work, we can compatibilizite it with a tool called backports.
We’ve downloaded backports v3.16 (from here) and since wcn36xx needs the wcn36xx_msm.ko module, we downloaded wcn36xx from Eugene Krasnikov’s Github. This contains the wcn36xx_msm code to compile.
As previously stated, in order to successfully compile, we need to match the kernel version against our own version.
To do so, we download backports 3.16 from
and wcn36xx from .
We had to setup backports. To do so, in backports folder:
$ make KLIB=$CM_ROOT/out/target/product/$CM_BUILD/obj/KERNEL_OBJ KLIB_BUILD=$CM_ROOT/out/target/product/$CM_BUILD/obj/KERNEL_OBJ ARCH=arm CROSS_COMPILE=$CM_ROOT/prebuilts/gcc/linux-x86/arm/arm-eabi-4.8/bin/arm-eabi- defconfig-wcn36xx
and then
make KLIB=$CM_ROOT/out/target/product/$CM_BUILD/obj/KERNEL_OBJ KLIB_BUILD=$CM_ROOT/out/target/product/$CM_BUILD/obj/KERNEL_OBJ ARCH=arm CROSS_COMPILE=$CM_ROOT/prebuilts/gcc/linux-x86/arm/arm-eabi-4.8/bin/arm-eabi- menuconfig
and select [*] Enable mac80211 mesh networking (pre-802.11s) support
The reason we need to download wcn36xx appart from backports is that the driver requires the wcn36xx_msm module as well as the wcn36xx module. Backports will only generate the wcn36xx module and the wcn36xx code from Eugene enables the compilation of the wcn36xx_msm module
go to ~/android/system/external and create a folder named wcn36xx. inside it place the backports you’ve downloaded and rename its folder to “backports”. Then, copy the wcn36xx code and rename it to “wcn36xx_msm”.
We want to integrate this process within the cyanogenmod build. Cyanogenmods build process is based on an hierarchy of makefiles, where every makefile present in a given folder will be called in its proper order. In order to guarantee that our modules will be matched to the specific kernel we are building, we need to integrate this process right after the kernel build.
The way I personally did this, was by finding where the kernel build process was called and add the proper lines after.
I’ve found out that the kernel compilation steps are done in
nano build/core/tasks/kernel.mk
Also, I found specifically where the kernel was beeing compiled.
So you must search for this piece of code:
echo -e ${CL_GRN}”Building Kernel Modules”${CL_RST} ; \
$(MAKE) $(MAKE_FLAGS) -C $(KERNEL_SRC) O=$(KERNEL_OUT) ARCH=$(KERNEL_ARCH) $(KERNEL_CROSS_COMPILE) modules && \
$(MAKE) $(MAKE_FLAGS) -C $(KERNEL_SRC) O=$(KERNEL_OUT) INSTALL_MOD_PATH=../../$(KERNEL_MODULES_INSTALL) ARCH=$(KERNEL_ARCH) $(KERNEL_CROSS_COMPILE) modules_install && \
$(mv-modules) && \
$(clean-module-folder) ; \
else \
echo “Kernel Modules not enabled” ; \
fi ;
And immediatly after, paste:
#——————————————————————————-
echo -e ${CL_GRN}”Compiling wcn36xx wireless driver for 3.4 msm kernel”${CL_RST}$(MAKE) -C $(ANDROID_BUILD_TOP)/external/wcn36xx/backports KLIB=$(KERNEL_OUT) KLIB_BUILD=$(KERNEL_OUT) ARCH=$(KERNEL_ARCH) $(KERNEL_CROSS_COMPILE)
$(MAKE) -C $(ANDROID_BUILD_TOP)/external/wcn36xx/wcn36xx_msm/wcn36xx_msm KLIB=$(KERNEL_OUT) KLIB_BUILD=$(KERNEL_OUT) ARCH=$(KERNEL_ARCH) $(KERNEL_CROSS_COMPILE)
cp $(ANDROID_BUILD_TOP)/external/wcn36xx/wcn36xx_msm/wcn36xx_msm/wcn36xx_msm.ko $(OUT)/system/lib/modules/wcn36xx_msm.ko
cp $(ANDROID_BUILD_TOP)/external/wcn36xx/backports/compat/compat.ko $(OUT)/system/lib/modules
cp $(ANDROID_BUILD_TOP)/external/wcn36xx/backports/net/wireless/cfg80211.ko $(OUT)/system/lib/modules/
cp $(ANDROID_BUILD_TOP)/external/wcn36xx/backports/net/mac80211/mac80211.ko $(OUT)/system/lib/modules/
cp $(ANDROID_BUILD_TOP)/external/wcn36xx/backports/drivers/net/wireless/ath/wcn36xx/wcn36xx.ko $(OUT)/system/lib/modules/
# ——————————————————————————
This will call the backports and wcn36xx makefiles right after the compilation process as well as place them on the proper location in the output directory to be finally compressed onto the flashable zip file.
Now, we need to instruct the ROM to load the modules on boot. To do so, we create :
$ touch device/lge/mako/init.mako.rc
and add to the On Boot section:
insmod /system/lib/modules/wcn36xx_msm.ko
insmod /system/lib/modules/compat.ko
insmod /system/lib/modules/cfg80211.ko
insmod /system/lib/modules/mac80211.ko
insmod /system/lib/modules/wcn36xx.ko
insmod /system/lib/modules/ccm.ko
This will load the modules we created.
Now we need to edit device/lge/mako/init.mako.rc
We will copy the wpa_supplicant service and rename it to service p2p_service. This will enable direct communication between devices (previously wi-fi direct was the norm for direct communication via wifi)
So we will copy the wpa_supplicant service, comment the current p2p_supplicant and rename the copied wpa_supplicant entry to p2p_supplicant, leaving us with:
service p2p_supplicant /system/bin/wpa_supplicant \
-iwlan0 -Dnl80211 -c/data/misc/wifi/wpa_supplicant.conf \
-I/system/etc/wifi/wpa_supplicant_overlay.conf \
-e/data/misc/wifi/entropy.bin -g@android:wpa_wlan0
# we will start as root and wpa_supplicant will switch to user wifi
# after setting up the capabilities required for WEXT
# user wifi
# group wifi inet keystore
class main
socket wpa_wlan0 dgram 660 wifi wifi
disabled
oneshot
#service p2p_supplicant /system/bin/wpa_supplicant \
# -ip2p0 -Dnl80211 -c/data/misc/wifi/p2p_supplicant.conf \
# -I/system/etc/wifi/p2p_supplicant_overlay.conf -N \
# -iwlan0 -Dnl80211 -c/data/misc/wifi/wpa_supplicant.conf \
# -I/system/etc/wifi/wpa_supplicant_overlay.conf \
# -puse_p2p_group_interface=1 \
# -e/data/misc/wifi/entropy.bin -g@android:wpa_wlan0
# we will start as root and wpa_supplicant will switch to user wifi
# after setting up the capabilities required for WEXT
# user wifi
# group wifi inet keystore
# class main
# socket wpa_wlan0 dgram 660 wifi wifi
# disabled
# oneshot
Now we need to add a new service (instruction) and instruct it to be ran
The insmod instruction is beeing run onBoot by the init process. This process is restrained by SELinux (Security layer) to load modules, so we must grant him permissions:
$ nano ~/android/system/device/lge/mako/selinux/init.te
And we add
allow init self:capability sys_module;
Now we can test our changed by compilling the rom.
$ croot
$ brunch mako
Once uploaded, the problems started. My modules were not showing up, and I was getting
root@mako:/system/lib/modules # insmod wcn36xx_msm.ko
insmod: failed to load wcn36xx_msm.ko: Bad address
This produced zero kernel errors or log entries.
Thanks to the kindness of forkbomb in #cyanogenmod-dev in irc.freenode which pointed me to
https://review.cyanogenmod.org/#/c/136537/2
which led to
once I downloaded the toybox file, I’ve uploaded it to the phone and ran
$ adb push toyboy /system/lib/modules
$ adb shell
$ cd /system/lib/modules
$ chmod +x toybox
$ ./toybox insmod wcn36xx_msm.ko
and it worked!! So, to make this an defenitive solution, what I did was:
$ mkdir ~/android/system/external/working_toybox
$ cp toybox ~/android/system/external/working_toybox
$ chmod a+x ~/android/system/external/working_toybox/toybox
and addded:
#Toybox
PRODUCT_COPY_FILES += \
external/working_toybox/toybox:system/bin/toybox
To the end of ~/android/system/device/lge/mako/device.mk
To finish, you can add iw to your build. To do so download the pre-compiled iw version from here.
Create a new folder under external named “working_iw”
then add:
#Toybox
PRODUCT_COPY_FILES += \
external/working_iw/iw:system/bin/iw \
external/working_iw/libnl-3.so:system/lib/modules/libnl-3.so \
external/working_iw/libnl-genl-3.so:system/lib/modules/libnl-genl-3.so
I then compiled a final version of the rom, and flashed the device.