11 Feb 2021
Planet Maemo
reBounce - softfp-to-hardfp LD_PRELOAD hack for Bounce on N9
This depends on Bounce (the N900 .deb) and SDL 1.2 being installed. Google "bounce_1.0.0_armel.deb" for the former, and use n9repomirror for the latter.
#if 0
gcc -Os -shared -fPIC -lSDL -o librebounce.so rebounce.c
LD_PRELOAD=$(pwd)/librebounce.so /opt/bounce/bin/bounce
exit 0
#endif
/**
* reBounce -- softfp-to-hardfp LD_PRELOAD hack for Bounce on N9
*
* Bounce was a really nice 2009 tech demo on the N900. This
* makes this tech demo work on an N9 by translating the calls
* that use floating point arguments to the hardfp ABI. It also
* fixes input using SDL1.2 to get sensor values from sensorfw.
*
* Known issues: Audio is muted on startup until mute is toggled.
*
* 2021-02-11 Thomas Perl <m@thp.io>
**/
#define _GNU_SOURCE
#include <stdio.h>
#include <dlfcn.h>
#include <pthread.h>
#include <SDL/SDL.h>
#define SFP __attribute__((pcs("aapcs")))
typedef unsigned int GLuint;
static void *
sensor_thread(void *user_data)
{
SDL_Init(SDL_INIT_JOYSTICK | SDL_INIT_VIDEO);
SDL_Joystick *accelerometer = SDL_JoystickOpen(0);
while (1) {
SDL_JoystickUpdate();
float x = 0.053888f * SDL_JoystickGetAxis(accelerometer, 0);
float y = 0.053888f * SDL_JoystickGetAxis(accelerometer, 1);
float z = 0.053888f * SDL_JoystickGetAxis(accelerometer, 2);
FILE *out = fopen("/dev/shm/bounce.sensor.tmp", "wb");
fprintf(out, "%f %f %f\n", -y, x, z);
fclose(out);
rename("/dev/shm/bounce.sensor.tmp", "/dev/shm/bounce.sensor");
SDL_Delay(10);
}
return NULL;
}
FILE *
fopen(const char *filename, const char *mode)
{
FILE *(*fopen_orig)(const char *, const char *) = dlsym(RTLD_NEXT, "fopen");
if (strcmp(filename, "/sys/class/i2c-adapter/i2c-3/3-001d/coord") == 0) {
static int sensor_inited = 0;
if (!sensor_inited) {
sensor_inited = 1;
pthread_t thread;
pthread_create(&thread, NULL, sensor_thread, NULL);
}
filename = "/dev/shm/bounce.sensor";
}
return fopen_orig(filename, mode);
}
#define f_f(name) float SFP name(float x) { return ((float (*)(float))dlsym(RTLD_NEXT, #name))(x); }
#define d_d(name) double SFP name(double x) { return ((double (*)(double))dlsym(RTLD_NEXT, #name))(x); }
#define f_ff(name) float SFP name(float x, float y) { return ((float (*)(float, float))dlsym(RTLD_NEXT, #name))(x, y); }
#define d_dd(name) double SFP name(double x, double y) { return ((double (*)(double, double))dlsym(RTLD_NEXT, #name))(x, y); }
f_f(sinhf) f_f(coshf) f_f(tanhf) f_f(asinf) f_f(acosf) f_f(atanf) f_f(sinf) f_f(cosf) f_f(tanf) f_f(expf) f_f(logf)
f_f(log10f) f_f(ceilf) f_f(floorf) d_d(log) d_d(sin) f_ff(atan2f) f_ff(fmodf) d_dd(atan2) d_dd(pow) d_dd(fmod)
double SFP
frexp(double value, int *exp)
{
return ((double (*)(double, int *))dlsym(RTLD_NEXT, "frexp"))(value, exp);
}
double SFP
ldexp(double x, int n)
{
return ((double (*)(double, int))dlsym(RTLD_NEXT, "ldexp"))(x, n);
}
double SFP
modf(double value, double *iptr)
{
return ((double (*)(double, double *))dlsym(RTLD_NEXT, "modf"))(value, iptr);
}
void SFP
glClearColor(float r, float g, float b, float a)
{
((void (*)(float, float, float, float))dlsym(RTLD_NEXT, "glClearColor"))(r, g, b, a);
}
void SFP
glUniform4f(GLuint location, float v0, float v1, float v2, float v3)
{
((void (*)(GLuint, float, float, float, float))dlsym(RTLD_NEXT, "glUniform4f"))(location, v0, v1, v2, v3);
}
void SFP
glUniform1f(GLuint location, float v0)
{
((void (*)(GLuint, float))dlsym(RTLD_NEXT, "glUniform1f"))(location, v0);
}
11 Feb 2021 7:41am GMT
04 Feb 2021
Planet Maemo
Ubuntu Touch porting notes for the Redmi Note 7 Pro, part 2
This is the second part of my porting odyssey; for the first part, follow this link.
The good news is that I've done some progress; the bad news is that there are still plenty of issues, and solving them involves deep diving into nearly all components and technologies that make up the core of an Android device, so completing the porting is going to take quite some time. On the other hand, I'm learning a lot of new stuff, and I might be able to share it by writing some documentation. And, who knows, maybe work on some other device port.
Anyway, enough with the introduction! Let's see what progress I've been doing so far.
The new device tree
While asking for help to debug the audio issue I was facing (more about that later), I was also told that the lavender
tree, which I was using as a reference, was obsolete. The new one was in gitlab, and was build with a totally different system, described here.
So, I picked the lavender
tree and adapted it for violet
: I changed the deviceinfo
file to point to my kernel tree, use my kernel configuration, and use the same boot command line as before. By the way, here's my current "new" device tree. The build failed, with errors like:
In function 'memcpy', inlined from 'proc_ipc_auto_msgmni.part.1' at /home/mardy/projects/port/xiaomi-violet/bd/downloads/android_kernel_xiaomi_violet/ipc/ipc_sysctl.c:82:2: /home/mardy/projects/port/xiaomi-violet/bd/downloads/android_kernel_xiaomi_violet/include/linux/string.h:340:4: error: call to '__read_overflow2' declared with attribute error: detected read beyond size of object passed as 2nd parameter __read_overflow2(); ^
I then replayed the kernel build using the old system, and noticed that it was using clang as the compiler; so I changed the related flag in the deviceinfo
file, and the build went past that point. It failed later, though:
/home/mardy/projects/port/xiaomi-violet/bd/downloads/android_kernel_xiaomi_violet/drivers/staging/qcacld-3.0/core/sap/src/sap_fsm.c:1498:26: error: cast to smaller integer type 'eSapStatus' from 'void *' [-Werror,-Wvoid-pointer-to-enum-cast] bss_complete->status = (eSapStatus) context; ^~~~~~~~~~~~~~~~~~~~
I ended up editing the Kbuild file in the module source directory, and removed the -Werror
from there (as well in another place that failed a bit later). This made the build proceed until the end, where it failed because the device tree file was not found:
+ cp -v /home/mardy/projects/port/xiaomi-violet/bd/downloads/KERNEL_OBJ/arch/arm64/boot/dts/qcom/sm8150-mtp-overlay.dtbo /home/mardy/projects/port/xiaomi-violet/bd/tmp/partitions/dtbo.img cp: cannot stat '/home/mardy/projects/port/xiaomi-violet/bd/downloads/KERNEL_OBJ/arch/arm64/boot/dts/qcom/sm8150-mtp-overlay.dtbo': No such file or directory
I quickly realized that this was due to an error of mine: the Xiaomi violet
is a sm6150
, not a sm8150
as mentioned in my deviceinfo
file! But what overlay file should I use then, since there isn't a sm6150-mtp-overlay.dts
in my source tree? Having a loot at other deviceinfo files, I saw that the deviceinfo_kernel_dtb_overlay
line is not always there, so I tried commenting it out, and it helped.
The next step was getting flashable images, of course. While the device is not supported by the UBports system-image server, we can use use some scripts to create a fake OTA (Over The Air update) and generate flashable images starting from it. The steps can be read in the devel-flashable
target of the .gitlab-ci.yml
file (if this step is not present, we should try to find another device repository which has it. They are the following:
./build/prepare-fake-ota.sh out/device_violet.tar.xz ota ./build/system-image-from-ota.sh ota/ubuntu_command out
Once these commands have completed their execution, these commands will push the images to the device:
fastboot flash boot out/boot.img; fastboot flash system out/system.img
There's also fastboot flash recovery out/recovery.img
, but I left it out since I was not interested in the recovery image at this stage. And unless you are ready to submit your port for inclusion into the "supported devices" list, I'd recommend not flashing the UT recovery image since TWRP is more powerful and will likely help you in recover your device from a broken image.
Kernel command line
It's important that the kernel command line contains the systempart
parameter. In my case, the first boot failed because the command line was longer than 512 bytes, and this parameter was getting truncated. So one thing to be careful about is the length of the kernel command line.
This was fixed by removing some unnecessary kernel parameters from the deviceinfo
file.
Missing thumbnails in the Gallery app, content-hub not working
Another issue I noticed is that photo thumbnails were all black in the Gallery app, and the content-hub also was not working. I noticed a relevant commit in the lavender kernel tree and found a launchpad bug which mentioned the issues I was seeing. In the Telegram channel I was told that patch is a forward port of a commit from kernel 3.4 that was present in all of the cores devices, and that it was indeed needed to have the ContentHub working.
The patch did not apply cleanly on top of my kernel tree, but luckily it was just an offset issue: adjusting the patch was easy, and indeed after applying it thumbnails started appearing in the Gallery and Imaginario could import photos again via the ContentHub.
Time and date
Time was always reset to a far away date after a reboot. This is a common issue on Qualcomm devices, and can be fixed by disabling the time service in the Android container.
https://github.com/Halium/android_device_halium_halium_arm64/pull/3
For some reason, at a certain point my override stopped working (or, more likely, the override never worked, but I happened to have fixed the issue directly modifying the vendor init file). I had to copy /android/vendor/etc/init/hw/init.qcom.rc
into /usr/share/halium-overrides/
, modify it and bind mount the modified file in order to get it working.
This actually seems to match my understanding of the Android init documentation, because according to the path priorities a configuration file stored under /system/
will never be able to override one stored under /vendor
.
Fixing the audio configuration
Audio was not working at all. The sound indicator icon was not shown, only the raw (unstranslated) "indicator-sound" text was shown in the panel. pulseaudio
was not running. Trying to run it manually (as the phablet user, since pulseaudio is run in the user session) led to this:
phablet@ubuntu-phablet:~$ pulseaudio -n -vvv -F /etc/pulse/touch-android9.pa I: [pulseaudio] main.c: setrlimit(RLIMIT_NICE, (31, 31)) failed: Operation not permitted I: [pulseaudio] main.c: setrlimit(RLIMIT_RTPRIO, (9, 9)) failed: Operation not permitted ... D: [pulseaudio] cli-command.c: Parsing script '/etc/pulse/touch-android9.pa' D: [pulseaudio] database-tdb.c: Opened TDB database '/home/phablet/.config/pulse/ubuntu-phablet-stream-volumes.tdb' I: [pulseaudio] module-stream-restore.c: Successfully opened database file '/home/phablet/.config/pulse/ubuntu-phablet-stream-volumes'. ... D: [pulseaudio] module.c: Checking for existence of '/usr/lib/pulse-8.0/modules/module-droid-card-28.so': success I: [pulseaudio] module-droid-card.c: Create new droid-card D: [pulseaudio] droid-util.c: No configuration provided for opening module with id primary I: [pulseaudio] config-parser-xml.c: Failed to open file (/odm/etc/audio_policy_configuration.xml): No such file or directory D: [pulseaudio] droid-config.c: Failed to parse configuration from /odm/etc/audio_policy_configuration.xml D: [pulseaudio] config-parser-xml.c: Read /vendor/etc/audio/audio_policy_configuration.xml ... D: [pulseaudio] config-parser-xml.c: New module: "primary" W: [pulseaudio] config-parser-xml.c: [/vendor/etc/audio/audio_policy_configuration.xml:78] Could not find element attribute "samplingRates" E: [pulseaudio] config-parser-xml.c: [/vendor/etc/audio/audio_policy_configuration.xml:78] Failed to parse element <profile> E: [pulseaudio] config-parser-xml.c: parsing aborted at line 78 D: [pulseaudio] droid-config.c: Failed to parse configuration from /vendor/etc/audio/audio_policy_configuration.xml D: [pulseaudio] config-parser-xml.c: Read /vendor/etc/audio_policy_configuration.xml ... D: [pulseaudio] config-parser-xml.c: New module: "primary" W: [pulseaudio] config-parser-xml.c: [/vendor/etc/audio_policy_configuration.xml:78] Could not find element attribute "samplingRates" E: [pulseaudio] config-parser-xml.c: [/vendor/etc/audio_policy_configuration.xml:78] Failed to parse element <profile> E: [pulseaudio] config-parser-xml.c: parsing aborted at line 78 D: [pulseaudio] droid-config.c: Failed to parse configuration from /vendor/etc/audio_policy_configuration.xml I: [pulseaudio] config-parser-legacy.c: Failed to open config file (/vendor/etc/audio_policy.conf): No such file or directory D: [pulseaudio] droid-config.c: Failed to parse configuration from /vendor/etc/audio_policy.conf I: [pulseaudio] config-parser-xml.c: Failed to open file (/system/etc/audio_policy_configuration.xml): No such file or directory D: [pulseaudio] droid-config.c: Failed to parse configuration from /system/etc/audio_policy_configuration.xml I: [pulseaudio] config-parser-legacy.c: Failed to open config file (/system/etc/audio_policy.conf): No such file or directory D: [pulseaudio] droid-config.c: Failed to parse configuration from /system/etc/audio_policy.conf E: [pulseaudio] droid-config.c: Failed to parse any configuration. ...
Indeed, line 78 of /vendor/etc/audio_policy_configuration.xml
had an error, where a property was spelt like simplingRate
instead of samplingRate
. However, the "vendor" partition is read-only, so I couldn't change that file directly. Another option could have been creating a fixed copy of the file and place it with /system/etc/audio_policy_configuration.xml
, but the "system" is also read-only (there are ways to modify these partitions, of course, but I couldn't find a clean way to do it from the device tree scripts. So I went for the bind-mount approach: I would ship the fixed file in some other directory of the file-system, and then modify the /etc/init/mount-android.conf
file (this is the job that upstart executes before starting the Android LXC container) to bind-mount the file onto /vendor/etc/audio_policy_configuration.xml
.
This worked, but my joy was short-lived: audio was coming up only once every 5 boots or so. I will not list here all the things I tried, as they were plenty of them; and more than once I went to sleep happy and convinced of having fixed the issue for good, until the next day the device booted without audio. It was clearly a timing issue occurring in the early boot, because one thing I clearly noticed very early on is that in those cases when the audio was booting, the following lines appeared in the kernel log:
[ 7.130057] wcd937x_codec wcd937x-codec: msm_cdc_dt_parse_vreg_info: cdc-vdd-ldo-rxtx: vol=[1800000 1800000]uV, curr=[25000]uA, ond 0 [ 7.130068] wcd937x_codec wcd937x-codec: msm_cdc_dt_parse_vreg_info: cdc-vddpx-1: vol=[1800000 1800000]uV, curr=[10000]uA, ond 0 [ 7.130076] wcd937x_codec wcd937x-codec: msm_cdc_dt_parse_vreg_info: cdc-vdd-mic-bias: vol=[3296000 3296000]uV, curr=[25000]uA, ond 0 [ 7.130084] wcd937x_codec wcd937x-codec: msm_cdc_dt_parse_vreg_info: cdc-vdd-buck: vol=[1800000 1800000]uV, curr=[650000]uA, ond 1 [ 7.137759] wcd937x_codec wcd937x-codec: bound wcd937x-slave.1170224 (ops cleanup_module [wcd937x_slave_dlkm]) [ 7.138065] wcd937x_codec wcd937x-codec: bound wcd937x-slave.1170223 (ops cleanup_module [wcd937x_slave_dlkm])
I started adding some printk to the kernel driver, and modified it slightly to register itself with the module_driver()
macro instead of the simpler, but logless, module_platform_driver()
. This showed that the driver was always loaded at about 7 seconds, but the platform_driver_register()
method only called the driver's bind method (wcd937x_bind()
) in those boots where audio was working.
After more debugging into platform_driver_register()
, I stumbled upon the platform_match()
function, added some debugging message in there to print the device name and the driver name, and observed how in those boots where audio was failing this function was called to find a driver for the wcd937x_codec
device before the wcd937x_codec
driver (provided by the wcd937x_dlmk
module) was available. So, I tried adding wcd937x_dlmk
to /etc/modules-load.d/modules.conf
and this caused the driver to be loaded at about 3 seconds, and apparently fixed the audio issue. At least, till the time of writing this, I never had my phone boot without audio anymore.
Not all is fine with the audio, unfortunately: the mic records a background noise along with the actual sounds, and the recording volume is quite low. This also affects the call quality. On the other hand, the noise disappears when recording happens via the earphones. But I've yet to investigate this; I hope to give you some updates in part three.
04 Feb 2021 8:34pm GMT
17 Jan 2021
Planet Maemo
Ubuntu Touch porting notes for the Redmi Note 7 Pro
In case you have a sense of deja-vu when reading this post, it's because indeed this is not the first time I try porting a device to Ubuntu Touch. The previous attempt, however, was with another phone model (and manufacturer), and did not have a happy ending. This time it went better, although the real ending is still far away; but at least I have something to celebrate.
The phone
I made myself a Christmas present and bought a Xiaomi Redmi Note 7 Pro, a dual-SIM phone from 2019 with 6GB of RAM and 128GB of flash storage. To be totally honest, I bought it by mistake: the phone I really wanted to buy is the Redmi Note 7 (without the "Pro"), because it's a modern phone that is working reasonable well with Ubuntu Touch. The online shop where I bought it from let me choose some options, including the RAM size, so I chose the maximum available (6GB) without being aware that this would mean that I would be buying the "Pro" device - the shop did not alter the item name, so I couldn't really know. Unfortunately, the two versions are two rather different beasts, powered by different SoC; both are produced by Qualcomm, so they are not that different, but it's enough to make the installation of Ubuntu Touch impossible.
But between the choice of retuning the phone to the shop and begin a new porting adventure, I stood firm and went for the latter. Hopefully I won't regret it (if everything goes bad, I can still use it with LineageOS, which runs perfectly on it).
Moreover, there already exist a port of Ubuntu Touch for this phone, which actually works reasonably well (I tried it briefly, and many things were working), but the author claims to be a novice and indeed did not follow the best git practices when working on the source code, so it's hard to understand what was changed and why. But if you are looking for a quick way to get Ubuntu Touch working on this phone, you are welcome to have a look at this Telegram channel.
What follows are the raw notes of my attempts. They are here so that search engines can index the error messages and the various logs, and hopefully help someone hitting similar errors on other devices to find his way out.
Getting Halium and the device source code
Installed the dependencies like in the first step. repo
was not found in the Ubuntu 20.04 archives, but I had it installed anyway due to my work on a Yocto device.
Since my device has Android 9 on it, I went for Halium 9:
repo init -u git://github.com/Halium/android.git -b halium-9.0 --depth=1 repo sync -c -j 16
The official LineageOS repository for the Xiaomi Redmi Note 7 Pro is android_device_xiaomi_violet, but the page with the official build has been taken down (some DMCA violation, if you believe the forums) and no development has been happening since last year. A more active forum thread uses another repository which seems to be receiving more frequent updates, so I chose to base my port on that.
I actually tested that LineageOS image on my phone, and verified that all the hardware was working properly.
Initially, I created forks of the relevant repository under my own gitlab account, but then I though (especially looking at the Note 7 port) that creating a group just for this port would make people's life easier, because they wouldn't need to navigate through my 1000 personal projects to find what is relevant for this port. So, I created these forks:
- gitlab.com/ubuntu-touch-xiaomi-violet/android_device_xiaomi_violet
- gitlab.com/ubuntu-touch-xiaomi-violet/android_kernel_xiaomi_violet
- gitlab.com/ubuntu-touch-xiaomi-violet/android_vendor_xiaomi_violet
Next, created the halium/devices/manifests/xiaomi_violet.xml
file with this content:
<?xml version="1.0" encoding="UTF-8"?> <manifest> <!-- Remotes --> <remote name="ubuntu-touch-xiaomi-violet" fetch="https://gitlab.com/ubuntu-touch-xiaomi-violet" revision="halium-9.0"/> <!-- Device Tree --> <project path="device/xiaomi/violet" name="android_device_xiaomi_violet" remote="ubuntu-touch-xiaomi-violet" /> <!-- Kernel --> <project path="kernel/xiaomi/violet" name="android_kernel_xiaomi_violet" remote="ubuntu-touch-xiaomi-violet" /> <!-- Proprietary/Vendor blobs --> <project path="vendor/xiaomi/violet" name="android_vendor_xiaomi_violet" remote="ubuntu-touch-xiaomi-violet" /> </manifest>
Fetching all the sources mentioned in the manifest:
./halium/devices/setup violet
Full output:
***************************************** I: Configuring for device xiaomi_violet ***************************************** Fetching projects: 100% (393/393), done. hardware/qcom/audio-caf/apq8084: Shared project LineageOS/android_hardware_qcom_audio found, disabling pruning. hardware/qcom/audio-caf/msm8916: Shared project LineageOS/android_hardware_qcom_audio found, disabling pruning. hardware/qcom/audio-caf/msm8952: Shared project LineageOS/android_hardware_qcom_audio found, disabling pruning. hardware/qcom/audio-caf/msm8960: Shared project LineageOS/android_hardware_qcom_audio found, disabling pruning. hardware/qcom/audio-caf/msm8974: Shared project LineageOS/android_hardware_qcom_audio found, disabling pruning. hardware/qcom/audio-caf/msm8994: Shared project LineageOS/android_hardware_qcom_audio found, disabling pruning. hardware/qcom/audio-caf/msm8996: Shared project LineageOS/android_hardware_qcom_audio found, disabling pruning. hardware/qcom/audio-caf/msm8998: Shared project LineageOS/android_hardware_qcom_audio found, disabling pruning. hardware/qcom/audio-caf/sdm845: Shared project LineageOS/android_hardware_qcom_audio found, disabling pruning. hardware/qcom/audio-caf/sm8150: Shared project LineageOS/android_hardware_qcom_audio found, disabling pruning. hardware/qcom/audio/default: Shared project LineageOS/android_hardware_qcom_audio found, disabling pruning. hardware/qcom/display: Shared project LineageOS/android_hardware_qcom_display found, disabling pruning. hardware/qcom/display-caf/apq8084: Shared project LineageOS/android_hardware_qcom_display found, disabling pruning. hardware/qcom/display-caf/msm8916: Shared project LineageOS/android_hardware_qcom_display found, disabling pruning. hardware/qcom/display-caf/msm8952: Shared project LineageOS/android_hardware_qcom_display found, disabling pruning. hardware/qcom/display-caf/msm8960: Shared project LineageOS/android_hardware_qcom_display found, disabling pruning. hardware/qcom/display-caf/msm8974: Shared project LineageOS/android_hardware_qcom_display found, disabling pruning. hardware/qcom/display-caf/msm8994: Shared project LineageOS/android_hardware_qcom_display found, disabling pruning. hardware/qcom/display-caf/msm8996: Shared project LineageOS/android_hardware_qcom_display found, disabling pruning. hardware/qcom/display-caf/msm8998: Shared project LineageOS/android_hardware_qcom_display found, disabling pruning. hardware/qcom/display-caf/sdm845: Shared project LineageOS/android_hardware_qcom_display found, disabling pruning. hardware/qcom/display-caf/sm8150: Shared project LineageOS/android_hardware_qcom_display found, disabling pruning. hardware/qcom/media: Shared project LineageOS/android_hardware_qcom_media found, disabling pruning. hardware/qcom/media-caf/apq8084: Shared project LineageOS/android_hardware_qcom_media found, disabling pruning. hardware/qcom/media-caf/msm8916: Shared project LineageOS/android_hardware_qcom_media found, disabling pruning. hardware/qcom/media-caf/msm8952: Shared project LineageOS/android_hardware_qcom_media found, disabling pruning. hardware/qcom/media-caf/msm8960: Shared project LineageOS/android_hardware_qcom_media found, disabling pruning. hardware/qcom/media-caf/msm8974: Shared project LineageOS/android_hardware_qcom_media found, disabling pruning. hardware/qcom/media-caf/msm8994: Shared project LineageOS/android_hardware_qcom_media found, disabling pruning. hardware/qcom/media-caf/msm8996: Shared project LineageOS/android_hardware_qcom_media found, disabling pruning. hardware/qcom/media-caf/msm8998: Shared project LineageOS/android_hardware_qcom_media found, disabling pruning. hardware/qcom/media-caf/sdm845: Shared project LineageOS/android_hardware_qcom_media found, disabling pruning. hardware/qcom/media-caf/sm8150: Shared project LineageOS/android_hardware_qcom_media found, disabling pruning. hardware/ril: Shared project LineageOS/android_hardware_ril found, disabling pruning. hardware/ril-caf: Shared project LineageOS/android_hardware_ril found, disabling pruning. hardware/qcom/wlan: Shared project LineageOS/android_hardware_qcom_wlan found, disabling pruning. hardware/qcom/wlan-caf: Shared project LineageOS/android_hardware_qcom_wlan found, disabling pruning. hardware/qcom/bt: Shared project LineageOS/android_hardware_qcom_bt found, disabling pruning. hardware/qcom/bt-caf: Shared project LineageOS/android_hardware_qcom_bt found, disabling pruning. Updating files: 100% (67031/67031), done.nel/testsUpdating files: 36% (24663/67031) lineage/scripts/: discarding 1 commits Updating files: 100% (1406/1406), done.ineageOS/android_vendor_qcom_opensource_thermal-engineUpdating files: 47% (666/1406) Checking out projects: 100% (392/392), done. ******************************************* I: Refreshing device vendor repository: device/xiaomi/violet I: Processing proprietary blob file: device/xiaomi/violet/./proprietary-files.txt I: Processing fstab file: device/xiaomi/violet/./rootdir/etc/fstab.qcom I: Removing components relying on SettingsLib from: device/xiaomi/violet *******************************************
Starting the build
Setting up the environment:
$ source build/envsetup.sh including device/xiaomi/violet/vendorsetup.sh including vendor/lineage/vendorsetup.sh $
Running the breakfast command:
$ breakfast violet including vendor/lineage/vendorsetup.sh Trying dependencies-only mode on a non-existing device tree? ============================================ PLATFORM_VERSION_CODENAME=REL PLATFORM_VERSION=9 LINEAGE_VERSION=16.0-20210109-UNOFFICIAL-violet TARGET_PRODUCT=lineage_violet TARGET_BUILD_VARIANT=userdebug TARGET_BUILD_TYPE=release TARGET_ARCH=arm64 TARGET_ARCH_VARIANT=armv8-a TARGET_CPU_VARIANT=kryo300 TARGET_2ND_ARCH=arm TARGET_2ND_ARCH_VARIANT=armv8-a TARGET_2ND_CPU_VARIANT=cortex-a75 HOST_ARCH=x86_64 HOST_2ND_ARCH=x86 HOST_OS=linux HOST_OS_EXTRA=Linux-5.4.0-58-generic-x86_64-Ubuntu-20.04.1-LTS HOST_CROSS_OS=windows HOST_CROSS_ARCH=x86 HOST_CROSS_2ND_ARCH=x86_64 HOST_BUILD_TYPE=release BUILD_ID=PQ3A.190801.002 OUT_DIR=/home/mardy/projects/port/halium/out PRODUCT_SOONG_NAMESPACES= hardware/qcom/audio-caf/sm8150 hardware/qcom/display-caf/sm8150 hardware/qcom/media-caf/sm8150 ============================================
Building the kernel
The configuration needs to be adapted for Halium. Locating the kernel config:
$ grep "TARGET_KERNEL_CONFIG" device/xiaomi/violet/BoardConfig.mk TARGET_KERNEL_CONFIG := vendor/violet-perf_defconfig $
Getting the Mer checker tool and running it:
git clone https://github.com/mer-hybris/mer-kernel-check cd mer-kernel-check ./mer_verify_kernel_config ../kernel/xiaomi/violet/arch/arm64/configs/vendor/violet-perf_defconfig
Prints lots of warnings, starting with:
WARNING: kernel version missing, some reports maybe misleading
To help the tool, we need to let him know the kernel version. It can be seen at the beginning of the kernel Makefile, located in ../kernel/xiaomi/violet/Makefile
; in my case it was
VERSION = 4 PATCHLEVEL = 14 SUBLEVEL = 83 EXTRAVERSION =
So I edited the configuration file ../kernel/xiaomi/violet/arch/arm64/configs/vendor/violet-perf_defconfig
and added this line at the beginning:
# Version 4.14.83
then ran the checker tool again. This time the output was a long list of kernel options that needed to be fixed, but as I went asking for some explanation in the Telegram channel for Ubuntu Touch porters, I was told that I could/should skip this test and instead use the check-kernel-config
tool from Halium. I downloaded it, made it executable (chmod +x check-kernel-config
) and ran it:
./check-kernel-config kernel/xiaomi/violet/arch/arm64/configs/vendor/violet-perf_defconfig [...lots of red and green lines...] Config file checked, found 288 errors that I did not fix.
I ran it again with the -w
option, and it reportedly fixed 287 errors. Weird, does that mean that an error was still left? I ran the tool again (without -w
), and it reported 2 errors. Ran it once more in write mode, and if fixed them. So, one might need to run it twice.
Next, added the line
BOARD_KERNEL_CMDLINE += console=tty0
in device/xiaomi/violet/BoardConfig.mk
. One more thing that needs to be done before starting the build is fixing the mount points, so I opened device/xiaomi/violet/rootdir/etc/fstab.qcom
and changed the type of the userdata partition from f2fs
to ext4
. There were no lines with the context
option, so that was the only change I needed to do.
At this point, while browsing throught the documentation, I found a link to this page which contains some notes on Halium 9 porting, which are substantially different from the official porting guide in halium.org (which I was already told in Telegram to be obsolete in several points).
So, following the instructions from this new link I ran
hybris-patches/apply-patches.sh --mb
which completed successfully.
Then, continuing following the points from this page, I edited device/xiaomi/violet/lineage_violet.mk
, commented out the lines
$(call inherit-product, $(SRC_TARGET_DIR)/product/full_base_telephony.mk) # ... $(call inherit-product, vendor/lineage/config/common_full_phone.mk)
and replaced the first one with a similar line pointing to halium.mk
. For the record, a find
revealed that the SRC_TARGET_DIR
variable in my case was build/make/target/
. It contained also the halium.mk
file, which was created by the hybris patches before. As for removing the Java dependencies, I cound't find any modules similar to those listed in this commit in any of the makefiles in my source tree, so I just started the build:
source build/envsetup.sh && breakfast violet make halium-boot
This failed the first time with the compiler being killed (out of memory, most likely), but it succeeded on the second run. There was a suspicious warning, though:
drivers/input/touchscreen/Kconfig:1290:warning: multi-line strings not supported
And indeed my kernel/xiaomi/violet/drivers/input/touchscreen/Kconfig
had this line:
source "drivers/input/touchscreen/ft8719_touch_f7b/Kconfig
(notice the unterminated string quote). I don't know if this had any impact on the build, but just to be on the safe side I added the missing quote and rebuilt.
Building the system image
Running
make systemimage
failed pretty soon:
ninja: error: '/home/mardy/projects/port/halium/out/soong/host/linux-x86/framework/turbine.jar', needed by '/home/mardy/projects/port/halium/out/soong/.intermediates/libcore/core-oj/android_common/turbine/core-oj.jar', missing and no known rule to make it
The error is due to all the Java-related stuff that I should have disabled but couldn't find. So, I tried to have a look at the changes made on another xiaomi device (lavender
, the Redmi note 7, which might not be that different, I thought) and started editing device/xiaomi/violet/device.mk
and removing a couple of Android packages. Eventually the build proceeded, just to stop at a python error:
File "build/make/tools/check_radio_versions.py", line 56 print "*** Error opening \"%s.sha1\"; can't verify %s" % (fn, key) ^ SyntaxError: invalid syntax
Yes, it's the python3 vs python2 issue, since in my system python
is python version 3. In order to fix it, I created a virtual environment:
virtualenv --python 2.7 ../python27 # adjust the path to your prefs source ../python27/bin/activate
Remember that the second line must be run every time you'll need to setup the Halium build environment (that is, every time you run breakfast
).
The build then proceeded for several minutes, until it failed due to some unresolved symbols:
prebuilts/gcc/linux-x86/aarch64/aarch64-linux-android-4.9/aarch64-linux-android/bin/ld.gold: error: /home/mardy/projects/port/halium/out/target/product/violet/obj/STATIC_LIBRARIES/lib_driver_cmd_qcwcn_intermediates/lib_driver_cmd_qcwcn.a: member at 7694 is not an ELF object external/wpa_supplicant_8/hostapd/src/drivers/driver_nl80211.c:7936: error: undefined reference to 'wpa_driver_set_p2p_ps' /home/mardy/projects/port/halium/out/target/product/violet/obj/EXECUTABLES/hostapd_intermediates/src/drivers/driver_nl80211.o:driver_nl80211.c:wpa_driver_nl80211_ops: error: undefined reference to 'wpa_driver_set_ap_wps_p2p_ie' /home/mardy/projects/port/halium/out/target/product/violet/obj/EXECUTABLES/hostapd_intermediates/src/drivers/driver_nl80211.o:driver_nl80211.c:wpa_driver_nl80211_ops: error: undefined reference to 'wpa_driver_get_p2p_noa' /home/mardy/projects/port/halium/out/target/product/violet/obj/EXECUTABLES/hostapd_intermediates/src/drivers/driver_nl80211.o:driver_nl80211.c:wpa_driver_nl80211_ops: error: undefined reference to 'wpa_driver_set_p2p_noa' /home/mardy/projects/port/halium/out/target/product/violet/obj/EXECUTABLES/hostapd_intermediates/src/drivers/driver_nl80211.o:driver_nl80211.c:wpa_driver_nl80211_ops: error: undefined reference to 'wpa_driver_nl80211_driver_cmd'
As people told me in the Telegram channel, this driver is not used since "our wpa_supplicant
talks to kernel directly". OK, so I simply disabled the driver, copying again from the lavender
Halium changes: commented out the BOARD_WLAN_DEVICE := qcwcn
line (and all lines referring this variable) from device/xiaomi/violet/BoardConfig.mk
and ran make systemimage
again. This time, surprisingly, it all worked.
Try it out
I first tried to flash the kernel only. I rebooted into fastboot, and on my PC ran this:
cout fastboot flash boot halium-boot.img
I then rebooted my phone, but after showing the boot logo for a few seconds it would jump to the fastboot screen. Indeed, flashing the previous kernel would restore the normal boot, so there had to be something wrong with my own kernel.
While looking at what the problem could be, I noticed that in the lavender
port the author did not modify the lineageos kernel config file in place, but instead created a new one and changed the BoardConfig.mk
file to point to his new copy. Since it sounded like a good idea, I did the same and created kernel/xiaomi/violet/arch/arm64/configs/vendor/violet_halium_defconfig
for the Halium changes. And then the line in the board config file became
TARGET_KERNEL_CONFIG := vendor/violet_halium_defconfig
I then continued investigating the boot issue, and I was told that it might have been due to an Android option, skip_initramfs
, which is set by the bootloader and causes our Halium boot to fail. The fix is to just disable this option in the kernel, by editing init/initramfs.c
and change the skip_initramfs_param
function to always set the do_skip_initramfs
variable to 0
, rather than to 1
. After doing this, the boot proceeded to show the Ubuntu splash screen with the five dots being lit, but it didn't proceed from there.
Setting up the udev rules
Even in this state, the device was detected by my host PC and these lines appeared in the system log:
kernel: usb 1-3: new high-speed USB device number 26 using xhci_hcd kernel: usb 1-3: New USB device found, idVendor=0fce, idProduct=7169, bcdDevice= 4.14 kernel: usb 1-3: New USB device strings: Mfr=1, Product=2, SerialNumber=3 kernel: usb 1-3: Product: Unknown kernel: usb 1-3: Manufacturer: GNU/Linux Device kernel: usb 1-3: SerialNumber: GNU/Linux Device on usb0 10.15.19.82
Indeed, I guess the reason I could do this without even flashing my systemimage is because I had first flashed another UT system image from another porter. I'm not sure if I'd had the same results with my own image. Anyway, the USB networking was there, so I connected and ran the following commands to generate the udev rules:
ssh phablet@10.15.19.82 # used 0000 as password sudo -i # same password again cd /home/phablet DEVICE=violet cat /var/lib/lxc/android/rootfs/ueventd*.rc /vendor/ueventd*.rc \ | grep ^/dev \ | sed -e 's/^\/dev\///' \ | awk '{printf "ACTION==\"add\", KERNEL==\"%s\", OWNER=\"%s\", GROUP=\"%s\", MODE=\"%s\"\n",$1,$3,$4,$2}' \ | sed -e 's/\r//' \ >70-$DEVICE.rules
I then copied (with scp
) this file to my host PC, and I moved it to device/xiaomi/violet/ubuntu/70-violet.rules
(I had to create the ubuntu
directory first). Then I edited the device/xiaomi/violet/device.mk
file and added these lines at the end:
### Ubuntu Touch ### PRODUCT_COPY_FILES += \ $(LOCAL_PATH)/ubuntu/70-violet.rules:system/halium/lib/udev/rules.d/70-android.rules ### End Ubuntu Touch ###
It was now time to try my own system image. I rebooted my device into TWRP, I cloned the halium-install repository into my halium build dir, downloaded a rootfs for Halium9, and ran
./halium-install/halium-install -p ut \ ~/Downloads/ubuntu-touch-android9-arm64.tar.gz \ out/target/product/violet/system.img
The first time this failed because the simg2img
tool was not installed. The second time it proceeded to create the image, asked me for a password for the phablet user (gave "0000") and pushed the image onto the device, into the /data/
partition. I then rebooted.
My first Ubuntu Touch image
Upon reboot, the usual splash screen appeared, followed by several seconds of black screen. It definitely didn't look right, but at least it proved that something had been flashed. After some more seconds, to my big surprise, the Ubuntu boot screen appeared, just with a smaller logo than how it used to be before, which also confimed that my system image was being used - in fact, I did not adjust the GRID_UNIT_PX
variable before. Since this was an easy fix, I chose to focus on that, rather than fix the boot issues (indeed, my device did not move on from the Ubuntu boot screen). SSH was working.
I took the scaling.conf file from the lavender changes, put it in device/xiaomi/violet/ubuntu/
and added this line in the PRODUCT_COPY_FILES
in device.mk
:
$(LOCAL_PATH)/ubuntu/scaling.conf:system/halium/etc/ubuntu-touch-session.d/android.conf
I initially used system/ubuntu/etc/...
as the target destination for the config file, like in the lavender commit, but this didn't work out for me. Then I changed the path to start with system/halium/
, like it's mentioned in the ubports documentation, but it apparently had no effect either.
After a couple of days spent trying to understand why my files were not appearing under /etc
, it turned out that the device was not using my system image at all: with halium-install
I had my image installed in /userdata/system.img
, while the correct path for Halium 9 devices is /userdata/android-rootfs.img
. I was told that the option -s
of halium-install
would do the trick. Instead of re-running the script, I took the shortcut of renaming /userdata/system.img
to /userdata/android-rootfs.img
and after rebooting I could see that the Ubuntu logo was displayed at the correct size. And indeed my system image was being used.
So I started to debug why unity8 didn't start. The logs terminated with this line:
terminate called after throwing an instance of 'std::runtime_error' what(): org.freedesktop.DBus.Error.NoReply: Message recipient disconnected from message bus without replying initctl: Event failed
It means, that unity8 did not handle correctly the situation where another D-Bus service crashed while handing a call from unity8. The problem now was how to figure out which service it was. I ran dbus-monitor
and restarted unity8 (initctl start unity8
), then examined the dbus logs; I saw the point where unity8 got disconnected from the bus, but before that point I didn't find any failed D-Bus calls. So it had to be the system bus. I did exactly the same steps, just this time after running dbus-monitor --system
as root, I found the place where unity8 got disconnected, and found this D-bus error shortly before that:
method call time=1605676421.968667 sender=:1.306 -> destination=com.ubuntu.biometryd.Service serial=3 path=/default_device; interface=com.ubuntu.biometryd.Device; member=Identifier method return time=1605676421.970222 sender=:1.283 -> destination=:1.306 serial=4 reply_serial=3 object path "/default_device/identifier" ... method call time=1605676421.974218 sender=:1.283 -> destination=org.freedesktop.DBus serial=6 path=/org/freedesktop/DBus; interface=org.freedesktop.DBus; member=GetConnectionAppArmorSecurityContext string ":1.306" error time=1605676421.974278 sender=org.freedesktop.DBus -> destination=:1.283 error_name=org.freedesktop.DBus.Error.AppArmorSecurityContextUnknown reply_serial=6 string "Could not determine security context for ':1.306'" ... signal time=1605676421.989074 sender=org.freedesktop.DBus -> destination=:1.283 serial=6 path=/org/freedesktop/DBus; interface=org.freedesktop.DBus; member=NameLost string "com.ubuntu.biometryd.Service" ... error time=1605676421.989302 sender=org.freedesktop.DBus -> destination=:1.306 error_name=org.freedesktop.DBus.Error.NoReply reply_serial=4 string "Message recipient disconnected from message bus without replying"
So, it looks like unity8 (:1.306
) made a request to biometryd, who asked the D-Bus daemon what was the AppArmor label of the caller, but since I didn't backport the D-Bus mediation patches for AppArmor, the label could not be resolved. biometryd
decided to crash instead of properly handling the error, and so did unity8.
So I backported the AppArmor patches. I took them from the Canonical kernel repo, but they did not apply cleanly because they are meant to be applied on top of a pristine 4.14 branch, whereas the Android kernel had already some AppArmor security fixes backported from later releases. So I set and quickly inspected the contents of each patch, and found out that a couple of them had already been applied, and the last patch had to be applied as the first (yeah, it's hard to explain, but it all depends on other patches that Android has backported from newer kernels). Anyway, after rebuilding the kernel and reflashing it, my phone could finally boot to unity8!
Conclusion (of the first episode)
The actual porting, as I've been told, starts here. What has been documented here are only the very first steps of the bring-up; what awaits me now is to make all the hardware subsystems work properly, and this, according to people more experienced in porting, is the harder part.
So far, very few things work, to the point that it's faster to me to list the things that do work; it's safe to assume that all what is not listed here is not working:
- Mir, with graphics and input: Unity8 starts and is usable
- Camera: can take photos; video recording start but an error appears when the stop button is pressed
- Flashlight
- Fingerprint reader: surprisingly, this worked out of the box
- Screen brightness (though it can be changed only manually)
For all the rest, please keep an eye on this blog: I'll write, when I make some progress!
17 Jan 2021 7:15pm GMT
17 Dec 2020
Planet Maemo
Avast, Qt6 announcing new QPromise and QFuture APIs
Qt published its New_Features in Qt 6.0.
Some noteworthy items in their list:
- QPromise allows setting values, progress and exceptions to QFuture
- QFuture supports attaching continuations
I like to think I had my pirate-hook in it at least a little bit with QTBUG-61928.
17 Dec 2020 9:29pm GMT
06 Dec 2020
Planet Maemo
How to generate random points on a sphere
This question often pops up, when you need a random direction vector to place things in 3D or you want to do a particle simulation.
We recall that a 3D unit-sphere (and hence a direction) is parametrized only by two variables; elevation \theta \in [0; \pi] and azimuth \varphi \in [0; 2\,\pi] which can be converted to Cartesian coordinates as
\begin{aligned} x &= \sin\theta \, \cos\varphi \\ y &= \sin\theta \, \sin\varphi \\ z &= \cos\theta \end{aligned}
If one takes the easy way and uniformly samples this parametrization in numpy like
phi = np.random.rand() * 2 * np.pi
theta = np.random.rand() * np.pi
One (i.e. you as you are reading this) ends with something like this:
While the 2D surface of polar coordinates uniformly sampled (left), we observe a bias of sampling density towards the poles when projecting to the Cartesian coordinates (right).
The issue is that the cos mapping of the elevation has an uneven step size in Cartesian space, as you can easily verify: cos^{'}(x) = sin(x).
The solution is to simply sample the elevation in the Cartesian space instead of the spherical space - i.e. sampling z \in [-1; 1]. From that we can get back to our elevation as \theta = \arccos z:
z = 1 - np.random.rand() * 2 # convert rand() range 0..1 to -1..1
theta = np.arccos(z)
As desired, this compensates the spherical coordinates such that we end up with uniform sampling in the Cartesian space:
Custom opening angle
If you want to further restrict the opening angle instead of sampling the full sphere you can also easily extend the above. Here, you must re-map the cos values from [1; -1] to [0; 2] as
cart_range = -np.cos(angle) + 1 # maximal range in cartesian coords
z = 1 - np.random.rand() * cart_range
theta = np.arccos(z)
Optimized computation
If you do not actually need the parameters \theta, \varphi, you can spare some trigonometric functions by using \sin \theta = \sqrt { 1 - z^2} as
\begin{aligned} x &= \sqrt { 1 - z^2} \, \cos\varphi \\ y &= \sqrt { 1 - z^2} \, \sin\varphi \end{aligned} 0 0
06 Dec 2020 1:12am GMT
18 Nov 2020
Planet Maemo
Self-built NAS for Nextcloud hosting
With Google cutting its unlimited storage and ending the Play Music service, I decided to use my own Nextcloud more seriously.
In part because Google forced all its competitors out of the market, but mostly because I want to be independent of any cloudy services.
The main drawback of my existing Nextcloud setup, that I have written about here, was missing redundancy; the nice thing about putting your stuff in the cloud is that you do not notice if one of the storage devices fails - Google will take care of providing you with a backup copy of your data.
Unfortunately, the Intel NUC based build I used, while offering great power efficiency did not support adding a second HDD to create a fail-safe RAID1 setup. Therefore I had to upgrade.
As I still wanted to keep things power-efficient in a small form-factor, my choice fell on the ASRock DeskMini series. Here, I went with the AMD Variant (A300) in order to avoid paying the toll of spectre mitigations with Intel (resulting in just 80% of baseline performance).
Size comparison between the A300 and a Gigabyte BRIX
The photos above show you the size difference, which is considerable - yet necessary to cram two 2.5″ SATA drives next to each other.
Here, keep in mind that while the NUC devices have their CPU soldered on, we are getting the standardized STX form-factor with the A300, which means you can replace and upgrade the mainboard and the CPU as you wish, while with a NUC you are basically stuck with what you bought initially.
The full config of the build is as follows and totals at about 270€
- ASRock DeskMini A300
- AMD Athlon 3000G
- 8GB Crucial DDR4-2666 RAM
- WD Red SA500 NAS 500GB
- Crucial MX500 500GB
Note, that I deliberately chose SSDs from different vendors to reduce the risk of simultaneous failure.
Also, while the 3000G is not the fastest AMD CPU, it is sufficient to host nextcloud and is still a nice upgrade from the Intel Celeron I used previously.
Furthermore, its 35W TDP nicely fits the constrained cooling options. Note, that you can limit for Ryzen 3/5 CPUs to 35W in the BIOS as well, so there is not need to get their GE variants.
However, for a private server you probably do not need that CPU power anyway, so just go with the Athlon 3000G for half the price.
Unfortunately, the A300 system is not designed for passive cooling and comes with a quite annoying CPU fan. To me the fan coming with the Athlon 3000G was less annoying, so I used that instead.
Anyway, you should set the fan RPM to 0% below 50° C in the BIOS, which results in 800 RPM and is unhearable while keeping the CPU reasonably cool.
Power Consumption
As the machine will run 24/7, power consumption is an important factor. As a rule of thumb you can calculate with 1€ for 1 Watt drained non-stop for 1 Year.
The 35W TDP gives us a upper limit of what the system will consume on persistent load - however the more interesting measure is the idle consuption as thats the state the system will be most of the time.
As I already tried some builds with different architectures, we have some interesting values to compare to, putting the A300 build in perspective
Build | CPU | Idle | Load |
---|---|---|---|
Odroid U3 | Exynos4412 | 3.7 W | 9 W |
Gigabyte BRIX | Intel N3350 | 4.5 W | 9.6 W |
A300 | Athlon 3000G | 7.3 W | 33.6 W |
While you can obviously push the system towards 35W by with multiple simultaneous users, the 7.3 W idle consumption is quite nice.
Keep in mind, that the A300 was measured with two SATA drives operating as RAID1. If you only use one you can subtract 1W - at which point it is only 1.5 W away from the considerably weaker NUC system.
You might now wonder, whether the load or the idle measure is closer to the typical consumption. For this I measured the consumption for one day, which totalled at 0.18 kWh - or 7.5 Watts.
Power optimizations
To reach that 7.3 W idle, you need to tune some settings though. The most important one and luckily the easiest to fix is using a recent kernel.
If you are on Ubuntu 18.04, update to 20.04 or install the hwe kernel (5.4.0) - it saves you 4 Watts (11.3 to 7.3).
For saving about 0.5 watts, you can downgrade the network interface from 1Gbit to 100Mbit by executing
ethtool -s enp2s0 speed 100 duplex full autoneg on
Additionally, you can use Intels powertop to tune your system settings for power saving as
powertop --auto-tune
18 Nov 2020 8:16pm GMT
12 Nov 2020
Planet Maemo
Imaginario 0.10
Today I released Imaginario 0.10. No bigger changes there, but two important bugfixes.
12 Nov 2020 4:12pm GMT
16 Oct 2020
Planet Maemo
Figuring out corrupt stacktraces on ARM
If you're developing C/C++ on embedded devices, you might already have stumbled upon a corrupt stacktrace like this when trying to debug with gdb:
(gdb) bt #0 0xb38e32c4 in pthread_getname_np () from /home/enrique/buildroot/output5/staging/lib/libpthread.so.0 #1 0xb38e103c in __lll_timedlock_wait () from /home/enrique/buildroot/output5/staging/lib/libpthread.so.0 Backtrace stopped: previous frame identical to this frame (corrupt stack?)
In these cases I usually give up gdb and try to solve my problems by adding printf()s and resorting to other tools. However, there are times when you really really need to know what is in that cursed stack.
ARM devices subroutine calls work by setting the return address in the Link Register (LR), so the subroutine knows where to point the Program Counter (PC) register to. While not jumping into subroutines, the values of the LR register is saved in the stack (to be restored later, right before the current subroutine returns to the caller) and the register can be used for other tasks (LR is a "scratch register"). This means that the functions in the backtrace are actually there, in the stack, in the form of older saved LRs, waiting for us to get them.
So, the first step would be to dump the memory contents of the backtrace, starting from the address pointed by the Stack Pointer (SP). Let's print the first 256 32-bit words and save them as a file from gdb:
(gdb) set logging overwrite on (gdb) set logging file /tmp/bt.txt (gdb) set logging on Copying output to /tmp/bt.txt. (gdb) x/256wa $sp 0xbe9772b0: 0x821e 0xb38e103d 0x1aef48 0xb1973df0 0xbe9772c0: 0x73d 0xb38dc51f 0x0 0x1 0xbe9772d0: 0x191d58 0x191da4 0x19f200 0xb31ae5ed ... 0xbe977560: 0xb28c6000 0xbe9776b4 0x5 0x10871 <main(int, char**)> 0xbe977570: 0xb6f93000 0xaaaaaaab 0xaf85fd4a 0xa36dbc17 0xbe977580: 0x130 0x0 0x109b9 <__libc_csu_init> 0x0 ... 0xbe977690: 0x0 0x0 0x108cd <_start> 0x0 0xbe9776a0: 0x0 0x108ed <_start+32> 0x10a19 <__libc_csu_fini> 0xb6f76969 (gdb) set logging off Done logging to /tmp/bt.txt.
Gdb already can name some of the functions (like main()
), but not all of them. At least not the ones more interesting for our purpose. We'll have to look for them by hand.
We first get the memory page mapping from the process (WebKit's WebProcess in my case) looking in /proc/pid/maps. I'm retrieving it from the device (named metro
) via ssh and saving it to a local file. I'm only interested in the code pages, those with executable ('x') permissions:
$ ssh metro 'cat /proc/$(ps axu | grep WebProcess | grep -v grep | { read _ P _ ; echo $P ; })/maps | grep " r.x. "' > /tmp/maps.txt
The file looks like this:
00010000-00011000 r-xp 00000000 103:04 2617 /usr/bin/WPEWebProcess ... b54f2000-b6e1e000 r-xp 00000000 103:04 1963 /usr/lib/libWPEWebKit-0.1.so.2.2.1 b6f6b000-b6f82000 r-xp 00000000 00:02 816 /lib/ld-2.24.so be957000-be978000 rwxp 00000000 00:00 0 [stack] be979000-be97a000 r-xp 00000000 00:00 0 [sigpage] be97b000-be97c000 r-xp 00000000 00:00 0 [vdso] ffff0000-ffff1000 r-xp 00000000 00:00 0 [vectors]
Now we process the backtrace to remove address markers and have one word per line:
$ cat /tmp/bt.txt | sed -e 's/^[^:]*://' -e 's/[<][^>]*[>]//g' | while read A B C D; do echo $A; echo $B; echo $C; echo $D; done | sed 's/^0x//' | while read P; do printf '%08x\n' "$((16#"$P"))"; done | sponge /tmp/bt.txt
Then merge and sort both files, so the addresses in the stack appear below their corresponding mappings:
$ cat /tmp/maps.txt /tmp/bt.txt | sort > /tmp/merged.txt
Now we process the resulting file to get each address in the stack with its corresponding mapping:
$ cat /tmp/merged.txt | while read LINE; do if [[ $LINE =~ - ]]; then MAPPING="$LINE"; else echo $LINE '-->' $MAPPING; fi; done | grep '/' | sed -E -e 's/([0-9a-f][0-9a-f]*)-([0-9a-f][0-9a-f]*)/\1 - \2/' > /tmp/mapped.txt
Like this (address in the stack, page start (or base), page end, page permissions, executable file load offset (base offset), etc.):
0001034c --> 00010000 - 00011000 r-xp 00000000 103:04 2617 /usr/bin/WPEWebProcess ... b550bfa4 --> b54f2000 - b6e1e000 r-xp 00000000 103:04 1963 /usr/lib/libWPEWebKit-0.1.so.2.2.1 b5937445 --> b54f2000 - b6e1e000 r-xp 00000000 103:04 1963 /usr/lib/libWPEWebKit-0.1.so.2.2.1 b5fb0319 --> b54f2000 - b6e1e000 r-xp 00000000 103:04 1963 /usr/lib/libWPEWebKit-0.1.so.2.2.1 ...
The addr2line
tool can give us the exact function an address belongs to, or even the function and source code line if the code has been built with symbols. But the addresses addr2line
understands are internal offsets, not absolute memory addresses. We can convert the addresses in the stack to offsets with this expression:
offset = address - page start + base offset
I'm using buildroot as my cross-build environment, so I need to pick the library files from the staging directory because those are the unstripped versions. The addr2line
tool is the one from the buldroot cross compiling toolchain. Written as a script:
$ cat /tmp/mapped.txt | while read ADDR _ BASE _ END _ BASEOFFSET _ _ FILE; do OFFSET=$(printf "%08x\n" $((0x$ADDR - 0x$BASE + 0x$BASEOFFSET))); FILE=~/buildroot/output/staging/$FILE; if [[ -f $FILE ]]; then LINE=$(~/buildroot/output/host/usr/bin/arm-buildroot-linux-gnueabihf-addr2line -p -f -C -e $FILE $OFFSET); echo "$ADDR $LINE"; fi; done > /tmp/addr2line.txt
Finally, we filter out the useless [??]
entries:
$ cat /tmp/bt.txt | while read DATA; do cat /tmp/addr2line.txt | grep "$DATA"; done | grep -v '[?][?]' > /tmp/fullbt.txt
What remains is something very similar to what the real backtrace should have been if everything had originally worked as it should in gdb:
b31ae5ed gst_pad_send_event_unchecked en /home/enrique/buildroot/output5/build/gstreamer1-1.10.4/gst/gstpad.c:5571 b31a46c1 gst_debug_log en /home/enrique/buildroot/output5/build/gstreamer1-1.10.4/gst/gstinfo.c:444 b31b7ead gst_pad_send_event en /home/enrique/buildroot/output5/build/gstreamer1-1.10.4/gst/gstpad.c:5775 b666250d WebCore::AppendPipeline::injectProtectionEventIfPending() en /home/enrique/buildroot/output5/build/wpewebkit-custom/build-Release/../Source/WebCore/platform/graphics/gstreamer/mse/AppendPipeline.cpp:1360 b657b411 WTF::GRefPtr<_GstEvent>::~GRefPtr() en /home/enrique/buildroot/output5/build/wpewebkit-custom/build-Release/DerivedSources/ForwardingHeaders/wtf/glib/GRefPtr.h:76 b5fb0319 WebCore::HTMLMediaElement::pendingActionTimerFired() en /home/enrique/buildroot/output5/build/wpewebkit-custom/build-Release/../Source/WebCore/html/HTMLMediaElement.cpp:1179 b61a524d WebCore::ThreadTimers::sharedTimerFiredInternal() en /home/enrique/buildroot/output5/build/wpewebkit-custom/build-Release/../Source/WebCore/platform/ThreadTimers.cpp:120 b61a5291 WTF::Function<void ()>::CallableWrapper<WebCore::ThreadTimers::setSharedTimer(WebCore::SharedTimer*)::{lambda()#1}>::call() en /home/enrique/buildroot/output5/build/wpewebkit-custom/build-Release/DerivedSources/ForwardingHeaders/wtf/Function.h:101 b6c809a3 operator() en /home/enrique/buildroot/output5/build/wpewebkit-custom/build-Release/../Source/WTF/wtf/glib/RunLoopGLib.cpp:171 b6c80991 WTF::RunLoop::TimerBase::TimerBase(WTF::RunLoop&)::{lambda(void*)#1}::_FUN(void*) en /home/enrique/buildroot/output5/build/wpewebkit-custom/build-Release/../Source/WTF/wtf/glib/RunLoopGLib.cpp:164 b6c80991 WTF::RunLoop::TimerBase::TimerBase(WTF::RunLoop&)::{lambda(void*)#1}::_FUN(void*) en /home/enrique/buildroot/output5/build/wpewebkit-custom/build-Release/../Source/WTF/wtf/glib/RunLoopGLib.cpp:164 b2ad4223 g_main_context_dispatch en :? b6c80601 WTF::{lambda(_GSource*, int (*)(void*), void*)#1}::_FUN(_GSource*, int (*)(void*), void*) en /home/enrique/buildroot/output5/build/wpewebkit-custom/build-Release/../Source/WTF/wtf/glib/RunLoopGLib.cpp:40 b6c80991 WTF::RunLoop::TimerBase::TimerBase(WTF::RunLoop&)::{lambda(void*)#1}::_FUN(void*) en /home/enrique/buildroot/output5/build/wpewebkit-custom/build-Release/../Source/WTF/wtf/glib/RunLoopGLib.cpp:164 b6c80991 WTF::RunLoop::TimerBase::TimerBase(WTF::RunLoop&)::{lambda(void*)#1}::_FUN(void*) en /home/enrique/buildroot/output5/build/wpewebkit-custom/build-Release/../Source/WTF/wtf/glib/RunLoopGLib.cpp:164 b2adfc49 g_poll en :? b2ad44b7 g_main_context_iterate.isra.29 en :? b2ad477d g_main_loop_run en :? b6c80de3 WTF::RunLoop::run() en /home/enrique/buildroot/output5/build/wpewebkit-custom/build-Release/../Source/WTF/wtf/glib/RunLoopGLib.cpp:97 b6c654ed WTF::RunLoop::dispatch(WTF::Function<void ()>&&) en /home/enrique/buildroot/output5/build/wpewebkit-custom/build-Release/../Source/WTF/wtf/RunLoop.cpp:128 b5937445 int WebKit::ChildProcessMain<WebKit::WebProcess, WebKit::WebProcessMain>(int, char**) en /home/enrique/buildroot/output5/build/wpewebkit-custom/build-Release/../Source/WebKit/Shared/unix/ChildProcessMain.h:64 b27b2978 __bss_start en :?
I hope you find this trick useful and the scripts handy in case you ever to resort to examining the raw stack to get a meaningful backtrace.
Happy debugging!
16 Oct 2020 6:07pm GMT
02 Aug 2020
Planet Maemo
Mi Band 5 Review / Mi Band Evolution
Xiaomi has recently released the new Mi Band 5. Since I have owned the each band starting with the Mi Band 2, I think it is time to look back and see where the Mi Band has gone in the recent years.
Actually, the Mi Band story started ahead of the Apple Watch in 2014 with the Mi Band 1, which was a pure fitness-tracking device without a display and even without a heart-beat sensor. This made the device not very appealing to me - even thought it already offered sleep monitoring.
It also already had that interchangeable wrist-bands that allow you to customize the look to your liking. The Mi Band 2 you see in the images uses a custom steel wrist-band as the original one broke after some years of usage.
Below you see a comparison of the Mi Bands, regarding the features that are most significant from my perspective
Mi Band 2
- Released 2016
- Clock
- Heartbeat
- Notifications
Mi Band 3
- Released 2018
- Clock
- Heartbeat
- Notifications
- Timer
- Weather
- Workouts
Mi Band 4
- Released 2019
- Clock
- Heartbeat
- Notifications
- Timer
- Weather
- Workouts
- Music control
Mi Band 5
- Released 2020
- Clock
- Heartbeat
- Notifications
- Timer
- Weather
- Workouts
- Music control
- Cam shutter
The first thing to note is probably that Xiaomi accelerated the release cycle from 2 years between the Bands 1, 2 and 3 to 1 year between Band 4 and 5. We will come back to this when talking about the Mi Band 5.
Screen legibility comparison
Lets start the comparison with the screen, which is the most obvious part and the one you will probably interact with the most.
Here, the most significant property is neither size nor resolution, but rather legibility in sunlight. For comparison, I set up a little benchmark as follows:
- set the Bands to maximal brightness (as applicable)
- legibility in direct sunlight on a bright sunny day as a worst-case
- legibility in shade/ with overcast condition as a more common scenario
You can find the results below. Also see the banner image for how the screens look indoors.
First I should note that the camera does not do justice to the Mi Band 2 & 3 as their displays are scanline-based and the fast shutter can not capture the whole screen being lit at once. Therefore you only see the top part of the Mi Band 2 and the right part of the Mi Band 3 on the overcast picture.
Nevertheless, one actually cannot read the Mi Band 2 in direct sunlight and only can barely read it in the shade. The other Bands are well readable in the shade. However, I would say that only the MiBand 5 is well readable in direct sunlight.
Next, we will look at how the information is presented. The screen size continuously increased from 0.78″ on the Mi Band 3 to 0.95″ on the Mi Band 4 (+22%) to 1.1″ on the Mi Band 5 (+16%).
As you can read the time on all of them, we will look at an app to find out whether it makes any difference in practice. Here, I picked the weather app as it is probably useful to the majority of the readers.
Looking at the Mi Band 4, it did not really take advantage of the larger screen-estate and shows virtually the same information as the Mi Band 3 - only adding the location info.
The Mi Band 5 on the other hand uses the extra space to show the rain probability. It generally displays more info like the wind strength and the current UV level - however you have scroll down for them.
The Mi Band 2 does not support weather and is thus turned off.
Apps/ on band Screens
Lets also briefly look at the other apps. The images were captured on the Mi Band 5 - however unless otherwise stated the look exactly the same on the Mi Band 4.
The music control app
Outdoor running workout app
Configurable watch face (Mi Band 5 only)
Cycles app (Mi Band 5 only)
Charging
The Mi Band 5 is the first band, with a magnetically attachable charger - hence you do not have to take the band out for charging. This convenience comes at the price of a reduced battery-life from about 20 days with the Mi Band 4 to only 14 days with the Mi Band 5.
As for compatibility, you can charge the Mi Band 2 with the Mi Band 3 charger - the other way round is not possible as the Mi Band 3 is too large for the older charger.
Even though, the Mi Band 4 & 5 have their charging pins at the same location, the chargers are not compatible as the Mi Band 4 lacks the magnetic hold and the Mi Band 5 is too large for the old charger.
The Mi-Fit app
For the Mi Band the accompanying app is quite important as it is the only way to view your sleep data and to monitor your weekly/ monthly stats.
First, lets take a look how you can customize the different Bands from the app. Here, we should note that all bands are still supported by the app.
On the Mi Band 2 only the core options were available
The screens on Mi Band 3 and newer are more customizable
With the Mi Band 5, you can also select the side screens
With the Mi Band, there is only a predefined set of screens/ apps out of which you can pick the ones you want. This is probably the largest difference to a real smart-watch, where you can install additional apps from a store.
With the Mi Band 2, the whole set fits on half a screen and you can only enable/ disable the items.
With the other Bands you can additionally re-order the items, which is quite useful as it allows to choose which item appears first when you swipe up or down on the home screen.
On the Mi Band 5, you can additionally configure which app appears when you swipe left/ and right. This is hard-coded to Music Control (and Ali Pay on the CN version) with the Mi Band 4.
So the basic things work. Lets look at some peculiarities of Mi Fit next.
The sports stats are quite useful - if only the measurement was correct
Be prepared for in-app ads for Xiaomi products
First you see the workout view for outdoor running, which displays some useful stats like your pace per km and the continuously measured heart-beat rate over time.
What you do not immediately see is that the app only counted ~7.3 km, while my running distance is actually 10 km, which I have verified on google-maps.
One might now think that this is due to imprecise measuring of the band - however on the activity overview, where the daily steps are counted, the running activity is correctly accounted as 10.1 km - which is impressively accurate, given that it only counted the steps.
So the error is only present in the workout app, which is still quite annoying as it also provides the live view during a run.
If someone from Xiaomi is reading this: the error factor of ~0.73 is suspiciously close to the km to miles conversion factor of 0.625.
The error is present with both the Mi Band 4 and Mi Band 5, so I guess it is actually in the App, where I already reported it several times.
If you want happy customers, you better fix this. Many other reviews actually blame this on the band!
Addendum: with the firmware update to v1.0.1.32, the band now measures ~9km which reduces the error factor to 0.9. We are getting there.
So having talked about the bad, lets continue with the ugly. The second screenshot shows you an in-app ad for some obscure Xiaomi product on the home-screen.
These do not show up too often and currently only advertise their own products. However, this is definitely the wrong path you are on.
Ultimately, this leaves me with mixed feelings about Mi Fit. In the Mi Band 2 days it started as a slim and functional app. However, at some point they decided to re-write it with the cards-look and animations. This rewrite moved core views one level down in the menu hierarchy and the added animations actually make the app feel sluggish.
Now, with each Band generation new features appear and are integrated in some sub-menu of the app.
For instance, you get weather-alerts nowadays. However, they are not controlled in the general Band notification settings, but rather in the weather menu.
Therefore, I doubt I would discover them as easily if I would not have watched the app grow.
The good news is that due to the popularity of the Mi Band, there are several alternative apps to try, which I probably will do next.
Mi Band history
In the following, I give a quick outline of how the Mi Band evolved. If you only came here for the Mi Band 5 review, skip forward to the Mi Band 4 section.
The Mi Band 2 was released 2016, about a year after the first Apple Watch launched, which brought the wearable category to the mainstream.
At a price of less then 20€ the Mi Band offered most interesting wearable features to me, like heart-beat measurement, sleep monitoring, forwarding of smartphone notifications and ultimately, simply being a wristwatch.
Also it was an ideal way to try this new wearable thing without spending 350€, that Apple called out.
To my surprise the step-based distance estimation was already accurate back then - except for the actual workout mode, that is - as explained in the Mi Fit section.
Mi Band 3
The larger and brighter screen is the obvious advance of the Mi Band 3. However, the significant part is that it also became a touch-screen - whereas the Mi Band 2 only had the single touch-button. This allowed you swiping forth and back of the screens instead of just cycling through them and it also made virtual buttons possible. These are necessary for starting the stopwatch and timer, which are probably the most important additions for me with the Mi Band 3.
You could also start a selection workouts directly from the watch, instead of going though the app. However, this only included a treadmill mode, while I am interested in outdoor running - so I continued using the activity view for that.
More importantly, it added the weather app. If find this to be surprisingly useful. As with the time - even though you find the same info on your phone - having it at hand is better.
Mi Band 4
Again, the colored screen is the most obvious advance. It does not improve usability in any way though. It displays the same data as the monochrome screen of the Mi Band 3, which is probably more power-efficient. It adds a lot of bling though and is brighter and thus better legible in sunlight.
Speaking of bling, you can install third-party watch-faces now and there is a heap of faces to chose from. Take a look here to get an impression.
Turning to something useful, the touch sensor was noticeably improved. With the Mi Band 3 your swipes were sometimes confused with taps, which does not happen with the Mi Band 4 anymore.
The workout app, now finally included outdoor running, which is still broken though (see Mi App section). This makes the music control app the most important addition for me. At least on android, it works with any music player and allows skipping forward/ back and adjusting volume.
This is quite useful when you play music from your phone at a party or for controlling your Bluetooth headphones.
One can use the same wrist-bands as for the Mi Band 3. This made upgrading for me back then a no-brainer, but is also a strong reason to choose the Mi Band v4 over v5, today.
Mi Band 5
This time, there are no obvious advances and the update is rather evolutionary. It does not mean it is insignificant though as it improves the usability on many levels. If you are new to the Mi Bands, you should pick this one.
The most important one is probably the new magnetic charger. Previously you had to take the "watch" out of the wrist-band to charge, whereas you can simply attach the magnetic charger now.
Next, the screen is slightly brighter which makes a difference in direct sunlight though (see screen comparison section) and also boasts more information.
Finally, the software was also noticeably improved. The band displays generally became more configurable. E.g. the custom left/ right swipes which now give you 4 quick access screens instead of 2. Then, the built-in watch-faces now allow customizing the additional info they display. And it continues with the small things like the configurable alerts in the workouts (although the workout app itself still needs to be fixed).
Also, the selection of predefined watch-faces is vastly better then with the Mi Band 4. On the latter you have a hard time finding a watch-face that is simple and does not feature some animated comic figure screaming at you.
These changes could be provided as an update to the Mi Band 4 as well, but are - at the time of writing - exclusive to the Mi Band 5.
Disclaimer
The Mi Band 5 was provided to me free of charge by banggood.com. So if you liked this review and want to support me consider buying using the following affiliate links:
02 Aug 2020 1:37pm GMT
19 Jul 2020
Planet Maemo
Meepo Mini 2 vs. Archos SK8
Having never skateboarded before, I saw the Archos SK8 electric skateboard for about 80€ at a sale and thought why not give it a try. This got me into this whole electric skateboarding thing.
Now that I have some more time at home during the summer, I upgraded to the Meepo Mini 2 and after having driven with it more than 100km, I thought I write down my experiences with the two boards and why I should have gotten the Meepo board from the start.
The competitors
The Meepo Mini 2 and the Archos SK8 are not really competing here, which should be clear looking at their price difference. But for completeness, also take a look at the specs of these two boards:
Meepo Mini 2 | Archos SK8 | |
Max. speed | 46 km/h | 15 km/h |
Max. range | 18 km | 7 km |
Max. Weight | 136 kg | 80 kg |
Motor | 2 x 540 W | 1 x 150 W |
Battery | 144 Wh | 50.4 Wh |
Weight | 7.4 kg | 3.9 kg |
Specs comparison
Actually, you can swap the Archos SK8 by any of the unbranded "cheap" Chinese boards that share the same design as the ones sold by Oppikle and Hiriyt.
Here, you might wonder how many Watts you actually need. For this I direct you to the Wikpedia article on bicycle performance that contains some sample calculations (and the formulae) which should roughly hold for electric skateboards as well.
Similarities
Before we dig into the differences, lets first note the similarities aka. the choices I made when picking these specific boards in the first place:
First, both boards are hub-motor driven. I made this choice on purpose, as electric skateboards are not road-legal where I live and hub motors are barely noticeable to the non-practiced eye. This reduces my risk of getting fined for riding one.
However, I would probably generally recommend hub-motors over belt-driven motors nowadays as they require less maintenance (no moving parts), while offering a larger range and allowing pushing the board (belt driven block due to the gear ratio). The latter is especially nice, when you have run out of battery or if you do not want to draw any attention.
When electric skateboards were first introduced by boosted, hub-motors were vastly inferior power-wise but that has changed now.
Next, both boards are of so-called "cruiser-style". This is a size in between a regular skateboard and a long-board. They share a stiff deck and a kicktail with the former, while the use the wheels of the latter.
At this point I should note that I mainly use the boards for leisure instead of a daily commute. This means that I value versatility of the board over comfort of ride.
Here, having a kicktail is a must and rules out long-boards. It allows doing sharp turn, "wheelies" and you are more agile with the short board.
However, you do notice the quality of the pavement very clearly in your feet and being out of the skating age my ankle did hurt the first couple of rides before it got used to it.
So if you want to commute large distances, you should probably get a long-board with a flexi-deck that can cushion away most of the bumps.
Differences
Both boards are of similar length, however the Meepo Mini 2 is considerable wider and heavier. It also has a larger wheel-base.
This results in a better grip and you also feel much more stable on the board. Flipping the board around, you see that the SK8 only uses a single-hub motor while the Mini 2 has two and each of them offers more than 3x the power.
If you do not expect the power or if you enable the pro-mode without being one, the Mini 2 can easily throw you off the board when accelerating or breaking. You can tame it though by using the beginner riding mode if you need to learn how to skate first. You can set the modes for acceleration and breaking separately and I would recommend always using at least the pro mode for breaking and learning to deal with it. In case of an emergency you want to be able to stop in time.
Turning to the SK8, the acceleration is.. meh and so are the breaks - in both of the two riding-modes. The difference between them is merely that the top speed is capped at 10km/h in the low-mode.
But I must say that if you are a beginner this is sufficent; if you do not know how to ride being able to get going and to break are your two primary concerns and the SK8 does deliver here. The main drawback of the Archos SK8 is its tiny battery.
Aside: Li-Ion batteries
At this point we should probably briefly discuss Li-Ion battery technology. Mainly, the following two properties:
- As you discharge the battery its Voltage drops, which is related to power (the Watts number) as P = V \cdot I, where I is the current in amps.
So the more you discharge the battery, the fewer Watts you get out of your motors. - A full discharge (down to 0%) of the battery severely reduces its live-span. For maximal durability, you should always keep its charge between 40%-80%. Typically, the displayed range is clipped by the controller to help here.
So where does this leave us with the SK8? I did about 7 rides, fully-discharging the board (you do not want to stop after 10min, right?). And now the second riding-mode is essentially gone: when I try to accelerate the motors draw so much current, that the voltage drops below a critical level and the board turns off. Depending on what state the controller was in, I have to pair the remote again afterwards.
But it also shows in the first mode: while the board initially could get me up a slight slope, it now immediately starts beeping due to critical voltage - again the motors need more voltage then the already worn down battery can give.
Remotes & charging indicators
Having covered the drive train, lets turn to the remotes. Both Archos and Meepo use a similar pistol-grip like design, where you control the motors with the thumb switch.
As one would expect, the Meepo remote is more sophisticated and offers detailed telemetry data on a nicely readable LCD display. There you find your current speed, drive mode and board charging level as well as the max. speed of the current ride.
On the Archos remote you only find 4 LEDs. Those are used quite well though: when you turn on the remote, they show the remote charging level. As soon as the board is connected, they indicate the board charging level, which is actually the most important information you need while riding.
Similarly to the remotes, there are only 4 LEDs on the Archos Battery for the charging level, while you find a numeric LED-display on the Meepo board.
An actually noticeable feature on the Meepo Mini 2 is push-to-start; that is, you only have to push the board to turn it on - no need to bend down for flipping a switch.
Verdict
So why do I say you should go straight with the Meepo Mini 2 even as a beginner? On paper the Archos SK8 has everything it takes to be a nice beginner board.
It is really the battery that kills it. With only 5-10 rides it is simply not worth the money, no matter how cheap it is.
Looking at the price difference between the Meepo Mini 2 and the ER version that solely differ in the battery, you grasp that the battery is the crucial part in an electric skateboard. And the Archos SK8 is cheap, precisely because of the bad battery.
With the Meepo Mini 2 on the other hand you get a board that can "grow with you": as you get more confident you can bump up the riding mode to get more power. Even if you decide that skateboarding is not for you, you can sell the Mini 2 as it will retain lots of its value - in contrast to just producing electric waste with the Archos SK8.
Riding the Meepo Mini 2
The Meepo Mini 2 is specified to go up to 46 km/h. Whether you can go that fast depends on your weight, the wind and the slope (see the Wikipedia link, mentioned above). In case you are fat and/ or there are lots of slopes where you live, you might also consider the ER version of the Mini 2, which comes with doubled battery capacity. As mentioned above this not only means that you can get further, but also that you have more power in the mid-range.
How fast can you go?
Having only previous experience on a Snowboard, I am a rather cautious rider. So far my max. speed (according to the remote) was 30 km/ h which I did uphill - in hope that stopping is easier that way.
Going downhill (only using the motors for breaking), I feel comfortable until around 22 km/ h.
I typically ride for about 30-45 min and the lowest the battery got was 40%, which means it should last for quite some time.
Note, that I do not go straight uphill for 30min and that I usually push to get rolling, as this is where most energy is used.
A suitable helmet
When lifting the board, the remote showed that the ESC only limits the speed at 50 km/h. When riding a skateboard at anything above 10 km/h without a cushion-zone and no nothing, I would highly recommend you to at least wear a helmet.
However, you should consider that a "normal" skate helmet is only specified (EN1078) up to 19.5 km/h impact speed - if you ride faster it does not guarantee protection.
Fortunately, one does not have to resort to heavy motorcycle helmets (ECE2205) as there is a specification (ASTM F1952) for downhill bike helmets, which was designed with speed in mind. It is specified for an impact speed of 22.3 km/h and typically the helmets provide full-face protection.
Recently, even variants for downhill longboarding started to pop-up, which should be even a better fit style-wise.
Addendum: replaced NTA8776 by ASTM F1952 in the last paragraph
19 Jul 2020 8:58pm GMT
30 Jun 2020
Planet Maemo
Developing on WebKitGTK with Qt Creator 4.12.2
After the latest migration of WebKitGTK test bots to use the new SDK based on Flatpak, the old development environment based on jhbuild became deprecated. It can still be used with export WEBKIT_JHBUILD=1
, though, but support for this way of working will gradually fade out.
I used to work on a chroot because I love the advantages of having an isolated and self-contained environment, but an issue in the way bubblewrap manages mountpoints basically made it impossible to use the new SDK from a chroot. It was time for me to update my development environment to the new ages and have it working in my main Kubuntu 18.04 distro.
My mail goal was to have a comfortable IDE that follows standard GUI conventions (that is, no emacs nor vim) and has code indexing features that (more or less) work with the WebKit codebase. Qt Creator was providing all that to me in the old chroot environment thanks to some configuration tricks by Alicia, so it should be good for the new one.
I preferred to use the Qt Creator 4.12.2 offline installer for Linux, so I can download exactly the same version in the future in case I need it, but other platforms and versions are also available.
The WebKit source code can be downloaded as always using git:
git clone git.webkit.org/WebKit.git
It's useful to add WebKit/Tools/Scripts
and WebKit/Tools/gtk
to your PATH, as well as any other custom tools you may have. You can customize your $HOME/.bashrc
for that, but I prefer to have an env.sh
environment script to be sourced from the current shell when I want to enter into my development environment (by running webkit
). If you're going to use it too, remember to adjust to your needs the paths used there.
Even if you have a pretty recent distro, it's still interesting to have the latests Flatpak tools. Add Alex Larsson's PPA to your apt sources:
sudo add-apt-repository ppa:alexlarsson/flatpak
In order to ensure that your distro has all the packages that webkit requires and to install the WebKit SDK, you have to run these commands (I omit the full path). Downloading the Flatpak modules will take a while, but at least you won't need to build everything from scratch. You will need to do this again from time to time, every time the WebKit base dependencies change:
install-dependencies
update-webkitgtk-libs
Now just build WebKit and check that MiniBrowser works:
build-webkit --gtk run-minibrowser --gtk
I have automated the previous steps as go full-rebuild
and runtest.sh
.
This build process should have generated a WebKit/WebKitBuild/GTK/Release/compile_commands.json
file with the right parameters and paths used to build each compilation unit in the project. This file can be leveraged by Qt Creator to get the right include paths and build flags after some preprocessing to translate the paths that make sense from inside Flatpak to paths that make sense from the perspective of your main distro. I wrote compile_commands.sh
to take care of those transformations. It can be run manually or automatically when calling go full-rebuild
or go update
.
The WebKit way of managing includes is a bit weird. Most of the cpp files include config.h
and, only after that, they include the header file related to the cpp file. Those header files depend on defines declared transitively when including config.h
, but that file isn't directly included by the header file. This breaks the intuitive rule of "headers should include any other header they depend on" and, among other things, completely confuse code indexers. So, in order to give the Qt Creator code indexer a hand, the compile_commands.sh
script pre-includes WebKit.config
for every file and includes config.h
from it.
With all the needed pieces in place, it's time to import the project into Qt Creator. To do that, click File → Open File or Project, and then select the compile_commands.json
file that compile_commands.sh
should have generated in the WebKit main directory.
Now make sure that Qt Creator has the right plugins enabled in Help → About Plugins…. Specifically: GenericProjectManager, ClangCodeModel, ClassView, CppEditor, CppTools, ClangTools, TextEditor and LanguageClient (more on that later).

With this setup, after a brief initial indexing time, you will have support for features like Switch header/source (F4), Follow symbol under cursor (F2), shading of disabled if-endif blocks, auto
variable type resolving and code outline. There are some oddities of compile_commands.json
based projects, though. There are no compilation units in that file for header files, so indexing features for them only work sometimes. For instance, you can switch from a method implementation in the cpp file to its declaration in the header file, but not the opposite. Also, you won't see all the source files under the Projects view, only the compilation units, which are often just a bunch of UnifiedSource-*.cpp
files. That's why I prefer to use the File System view.
Additional features like Open Type Hierarchy (Ctrl+Shift+T) and Find References to Symbol Under Cursor (Ctrl+Shift+U) are only available when a Language Client for Language Server Protocol is configured. Fortunately, the new WebKit SDK comes with the ccls C/C++/Objective-C language server included. To configure it, open Tools → Options… → Language Client and add a new item with the following properties:
- Name: ccls
- Language:
*.c;.cpp;*.h
- Startup behaviour: Always On
- Executable:
/home/enrique/work/webkit/WebKit/Tools/Scripts/webkit-flatpak
- Arguments:
--gtk -c ccls --index=/home/enrique/work/webkit/WebKit
Some "LanguageClient ccls: Unexpectedly finished. Restarting in 5 seconds.
" errors will appear in the General Messages panel after configuring the language client and every time you launch Qt Creator. It's just ccls taking its time to index the whole source code. It's "normal", don't worry about it. Things will get stable and start to work after some minutes.
Due to the way the Locator file indexer works in Qt Creator, it can become confused, run out of memory and die if it finds cycles in the project file tree. This is common when using Flatpak and running the MiniBrowser or the tests, since /proc
and other large filesystems are accessible from inside WebKit/WebKitBuild
. To avoid that, open Tools → Options… → Environment → Locator and set Refresh interval to 0 min.
I also prefer to call my own custom build and run scripts (go
and runtest.sh
) instead of letting Qt Creator build the project with the default builders and mess everything. To do that, from the Projects mode (Ctrl+5), click on Build & Run → Desktop → Build and edit the build configuration to be like this:
- Build directory:
/home/enrique/work/webkit/WebKit
- Add build step → Custom process step
- Command:
go
(no absolute route because I have it in my PATH) - Arguments:
- Working directory:
/home/enrique/work/webkit/WebKit
- Command:
Then, for Build & Run → Desktop → Run, use these options:
- Deployment: No deploy steps
- Run:
- Run configuration: Custom Executable → Add
- Executable: runtest.sh
- Command line arguments:
- Working directory:
- Run configuration: Custom Executable → Add
With these configuration you can build the project with Ctrl+B and run it with Ctrl+R.
I think I'm not forgetting anything more regarding environment setup. With the instructions in this post you can end up with a pretty complete IDE. Here's a screenshot of it working in its full glory:
Anyway, to be honest, nothing will ever reach the level of code indexing features I got with Eclipse some years ago. I could find usages of a variable/attribute and know where it was being read, written or read-written. Unfortunately, that environment stopped working for me long ago, so Qt Creator has been the best I've managed to get for a while.
Properly configured web based indexers such as the Searchfox instance configured in Igalia can also be useful alternatives to a local setup, although they lack features such as type hierarchy.
I hope you've found this post useful in case you try to setup an environment similar to the one described here. Enjoy!
30 Jun 2020 2:47pm GMT
22 Jun 2020
Planet Maemo
PhotoTeleport 0.13
Just a quick note to let the world know that PhotoTeleport 0.13 has been released.
22 Jun 2020 8:11pm GMT
15 May 2020
Planet Maemo
Lecture on Augmented Reality
Due to the current circumstances, I had to record the lectures on augmented reality, which I am typically holding live. This was much more work than anticipated..
On the other hand, this means that I can make them available via Youtube now.
So, if you ever wanted to learn about the basic algorithms behind Augmented Reality, now is your chance.
The lecture is structured in two parts
- Camera Calibration, and
- Object Pose Estimation.
You can click on the TOC links below to directly jump to a specific topic.
Camera Calibration
Object Pose Estimation
15 May 2020 2:57pm GMT
24 Apr 2020
Planet Maemo
Error handling and exceptions
Yes, this is yet another post in the internet talking about using exceptions versus error returns. The topic has been flaming up at my workplace for quite some time now, and I felt that writing a blog post about it during the week-end would help me focus my thoughts and give me time to explain my point with the due care. In case you didn't know, I'm against using exceptions for error handling (maybe having spent many years working with Qt has had an effect on this); that does not mean that I never write code using exceptions: I certainly do my good share of try
... catch
when dealing with third-party code (including the STL), but you won't find a throw
in my programs.
I'm not going to write here all the reasons why I refrain myself from implementing error handling using exceptions; I'd rather like to focus on the one I consider to be the major one, and which I rarely see being given the due weight in the debate.
And please note that this post is about C++ only; it may be that exception handling in other languages is designed in such a way that all my concerns are addressed (either by the language itself, or by common error handling policies).
Code safety
I was about to title this "Code readability", but this is more about code verifiability, that is making sure that the code is correct and, ultimately, safe. As we all know, code is written once but read many times, and even if it's code you've written yourself, chances are that in a few weeks time you'll have forgotten several details about it; error cases and error handling are one typical thing that doesn't stick in our memory for long.
When I look at a small piece of code, such as the one that can fit into my screen, or which I can read from a merge request diff, I want to be able to ascertain that the code I'm looking at is correct. Let's look at some examples.
A throw
-free project
assert(track != nullptr); Car car; car.setMaximumSpeed(90); car.setName("Herbie"); if (!car.executeLap(track)) { log("Car failed to complete track"); return false; } Path *path = car.getPath(); if (!path) { log("GPX path could not be retrieved"); return false; } double temperature = car.engineTemperature(); double boundingRectArea = path->boundingRectArea();
I just made this up, so please bear with me if it doesn't make any sense. What I want to show is that code like the above has very few fault risks, if found in a project which bans throwing errors as exceptions: if we exclude out-of-memory errors, that are generally not handled to let the application crash (though you can always catch them if you like), the reader can easily verify that this code is safe. Coding style policies and naming conventions can guarantee that setMaximumSpeed()
and setName()
won't have a return value that needs to be checked, and all other method calls either return an error that our code is properly handling, or return some value. Of course, by just looking at this piece of code we cannot know if the engineTemperature()
method has some other overloaded sibling which accepts passing a reference to a boolean and which could be used to detect an error; so, it may be that our code could be improved in that respect, if we had a look at the header files for the Car
class - but this does deny the fact that a simple glance at this snippet tells us exactly what errors are handled and what could be going wrong.
Let's look at this code instead:
assert(track != nullptr); Car car; car.setMaximumSpeed(90); car.setName("Herbie"); car.executeLap(track); Path *path = car.getPath(); double temperature = car.engineTemperature(); double boundingRectArea = path->boundingRectArea();
If we continue on the assumption that we are working on a project which bans throwing exceptions, we can immediately say that this code is not safe: we don't know if the car successfully executed a lap on the track, and our process will crash if boundingRectArea()
is invoked on a null object.
Enter the exception
In a project where exceptions are actively used, the code from the second snippet is not obviously wrong anymore: maybe executeLap()
cannot throw any exceptions, or, if does, the caller of this snippet is catching the exception? In order to figure out whether this code is correct, I need to see the declaration of the executeLap()
method, and hope that there's a nice noexcept
in there; if there isn't, I have to look at its implementation, and recursively descend through all the methods it calls - at which point the safest attitude is just to assume that it can throw. But that's only half of the story, because once I accept the fact that executeLap()
can throw, I need to check whether the exception is properly handled: I have to check the implementation of all the callers of my method, and if I don't find a catch
there, I'll have to recursively walk up the tree of their callers.
And indeed even the first snippet, which looked so harmless when exception throwing was banned, suddenly becomes not obviously correct anymore: what if executeLap()
or getPath()
also throw an exception? You might say that it would be quite a silly thing to do, and I'd certainly agree; but it may be that indeed they don't throw any exceptions in their implementation, but some of the methods they call does.
A compromise: catch early, catch often
The obvious solution to the above issue is having a policy of handling exceptions right away, and explicitly rethrowing them (or even better, rethrow a different, more appropriate exception) up the stack:
assert(track != nullptr); Car car; car.setMaximumSpeed(90); car.setName("Herbie"); try { car.executeLap(track); Path *path = car.getPath(); double temperature = car.engineTemperature(); double boundingRectArea = path->boundingRectArea(); } catch (std::runtime_error &e) { log("Car failed to complete track"); throw; }
What I can tell from the above snippet is that the code is handling errors, and this is somehow a relief. I'm sure some of you would suggest using a more specific catch clause, but for the sake of this example let's assume that this one is fine.
(Quick note: the above example does not catch std::exception
, because that would also catch the std::bad_alloc
exception which is typically thrown in out-of-memory situations; my advice is not handle it at all, unless you know what you are doing)
In real life, though, you might find that try
-ing on a rather large block of operations is not enough: suppose that the Car methods all emit the same exception type, and that you need to handle them differently depending on when they occur. Then you'd need to split up the try
into smaller blocks, and at that point your code won't look any cleaner than the equivalent code which uses if
s on return values. Of course if you own the Car class you could modify it to throw different exceptions, in order to keep more operations inside the try
block and have specific catches at the end.
The big catch (pun intended)
Even once you've refactored your methods to get the best out of exceptions (where "best" is highly subjective, but let's assume that it just means that you are happy with your exception-throwing code), there's something that still bothers me, and that's exactly the same thing that proponents of exceptions use as a "pro" in their argumentations: the business logic of your code gets separated from the error handling. You get a nice block of pure logic, not cluttered with error checking, and a catch section (which I call "the big catch") where error cases are handled.
I really don't see how that makes the code any more readable or safe: sure, the logic is not intertwined with error handling and might help focus on the expected flow of the operations (though, really, I do not think that normal brains have a problem skipping over if
blocks), but that's hardly what I'm interested in when I want to check that the code is correct. Most of program errors and bugs lie in handling the edge cases and the abnormal situations, the seldomly taken code paths, and that's where I need to focus my attention.
try { operationA(); if (value > B.maxValue()) { operationB(); } else { operationC(); } operationD(); } catch (ExceptionI &e) { ... } catch (ExceptionII &e) { ... } catch (ExceptionIII &e) { ... } catch (std::runtime_error &e) { ... }
When I see code like this one, I need to mentally build a mapping of "operationX()
→ possible exceptions" (which, unless exception naming is making this obvious, requires me to look at the implementation of the operationX()
functions), and then mentally reconstruct the possible code paths in case operationX()
fails, for each line of the try
block.
Not seeing the errors right there, right away makes the correctness verification harder, which in turns means that the code becomes less safe. It will make you focus on the best case scenario, while ignoring all those annoying edge cases - too bad that 90% of the bugs are there.
Reading through the ISO C++ propaganda FAQ
I've been given a link to the C++ FAQ about exceptions, and unfortunately I read it. While there isn't much to argue on the technical side of it, it also carries some misleading statements, which might be true in absolute terms but don't let you see the big picture by not mentioning all that you need to know (which is the fundamental technique behind propaganda). An example is when they mention that eliminating ifs makes for more robust code, without mentioning that the same applies to all code branches, including exceptions.
Another argument that bothered me when I read it is the one about error propagation; this is the example they make:
void f1() { try { // ... f2(); // ... } catch (some_exception& e) { // ...code that handles the error... } } void f2() { ...; f3(); ...; } void f3() { ...; f4(); ...; } void f4() { ...; f5(); ...; } void f5() { ...; f6(); ...; } void f6() { ...; f7(); ...; } void f7() { ...; f8(); ...; } void f8() { ...; f9(); ...; } void f9() { ...; f10(); ...; } void f10() { // ... if ( /*...some error condition...*/ ) throw some_exception(); // ... }
The claim is that this code is more readable than the one with explicit error handling, because all the f2()
, f3
, …, f9()
functions don't have to handle the error occurring in f10()
. It is indeed a convincing argument, when presented in these terms, but is this really how our code looks like? In real life, you'll hardly have a chain of 1-liner functions, all defined next to each other in the same file. The moment that you realize that each one of these fn()
functions might be twenty or thirty lines long, and that they might be scattered over different files, and be called not just by fn-1()
but by any other function in the codebase, the picture does not look so rosy anymore: we get back to my main point of pain, that is that looking at the code of, say, f5()
, I will not be able to tell if the errors thrown by it, or by any of the methods invoked by it, are properly handled.
Exceptions in APIs
A side note about projects using exceptions. I'm not really bothered when a library I need to use is throwing exceptions: having to write
try { Foo::fetch("http://example.com/resource.txt"); } catch (Foo::Exception &) { return false; }
is not less readable or less safe than the code I'd write if Foo::fetch()
returned an error code. I still do have a little complaint, because the library author has given himself the right to decide that a failure in his library should be considered a critical fault, whereas it may be that in my program it is an expected failure and using exceptions imposes a penalty which could have been avoided. But I digress.
As long as the library documents which exceptions are thrown, it is used by many people (which hopefully means that it has few bugs) and it is a library that I don't need to contribute to, wrapping some of its methods in try
blocks is something I can live with.
One situation where I actually wish that libraries threw an exception is in out-of-memory situations; in that case, of course, I'd expect them to throw nothing else than std::bad_alloc
, which is the exception emitted by the standard library in such situations. That allows the caller to decide whether to ignore the exception and have the process terminated (which is what I usually do, at least in desktop applications) or try their luck and handle the failure - the latter is not easy, but it can certainly be done.
This is one case where error returns can be problematic, because it's likely that your code would look something like
if (!Foo::open(fileName)) { // suppose that this returns Error::OutOfMemory log("Failed to open " << fileName); return false; }
and in this case there's actually a risk that your code is going to trigger an out-of-memory error in logging the message; this shouldn't be a concern in most cases, but I can imagine some situations where one might want to know which was the exact operation that first incurred in the out-of-memory failure.
So, I'm actually fine with new
throwing. As for my code, my throw
statement is actually spelt as return
.
24 Apr 2020 3:07pm GMT
12 Apr 2020
Planet Maemo
New website for Mappero Geotagger, and cross-compiling stuff
Mappero Geotagger has now moved from its previous page from this site to a new, separate website built with the awesome Nikola static website generator.
The main reason for this change is that I didn't have an online space where to host the application binaries, and I wanted to experiment with a different selling method. Now, downloads are (poorly) hidden behind a payment page, whereas in multiple places of the website I also mention that I can provide the application for free to whomever asks for it. While it might seem weird at first, I do honestly believe that this will not stop people from buying it: first of all, many people just think it's fair to pay for a software applications, and secondly, for some people writing an e-mail and establishing a personal contact with a stranger is actually harder than paying a small amount of money. And in all sincerity, the majority of the income I've had so far for Mappero Geotagger came from donations, rather than purchases; so, not much to lose here.
QBS and MXE
Anyway, since this is primarily a technical blog, I want to share my experiences with cross-building from Linux to Windows. As you might remember, some time ago I switched the build system of Mappero from qmake to QBS, and I haven't regretted it at all. I've managed to build the application in Linux (of course), macOS, as a Debian package on the Ubuntu PPA builders, on Windows with AppVeyor and, last but not least, on Linux for Windows using the mingw setup provided by the MXE project.
QBS worked surprisingly well also in this case, though I had to fight with a small bug on the toolchain detection, which is hopefully going to be fixed soon. For the few of you who are interested in achieving something similar, here's the steps I ran to configure QBS for mingw:
MXE_BASE=<path-to-mxe> MXE_TARGET=x86_64-w64-mingw32.shared # 32 bit or static targets are also available MXE_PROFILE="mxe" QT_PROFILE="${MXE_PROFILE}-qt" qbs setup-toolchains "${MXE_BASE}/usr/bin/${MXE_TARGET}-g++" $MXE_PROFILE qbs config profiles.$MXE_PROFILE.cpp.toolchainPrefix "${MXE_TARGET}-" # temporary workaround qbs setup-qt "$MXE_BASE/usr/$MXE_TARGET/qt5/bin/qmake" ${QT_PROFILE} qbs config profiles.${QT_PROFILE}.baseProfile $MXE_PROFILE
Sorry for using that many environment variables ☺. After qbs is configured, it's just a matter of running
qbs profile:$QT_PROFILE
to build the application. You will get a nice window binary and, once you collect all the needed library dependencies, you'll be able to run it on Windows. Or WINE ☺.
As part of this effort, I also had to build libraw, so I didn't miss the occasion to contribute its recipe to MXE. I'm also trying to get a change accepted, that would make MXE support the dynamic OpenGL selection available since Qt 5.4.
12 Apr 2020 9:20am GMT
23 Mar 2020
Planet Maemo
Are we dead yet?
I am quite frustrated with corona graphs in the news, since most reporters seem to have skipped math classes back then. For instance, just plotting the number of confirmed infections at the respective dates does not tell you anything due to the different time point of outbreak. So lets see whether I can do better:
https://paroj.github.io/arewedeadyet/
With the site above, I tried to improve on a few things:
- the Charts are live: they update themselves each time you load the site.
- The curves are normalized by the time-point of outbreak so you can compare the course in different countries.
- You can select the countries that you want to compare.
- Different metrics are computed that allow comparing the corona countermeasures and impact across countries with different population size.
23 Mar 2020 11:40am GMT